2024-04-25T18:34:23.217 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T18:34:23.218 DEBUG:teuthology.run:Teuthology command: teuthology --name leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} --archive /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615 --owner scheduled_leonidus@teuthology -- /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615/orig.config.yaml 2024-04-25T18:34:23.278 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T18:34:23.642 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7673615' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 7797745e6d8b3ac036504dc61491d368615024da owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4419 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: 7797745e6d8b3ac036504dc61491d368615024da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7S4l/sU2Ih5/Z1zbLtNc/c+/Km+0K4wCMDSLjjssClO32DmV+u1IWbw4q8J5/0qsU0er7khgk0DL7KjL03R+0= smithi062.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCubIc3+aXTOhZVxWOTDsi/fLST+zy6LnXqTd4GJ1DCphaUE3Ulzud8Wx4tqN4hq5beYKND1Xw+I2cyMSZXXfPY= smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxJtgLGG5kFmEu3zIZmb+ohE03sGZPu8qUZeFNkYmMQJRQ2kXmeKEN1+52vZSiQPMxtEr4xwC5yC+H7gJ5u4do= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_17:43:55 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T18:34:23.642 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa; will attempt to use it 2024-04-25T18:34:23.643 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks 2024-04-25T18:34:23.643 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T18:34:23.645 INFO:teuthology.task.internal:Checking packages... 2024-04-25T18:34:23.668 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T18:34:23.668 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T18:34:23.668 INFO:teuthology.packaging:ref: None 2024-04-25T18:34:23.668 INFO:teuthology.packaging:tag: None 2024-04-25T18:34:23.668 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T18:34:23.668 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:34:23.669 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T18:34:23.834 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T18:34:23.835 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T18:34:23.842 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T18:34:23.843 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T18:34:23.849 INFO:teuthology.task.internal:Saving configuration 2024-04-25T18:34:23.870 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T18:34:23.877 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T18:34:23.900 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:29:13.721296', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7S4l/sU2Ih5/Z1zbLtNc/c+/Km+0K4wCMDSLjjssClO32DmV+u1IWbw4q8J5/0qsU0er7khgk0DL7KjL03R+0='} 2024-04-25T18:34:23.922 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:29:13.724195', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCubIc3+aXTOhZVxWOTDsi/fLST+zy6LnXqTd4GJ1DCphaUE3Ulzud8Wx4tqN4hq5beYKND1Xw+I2cyMSZXXfPY='} 2024-04-25T18:34:23.946 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:29:13.722861', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxJtgLGG5kFmEu3zIZmb+ohE03sGZPu8qUZeFNkYmMQJRQ2kXmeKEN1+52vZSiQPMxtEr4xwC5yC+H7gJ5u4do='} 2024-04-25T18:34:23.947 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T18:34:23.953 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T18:34:23.953 INFO:teuthology.task.internal:roles: ubuntu@smithi062.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T18:34:23.953 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T18:34:23.953 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T18:34:24.038 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6d4ffcc0d0>, signals=[15]) 2024-04-25T18:34:24.038 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T18:34:24.044 INFO:teuthology.task.internal:Opening connections... 2024-04-25T18:34:24.045 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T18:34:24.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:34:24.125 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T18:34:24.126 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:34:24.200 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-04-25T18:34:24.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:34:24.274 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T18:34:24.279 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-04-25T18:34:24.298 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-04-25T18:34:24.298 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-04-25T18:34:24.355 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-04-25T18:34:24.355 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-04-25T18:34:24.355 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-04-25T18:34:24.355 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:34:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-04-25T18:34:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:34:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:34:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-04-25T18:34:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:34:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:34:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:34:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:34:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:34:24.356 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:34:24.357 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-04-25T18:34:24.383 DEBUG:teuthology.orchestra.run.smithi062:> uname -m 2024-04-25T18:34:24.401 INFO:teuthology.orchestra.run.smithi062.stdout:x86_64 2024-04-25T18:34:24.401 DEBUG:teuthology.orchestra.run.smithi062:> cat /etc/os-release 2024-04-25T18:34:24.459 INFO:teuthology.orchestra.run.smithi062.stdout:NAME="CentOS Stream" 2024-04-25T18:34:24.459 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION="9" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:ID="centos" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:VERSION_ID="9" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:ANSI_COLOR="0;31" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:34:24.460 INFO:teuthology.orchestra.run.smithi062.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:34:24.461 INFO:teuthology.lock.ops:Updating smithi062.front.sepia.ceph.com on lock server 2024-04-25T18:34:24.482 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-04-25T18:34:24.500 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-04-25T18:34:24.500 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T18:34:24.557 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2024-04-25T18:34:24.558 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T18:34:24.558 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T18:34:24.558 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T18:34:24.558 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-04-25T18:34:24.582 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T18:34:24.593 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T18:34:24.598 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T18:34:24.598 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:34:24.601 DEBUG:teuthology.orchestra.run.smithi062:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:34:24.603 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-04-25T18:34:24.619 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T18:34:24.625 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T18:34:24.625 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:34:24.657 DEBUG:teuthology.orchestra.run.smithi062:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:34:24.660 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-04-25T18:34:24.754 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T18:34:24.825 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T18:34:24.839 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T18:34:24.839 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-25T18:34:24.839 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-25T18:34:24.840 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:34:24.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:34:24.857 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T18:34:25.113 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:34:25.113 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T18:34:25.113 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-04-25T18:34:25.131 INFO:teuthology.orchestra.run.smithi045.stdout:5.14.0-437.el9.x86_64 2024-04-25T18:34:25.131 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:34:25.131 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T18:34:25.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum upgrade -y linux-firmware 2024-04-25T18:34:27.509 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T18:34:29.585 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-25T18:34:33.827 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T18:34:35.375 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-04-25T18:34:37.968 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-04-25T18:34:43.251 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T18:34:45.310 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-25T18:34:45.311 INFO:teuthology.orchestra.run.smithi045.stdout:Nothing to do. 2024-04-25T18:34:45.311 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-25T18:34:45.427 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T18:34:45.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T18:34:45.571 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T18:34:45.572 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T18:34:45.574 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi045.front.sepia.ceph.com to complete... 2024-04-25T18:34:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25 18:34:57 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T18:34:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:FINISHED --2024-04-25 18:34:57-- 2024-04-25T18:34:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:Total wall clock time: 12s 2024-04-25T18:34:57.707 INFO:teuthology.orchestra.run.smithi045.stderr:Downloaded: 1 files, 341M in 12s (28.8 MB/s) 2024-04-25T18:34:57.710 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T18:34:57.710 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T18:34:59.945 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying... ######################################## 2024-04-25T18:35:00.048 INFO:teuthology.orchestra.run.smithi045.stdout:Preparing... ######################################## 2024-04-25T18:35:00.152 INFO:teuthology.orchestra.run.smithi045.stdout:Updating / installing... 2024-04-25T18:35:08.541 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T18:35:48.751 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi045.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T18:35:48.751 DEBUG:teuthology.orchestra.run.smithi045:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T18:35:48.827 INFO:teuthology.orchestra.run.smithi045.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T18:35:48.827 INFO:teuthology.orchestra.run.smithi045.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T18:35:48.827 INFO:teuthology.orchestra.run.smithi045.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T18:35:48.828 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T18:35:48.829 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T18:35:48.830 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T18:35:48.831 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T18:35:48.832 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T18:35:48.833 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T18:35:48.834 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T18:35:48.835 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T18:35:48.836 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T18:35:48.837 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T18:35:48.838 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T18:35:48.839 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T18:35:48.840 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T18:35:48.841 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T18:35:48.842 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T18:35:48.843 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T18:35:48.844 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T18:35:48.845 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T18:35:48.846 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T18:35:48.847 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T18:35:48.848 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T18:35:48.849 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T18:35:48.850 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T18:35:48.851 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T18:35:48.852 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T18:35:48.853 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T18:35:48.854 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T18:35:48.855 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T18:35:48.856 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T18:35:48.857 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T18:35:48.858 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T18:35:48.859 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T18:35:48.860 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T18:35:48.861 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T18:35:48.862 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T18:35:48.863 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T18:35:48.864 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T18:35:48.865 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T18:35:48.866 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T18:35:48.867 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T18:35:48.868 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T18:35:48.869 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T18:35:48.870 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T18:35:48.871 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T18:35:48.872 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T18:35:48.873 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T18:35:48.874 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T18:35:48.875 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T18:35:48.876 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T18:35:48.877 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T18:35:48.878 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T18:35:48.879 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T18:35:48.880 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T18:35:48.881 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T18:35:48.882 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T18:35:48.883 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T18:35:48.884 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T18:35:48.885 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T18:35:48.886 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T18:35:48.887 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T18:35:48.888 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T18:35:48.889 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T18:35:48.890 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T18:35:48.891 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T18:35:48.892 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T18:35:48.893 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T18:35:48.894 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T18:35:48.895 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T18:35:48.896 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T18:35:48.897 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T18:35:48.898 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T18:35:48.899 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T18:35:48.900 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T18:35:48.901 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T18:35:48.902 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T18:35:48.903 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T18:35:48.904 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T18:35:48.905 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T18:35:48.906 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T18:35:48.907 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T18:35:48.908 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T18:35:48.909 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T18:35:48.910 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T18:35:48.911 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T18:35:48.912 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T18:35:48.913 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T18:35:48.914 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T18:35:48.915 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T18:35:48.916 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T18:35:48.917 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T18:35:48.918 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T18:35:48.919 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T18:35:48.920 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T18:35:48.921 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T18:35:48.922 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T18:35:48.923 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T18:35:48.924 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T18:35:48.925 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T18:35:48.926 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T18:35:48.927 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T18:35:48.928 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T18:35:48.929 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T18:35:48.930 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T18:35:48.931 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T18:35:48.932 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T18:35:48.933 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T18:35:48.934 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T18:35:48.935 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T18:35:48.936 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T18:35:48.937 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T18:35:48.938 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T18:35:48.939 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T18:35:48.940 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T18:35:48.941 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T18:35:48.942 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T18:35:48.943 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T18:35:48.944 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T18:35:48.945 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T18:35:48.946 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T18:35:48.947 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T18:35:48.948 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T18:35:48.949 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T18:35:48.950 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T18:35:48.951 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T18:35:48.952 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T18:35:48.953 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T18:35:48.954 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T18:35:48.955 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T18:35:48.956 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T18:35:48.956 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T18:35:48.956 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T18:35:48.956 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T18:35:48.956 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T18:35:48.956 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T18:35:48.956 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T18:35:48.956 INFO:teuthology.orchestra.run.smithi045.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T18:35:48.960 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T18:35:48.960 DEBUG:teuthology.orchestra.run.smithi045:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T18:35:48.986 INFO:teuthology.orchestra.run.smithi045.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T18:35:48.986 INFO:teuthology.orchestra.run.smithi045.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T18:35:48.986 INFO:teuthology.orchestra.run.smithi045.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout:for file in vmlinuz System.map config; do 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout: fi 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout:done 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2024-04-25T18:35:48.987 INFO:teuthology.orchestra.run.smithi045.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T18:35:48.988 INFO:teuthology.orchestra.run.smithi045.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T18:35:48.988 INFO:teuthology.orchestra.run.smithi045.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T18:35:48.988 INFO:teuthology.orchestra.run.smithi045.stdout:fi 2024-04-25T18:35:48.988 DEBUG:teuthology.orchestra.run.smithi045:> sudo rpm -qi grub2-tools 2024-04-25T18:35:49.076 INFO:teuthology.orchestra.run.smithi045.stdout:Name : grub2-tools 2024-04-25T18:35:49.076 INFO:teuthology.orchestra.run.smithi045.stdout:Epoch : 1 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Version : 2.06 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Release : 68.el9 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Architecture: x86_64 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Group : Unspecified 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Size : 8264254 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:License : GPLv3+ 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Packager : builder@centos.org 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:Vendor : CentOS 2024-04-25T18:35:49.077 INFO:teuthology.orchestra.run.smithi045.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T18:35:49.078 INFO:teuthology.orchestra.run.smithi045.stdout:Summary : Support tools for GRUB. 2024-04-25T18:35:49.078 INFO:teuthology.orchestra.run.smithi045.stdout:Description : 2024-04-25T18:35:49.078 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:35:49.078 INFO:teuthology.orchestra.run.smithi045.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T18:35:49.078 INFO:teuthology.orchestra.run.smithi045.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T18:35:49.078 INFO:teuthology.orchestra.run.smithi045.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T18:35:49.078 INFO:teuthology.orchestra.run.smithi045.stdout:hardware devices. 2024-04-25T18:35:49.078 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:35:49.078 INFO:teuthology.orchestra.run.smithi045.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T18:35:49.079 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T18:35:49.079 INFO:teuthology.task.kernel:Updating grub on smithi045 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T18:35:49.079 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T18:35:49.599 INFO:teuthology.orchestra.run.smithi045.stderr:Generating grub configuration file ... 2024-04-25T18:35:51.031 INFO:teuthology.orchestra.run.smithi045.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T18:35:51.061 INFO:teuthology.orchestra.run.smithi045.stderr:done 2024-04-25T18:35:51.063 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-04-25T18:35:51.090 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.76qPgEg1Ln 2024-04-25T18:35:51.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.76qPgEg1Ln 2024-04-25T18:35:51.156 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0666 /tmp/tmp.76qPgEg1Ln 2024-04-25T18:35:51.307 DEBUG:teuthology.orchestra.remote:smithi045:/tmp/tmp.76qPgEg1Ln is 6KB 2024-04-25T18:35:51.356 DEBUG:teuthology.orchestra.run.smithi045:> rm -fr /tmp/tmp.76qPgEg1Ln 2024-04-25T18:35:51.371 DEBUG:teuthology.orchestra.run.smithi045:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T18:35:51.433 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T18:35:51.433 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T18:35:51.433 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T18:35:51.433 INFO:teuthology.orchestra.run.smithi045.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T18:35:51.435 DEBUG:teuthology.orchestra.run.smithi045:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T18:35:51.583 DEBUG:teuthology.orchestra.run.smithi045:> sudo shutdown -r now 2024-04-25T18:36:21.586 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T18:36:21.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T18:36:21.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:36:40.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-25T18:36:49.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T18:36:49.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:36:52.158 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.45 2024-04-25T18:37:04.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T18:37:04.160 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:37:35.960 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-25T18:37:36.561 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T18:37:36.561 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T18:37:36.562 DEBUG:teuthology.orchestra.run.smithi045:> uname -r 2024-04-25T18:37:36.598 INFO:teuthology.orchestra.run.smithi045.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T18:37:36.598 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T18:37:36.599 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T18:37:36.599 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T18:37:37.599 DEBUG:teuthology.parallel:result is None 2024-04-25T18:37:37.600 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T18:37:37.608 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T18:37:37.608 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:37:37.616 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:37:37.618 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T18:37:37.663 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T18:37:37.669 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T18:37:37.684 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T18:37:37.684 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:37:37.690 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:37:37.693 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T18:37:37.757 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T18:37:37.767 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T18:37:37.768 DEBUG:teuthology.orchestra.run.smithi062:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:37:37.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:37:37.785 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T18:37:37.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:37:37.801 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:37:37.806 DEBUG:teuthology.orchestra.run.smithi062:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:37:37.828 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T18:37:37.858 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:37:37.891 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:37:37.893 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:37:37.923 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:37:37.935 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:37:37.978 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T18:37:37.982 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T18:37:37.991 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T18:37:37.991 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:37:37.995 DEBUG:teuthology.orchestra.run.smithi062:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:37:37.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T18:37:38.058 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T18:37:38.068 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T18:37:38.069 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:37:38.073 DEBUG:teuthology.orchestra.run.smithi062:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:37:38.075 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T18:37:38.114 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:37:38.228 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:37:38.299 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:37:38.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:37:38.463 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:37:38.463 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:37:38.563 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:37:38.609 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:37:38.660 DEBUG:teuthology.orchestra.run.smithi062:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:37:38.745 DEBUG:teuthology.orchestra.run.smithi062:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:37:38.788 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:37:38.788 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:37:38.854 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:37:38.897 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T18:37:38.950 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:37:39.032 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T18:37:39.075 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:37:39.075 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T18:37:39.141 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-04-25T18:37:39.148 DEBUG:teuthology.orchestra.run.smithi062:> sudo service rsyslog restart 2024-04-25T18:37:39.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-04-25T18:37:39.186 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:37:39.223 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:37:39.243 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T18:37:39.787 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T18:37:39.795 INFO:teuthology.task.internal:Starting timer... 2024-04-25T18:37:39.795 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T18:37:39.823 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T18:37:39.837 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T18:37:39.837 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-04-25T18:37:39.948 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-04-25T18:37:39.952 DEBUG:teuthology.orchestra.run.smithi062:> sudo service auditd rotate 2024-04-25T18:37:40.016 INFO:teuthology.orchestra.run.smithi062.stdout:Rotating logs: 2024-04-25T18:37:40.018 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2024-04-25T18:37:40.094 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2024-04-25T18:37:40.095 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T18:37:40.095 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-04-25T18:37:40.153 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-04-25T18:37:40.154 DEBUG:teuthology.orchestra.run.smithi062:> /usr/sbin/getenforce 2024-04-25T18:37:40.176 INFO:teuthology.orchestra.run.smithi062.stdout:Permissive 2024-04-25T18:37:40.176 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2024-04-25T18:37:40.200 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2024-04-25T18:37:40.200 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi062.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi079.front.sepia.ceph.com': 'permissive'} 2024-04-25T18:37:40.200 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:37:40.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:37:40.267 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:37:40.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:37:40.294 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T18:37:40.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:37:40.324 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T18:37:40.324 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:37:40.435 DEBUG:teuthology.orchestra.run.smithi062:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:37:40.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2024-04-25T18:37:40.495 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T18:37:40.504 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T18:37:40.520 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T18:37:40.521 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi045.front.sepia.ceph.com,smithi062.front.sepia.ceph.com,smithi079.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T18:42:57.012 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi062.front.sepia.ceph.com'), Remote(name='ubuntu@smithi079.front.sepia.ceph.com')] 2024-04-25T18:42:57.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T18:42:57.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:42:57.130 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-25T18:42:57.272 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-25T18:42:57.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T18:42:57.273 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi062.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:42:57.348 DEBUG:teuthology.orchestra.run.smithi062:> true 2024-04-25T18:42:57.427 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi062.front.sepia.ceph.com' 2024-04-25T18:42:57.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-04-25T18:42:57.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T18:42:57.504 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-04-25T18:42:57.584 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-04-25T18:42:57.584 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T18:42:57.598 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T18:42:57.598 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:42:57.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:42:57.606 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:42:57.606 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:42:57.609 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T18:42:57.609 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T18:42:57.639 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:42:57.654 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:42:57.656 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:42:57.670 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:42:57.678 INFO:teuthology.orchestra.run.smithi062.stderr:sudo: ntpd: command not found 2024-04-25T18:42:57.689 INFO:teuthology.orchestra.run.smithi062.stdout:506 Cannot talk to daemon 2024-04-25T18:42:57.695 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2024-04-25T18:42:57.697 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T18:42:57.703 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:42:57.706 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2024-04-25T18:42:57.717 INFO:teuthology.orchestra.run.smithi062.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:42:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:42:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:42:57.739 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T18:42:57.773 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:42:57.776 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:42:57.776 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-25T18:42:57.776 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:57.776 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:57.776 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:57.776 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:57.798 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:42:57.800 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:42:57.800 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-04-25T18:42:57.800 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:57.800 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:57.800 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:57.800 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:57.820 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-04-25T18:42:57.852 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-04-25T18:42:57.895 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T18:42:57.939 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T18:42:58.092 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-04-25T18:42:58.099 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T18:42:58.099 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-04-25T18:42:58.099 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:58.099 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:58.099 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:58.099 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T18:42:58.101 INFO:teuthology.run_tasks:Running task install... 2024-04-25T18:42:58.110 DEBUG:teuthology.task.install:project ceph 2024-04-25T18:42:58.110 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T18:42:58.110 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T18:42:58.110 INFO:teuthology.task.install:Using flavor: default 2024-04-25T18:42:58.116 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T18:42:58.116 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T18:42:58.116 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T18:42:58.117 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:42:58.118 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T18:42:58.119 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:42:58.119 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T18:42:58.120 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:42:58.268 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T18:42:58.268 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T18:42:58.334 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T18:42:58.335 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T18:42:58.338 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T18:42:58.338 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T18:42:58.438 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:42:58.438 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:42:58.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:42:58.445 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:42:58.445 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:42:58.445 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:42:58.464 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:42:58.465 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:42:58.472 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:42:58.473 DEBUG:teuthology.orchestra.run.smithi062:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:42:58.505 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T18:42:58.505 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:42:58.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T18:42:58.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:42:58.540 DEBUG:teuthology.orchestra.run.smithi062:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:42:58.600 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T18:42:58.600 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T18:42:58.607 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:42:58.616 DEBUG:teuthology.orchestra.run.smithi062:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:42:58.670 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2024-04-25T18:42:58.672 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2024-04-25T18:42:58.680 INFO:teuthology.orchestra.run.smithi062.stdout:check_obsoletes = 1 2024-04-25T18:42:58.682 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum clean all 2024-04-25T18:42:58.735 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T18:42:58.851 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T18:42:58.896 INFO:teuthology.orchestra.run.smithi079.stdout:48 files removed 2024-04-25T18:42:58.917 INFO:teuthology.orchestra.run.smithi062.stdout:48 files removed 2024-04-25T18:42:58.922 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:42:58.933 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-04-25T18:42:58.937 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-04-25T18:42:58.943 DEBUG:teuthology.orchestra.run.smithi062:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:42:59.283 INFO:teuthology.orchestra.run.smithi045.stdout:48 files removed 2024-04-25T18:42:59.323 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T18:42:59.571 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 213 kB/s | 83 kB 00:00 2024-04-25T18:42:59.630 INFO:teuthology.orchestra.run.smithi062.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-04-25T18:42:59.880 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 44 kB/s | 12 kB 00:00 2024-04-25T18:42:59.989 INFO:teuthology.orchestra.run.smithi062.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-25T18:43:00.176 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 202 kB/s | 83 kB 00:00 2024-04-25T18:43:00.306 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-25T18:43:00.398 INFO:teuthology.orchestra.run.smithi062.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-25T18:43:00.768 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-25T18:43:00.967 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T18:43:01.075 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-04-25T18:43:01.285 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-25T18:43:02.038 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-25T18:43:02.710 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-25T18:43:03.126 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-25T18:43:04.190 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T18:43:06.743 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T18:43:07.411 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T18:43:08.308 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-04-25T18:43:08.866 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.2 MB 00:00 2024-04-25T18:43:09.059 INFO:teuthology.orchestra.run.smithi062.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-25T18:43:10.577 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 16 kB 00:00 2024-04-25T18:43:10.762 INFO:teuthology.orchestra.run.smithi062.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T18:43:12.435 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T18:43:15.985 INFO:teuthology.orchestra.run.smithi062.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T18:43:16.366 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 2.6 MB/s | 21 MB 00:07 2024-04-25T18:43:18.195 INFO:teuthology.orchestra.run.smithi062.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:43:18.195 INFO:teuthology.orchestra.run.smithi062.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:43:18.207 INFO:teuthology.orchestra.run.smithi062.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:43:18.207 INFO:teuthology.orchestra.run.smithi062.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:43:18.208 INFO:teuthology.orchestra.run.smithi062.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:43:18.209 INFO:teuthology.orchestra.run.smithi062.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:43:18.211 INFO:teuthology.orchestra.run.smithi062.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:43:18.213 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:43:18.214 INFO:teuthology.orchestra.run.smithi062.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:43:18.278 INFO:teuthology.orchestra.run.smithi062.stdout:Dependencies resolved. 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: Package Arch Version Repository Size 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout:Installing: 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T18:43:18.293 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:43:18.294 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T18:43:18.295 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrading: 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout:Installing dependencies: 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T18:43:18.296 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:43:18.297 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:43:18.298 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T18:43:18.299 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:43:18.300 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:43:18.301 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:43:18.302 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:43:18.303 INFO:teuthology.orchestra.run.smithi062.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout:Installing weak dependencies: 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction Summary 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout:======================================================================================= 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout:Install 148 Packages 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout:Upgrade 4 Packages 2024-04-25T18:43:18.304 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:43:18.305 INFO:teuthology.orchestra.run.smithi062.stdout:Total download size: 222 M 2024-04-25T18:43:18.305 INFO:teuthology.orchestra.run.smithi062.stdout:Downloading Packages: 2024-04-25T18:43:18.578 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-25T18:43:18.953 INFO:teuthology.orchestra.run.smithi062.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-25T18:43:19.312 INFO:teuthology.orchestra.run.smithi062.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.4 MB/s | 890 kB 00:00 2024-04-25T18:43:19.395 INFO:teuthology.orchestra.run.smithi062.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.7 MB/s | 145 kB 00:00 2024-04-25T18:43:19.912 INFO:teuthology.orchestra.run.smithi062.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 4.6 MB/s | 5.4 MB 00:01 2024-04-25T18:43:19.963 INFO:teuthology.orchestra.run.smithi062.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 4.2 MB/s | 2.4 MB 00:00 2024-04-25T18:43:20.179 INFO:teuthology.orchestra.run.smithi062.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 5.6 MB/s | 1.5 MB 00:00 2024-04-25T18:43:20.405 INFO:teuthology.orchestra.run.smithi062.stdout:(7/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 13 MB/s | 21 MB 00:01 2024-04-25T18:43:20.805 INFO:teuthology.orchestra.run.smithi062.stdout:(8/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 5.6 MB/s | 4.7 MB 00:00 2024-04-25T18:43:20.905 INFO:teuthology.orchestra.run.smithi062.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 251 kB/s | 25 kB 00:00 2024-04-25T18:43:21.055 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:43:21.057 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:43:21.079 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:43:21.080 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:43:21.081 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:43:21.083 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:43:21.085 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:43:21.089 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:43:21.090 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:43:21.139 INFO:teuthology.orchestra.run.smithi062.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 14 MB/s | 11 MB 00:00 2024-04-25T18:43:21.166 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-25T18:43:21.185 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T18:43:21.186 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T18:43:21.187 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T18:43:21.188 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T18:43:21.189 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T18:43:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T18:43:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T18:43:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T18:43:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T18:43:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T18:43:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:43:21.190 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:43:21.191 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:43:21.192 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:43:21.193 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:43:21.194 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:43:21.195 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:43:21.196 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:43:21.197 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:43:21.198 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-04-25T18:43:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:43:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:43:21.198 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-25T18:43:21.198 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-25T18:43:21.198 INFO:teuthology.orchestra.run.smithi045.stdout:Install 148 Packages 2024-04-25T18:43:21.198 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 4 Packages 2024-04-25T18:43:21.198 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:43:21.199 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 222 M 2024-04-25T18:43:21.199 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-25T18:43:21.214 INFO:teuthology.orchestra.run.smithi062.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.9 MB/s | 225 kB 00:00 2024-04-25T18:43:21.273 INFO:teuthology.orchestra.run.smithi062.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 465 kB/s | 27 kB 00:00 2024-04-25T18:43:21.323 INFO:teuthology.orchestra.run.smithi062.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 634 kB/s | 32 kB 00:00 2024-04-25T18:43:21.390 INFO:teuthology.orchestra.run.smithi062.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 11 MB/s | 747 kB 00:00 2024-04-25T18:43:21.424 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-25T18:43:21.448 INFO:teuthology.orchestra.run.smithi062.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 163 kB 00:00 2024-04-25T18:43:21.516 INFO:teuthology.orchestra.run.smithi062.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.8 MB/s | 126 kB 00:00 2024-04-25T18:43:21.608 INFO:teuthology.orchestra.run.smithi062.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.3 MB/s | 500 kB 00:00 2024-04-25T18:43:22.133 INFO:teuthology.orchestra.run.smithi062.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 10 MB/s | 5.4 MB 00:00 2024-04-25T18:43:22.200 INFO:teuthology.orchestra.run.smithi062.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 679 kB/s | 45 kB 00:00 2024-04-25T18:43:22.258 INFO:teuthology.orchestra.run.smithi062.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 2.3 MB/s | 135 kB 00:00 2024-04-25T18:43:22.325 INFO:teuthology.orchestra.run.smithi062.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.4 MB/s | 161 kB 00:00 2024-04-25T18:43:22.397 INFO:teuthology.orchestra.run.smithi045.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-25T18:43:22.526 INFO:teuthology.orchestra.run.smithi062.stdout:(22/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 7.2 MB/s | 17 MB 00:02 2024-04-25T18:43:22.551 INFO:teuthology.orchestra.run.smithi062.stdout:(23/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.4 MB/s | 320 kB 00:00 2024-04-25T18:43:22.626 INFO:teuthology.orchestra.run.smithi062.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 298 kB 00:00 2024-04-25T18:43:22.651 INFO:teuthology.orchestra.run.smithi062.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 992 kB/s | 99 kB 00:00 2024-04-25T18:43:22.710 INFO:teuthology.orchestra.run.smithi062.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T18:43:22.852 INFO:teuthology.orchestra.run.smithi062.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.2 MB/s | 170 kB 00:00 2024-04-25T18:43:22.930 INFO:teuthology.orchestra.run.smithi045.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.6 MB/s | 890 kB 00:00 2024-04-25T18:43:22.944 INFO:teuthology.orchestra.run.smithi062.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 373 kB/s | 34 kB 00:00 2024-04-25T18:43:23.044 INFO:teuthology.orchestra.run.smithi062.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.4 MB/s | 142 kB 00:00 2024-04-25T18:43:23.080 INFO:teuthology.orchestra.run.smithi045.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 971 kB/s | 145 kB 00:00 2024-04-25T18:43:23.094 INFO:teuthology.orchestra.run.smithi062.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 7.0 MB/s | 3.1 MB 00:00 2024-04-25T18:43:23.584 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:43:23.584 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T18:43:23.596 INFO:teuthology.orchestra.run.smithi079.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T18:43:23.596 INFO:teuthology.orchestra.run.smithi079.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T18:43:23.597 INFO:teuthology.orchestra.run.smithi079.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T18:43:23.598 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T18:43:23.600 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T18:43:23.602 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T18:43:23.603 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T18:43:23.663 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-04-25T18:43:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-04-25T18:43:23.681 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-04-25T18:43:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-04-25T18:43:23.681 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-04-25T18:43:23.681 INFO:teuthology.orchestra.run.smithi079.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T18:43:23.681 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T18:43:23.681 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:43:23.681 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T18:43:23.681 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T18:43:23.681 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T18:43:23.682 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T18:43:23.683 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T18:43:23.684 INFO:teuthology.orchestra.run.smithi079.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T18:43:23.685 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T18:43:23.686 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T18:43:23.687 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T18:43:23.688 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T18:43:23.689 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T18:43:23.690 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T18:43:23.691 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T18:43:23.692 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:43:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-04-25T18:43:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-04-25T18:43:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:Install 148 Packages 2024-04-25T18:43:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 4 Packages 2024-04-25T18:43:23.693 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:43:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 222 M 2024-04-25T18:43:23.693 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-04-25T18:43:23.787 INFO:teuthology.orchestra.run.smithi062.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.8 MB/s | 3.5 MB 00:00 2024-04-25T18:43:23.903 INFO:teuthology.orchestra.run.smithi062.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.1 MB/s | 249 kB 00:00 2024-04-25T18:43:24.003 INFO:teuthology.orchestra.run.smithi062.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 493 kB/s | 49 kB 00:00 2024-04-25T18:43:24.112 INFO:teuthology.orchestra.run.smithi062.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 152 kB/s | 16 kB 00:00 2024-04-25T18:43:24.197 INFO:teuthology.orchestra.run.smithi045.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.6 MB/s | 5.4 MB 00:02 2024-04-25T18:43:24.221 INFO:teuthology.orchestra.run.smithi062.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.5 MB/s | 273 kB 00:00 2024-04-25T18:43:24.396 INFO:teuthology.orchestra.run.smithi062.stdout:(36/152): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-04-25T18:43:24.422 INFO:teuthology.orchestra.run.smithi045.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.8 MB/s | 2.4 MB 00:01 2024-04-25T18:43:24.446 INFO:teuthology.orchestra.run.smithi062.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 366 kB/s | 18 kB 00:00 2024-04-25T18:43:24.492 INFO:teuthology.orchestra.run.smithi079.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 35 kB/s | 6.6 kB 00:00 2024-04-25T18:43:24.581 INFO:teuthology.orchestra.run.smithi045.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.9 MB/s | 1.5 MB 00:00 2024-04-25T18:43:24.638 INFO:teuthology.orchestra.run.smithi062.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 419 kB/s | 80 kB 00:00 2024-04-25T18:43:24.759 INFO:teuthology.orchestra.run.smithi079.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 3.3 MB/s | 890 kB 00:00 2024-04-25T18:43:24.763 INFO:teuthology.orchestra.run.smithi062.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-04-25T18:43:24.797 INFO:teuthology.orchestra.run.smithi062.stdout:(40/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 1.3 MB/s | 762 kB 00:00 2024-04-25T18:43:24.822 INFO:teuthology.orchestra.run.smithi062.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-25T18:43:24.826 INFO:teuthology.orchestra.run.smithi079.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-25T18:43:24.848 INFO:teuthology.orchestra.run.smithi062.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-25T18:43:24.898 INFO:teuthology.orchestra.run.smithi062.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-25T18:43:24.923 INFO:teuthology.orchestra.run.smithi062.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-25T18:43:24.948 INFO:teuthology.orchestra.run.smithi062.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-25T18:43:24.974 INFO:teuthology.orchestra.run.smithi062.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T18:43:24.999 INFO:teuthology.orchestra.run.smithi062.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T18:43:25.024 INFO:teuthology.orchestra.run.smithi062.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T18:43:25.066 INFO:teuthology.orchestra.run.smithi062.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T18:43:25.092 INFO:teuthology.orchestra.run.smithi062.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T18:43:25.117 INFO:teuthology.orchestra.run.smithi062.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-25T18:43:25.142 INFO:teuthology.orchestra.run.smithi062.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T18:43:25.167 INFO:teuthology.orchestra.run.smithi062.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 662 kB/s | 33 kB 00:00 2024-04-25T18:43:25.193 INFO:teuthology.orchestra.run.smithi079.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 6.4 MB/s | 2.4 MB 00:00 2024-04-25T18:43:25.243 INFO:teuthology.orchestra.run.smithi062.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 30 MB/s | 3.0 MB 00:00 2024-04-25T18:43:25.268 INFO:teuthology.orchestra.run.smithi062.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 186 kB/s | 18 kB 00:00 2024-04-25T18:43:25.293 INFO:teuthology.orchestra.run.smithi062.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 224 kB/s | 11 kB 00:00 2024-04-25T18:43:25.318 INFO:teuthology.orchestra.run.smithi062.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T18:43:25.344 INFO:teuthology.orchestra.run.smithi062.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-04-25T18:43:25.369 INFO:teuthology.orchestra.run.smithi062.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T18:43:25.411 INFO:teuthology.orchestra.run.smithi062.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.7 MB/s | 582 kB 00:00 2024-04-25T18:43:25.418 INFO:teuthology.orchestra.run.smithi079.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 6.7 MB/s | 1.5 MB 00:00 2024-04-25T18:43:25.436 INFO:teuthology.orchestra.run.smithi062.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 387 kB/s | 25 kB 00:00 2024-04-25T18:43:25.462 INFO:teuthology.orchestra.run.smithi062.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-04-25T18:43:25.487 INFO:teuthology.orchestra.run.smithi062.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T18:43:25.512 INFO:teuthology.orchestra.run.smithi062.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T18:43:25.538 INFO:teuthology.orchestra.run.smithi062.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 711 kB/s | 35 kB 00:00 2024-04-25T18:43:25.638 INFO:teuthology.orchestra.run.smithi062.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-04-25T18:43:25.721 INFO:teuthology.orchestra.run.smithi062.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-04-25T18:43:25.765 INFO:teuthology.orchestra.run.smithi062.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T18:43:25.807 INFO:teuthology.orchestra.run.smithi062.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T18:43:25.832 INFO:teuthology.orchestra.run.smithi062.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T18:43:25.926 INFO:teuthology.orchestra.run.smithi079.stdout:(6/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 9.2 MB/s | 4.7 MB 00:00 2024-04-25T18:43:25.991 INFO:teuthology.orchestra.run.smithi062.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 32 MB/s | 5.8 MB 00:00 2024-04-25T18:43:26.006 INFO:teuthology.orchestra.run.smithi045.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.0 MB/s | 4.7 MB 00:01 2024-04-25T18:43:26.025 INFO:teuthology.orchestra.run.smithi062.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-04-25T18:43:26.050 INFO:teuthology.orchestra.run.smithi062.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T18:43:26.192 INFO:teuthology.orchestra.run.smithi062.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-04-25T18:43:26.217 INFO:teuthology.orchestra.run.smithi062.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-04-25T18:43:26.243 INFO:teuthology.orchestra.run.smithi062.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T18:43:26.268 INFO:teuthology.orchestra.run.smithi062.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-04-25T18:43:26.293 INFO:teuthology.orchestra.run.smithi062.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-25T18:43:26.318 INFO:teuthology.orchestra.run.smithi062.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T18:43:26.344 INFO:teuthology.orchestra.run.smithi062.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-04-25T18:43:26.461 INFO:teuthology.orchestra.run.smithi062.stdout:(81/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-04-25T18:43:26.486 INFO:teuthology.orchestra.run.smithi062.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.8 MB/s | 250 kB 00:00 2024-04-25T18:43:26.511 INFO:teuthology.orchestra.run.smithi062.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-25T18:43:26.561 INFO:teuthology.orchestra.run.smithi062.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-25T18:43:26.587 INFO:teuthology.orchestra.run.smithi062.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-04-25T18:43:26.612 INFO:teuthology.orchestra.run.smithi062.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T18:43:26.637 INFO:teuthology.orchestra.run.smithi062.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T18:43:26.662 INFO:teuthology.orchestra.run.smithi062.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T18:43:26.688 INFO:teuthology.orchestra.run.smithi062.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T18:43:26.713 INFO:teuthology.orchestra.run.smithi062.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T18:43:26.888 INFO:teuthology.orchestra.run.smithi062.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 241 kB/s | 42 kB 00:00 2024-04-25T18:43:27.055 INFO:teuthology.orchestra.run.smithi062.stdout:(92/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-04-25T18:43:27.080 INFO:teuthology.orchestra.run.smithi062.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 233 kB/s | 44 kB 00:00 2024-04-25T18:43:27.106 INFO:teuthology.orchestra.run.smithi062.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T18:43:27.131 INFO:teuthology.orchestra.run.smithi062.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-04-25T18:43:27.156 INFO:teuthology.orchestra.run.smithi062.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-25T18:43:27.190 INFO:teuthology.orchestra.run.smithi062.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 305 kB 00:00 2024-04-25T18:43:27.215 INFO:teuthology.orchestra.run.smithi062.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-04-25T18:43:27.241 INFO:teuthology.orchestra.run.smithi062.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T18:43:27.266 INFO:teuthology.orchestra.run.smithi062.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T18:43:27.283 INFO:teuthology.orchestra.run.smithi062.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T18:43:27.300 INFO:teuthology.orchestra.run.smithi062.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 665 kB/s | 22 kB 00:00 2024-04-25T18:43:27.301 INFO:teuthology.orchestra.run.smithi079.stdout:(7/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.8 MB/s | 5.4 MB 00:02 2024-04-25T18:43:27.342 INFO:teuthology.orchestra.run.smithi062.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T18:43:27.493 INFO:teuthology.orchestra.run.smithi079.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 11 MB/s | 17 MB 00:01 2024-04-25T18:43:27.543 INFO:teuthology.orchestra.run.smithi079.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 505 kB/s | 25 kB 00:00 2024-04-25T18:43:27.918 INFO:teuthology.orchestra.run.smithi062.stdout:(104/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 7.0 MB/s | 49 MB 00:07 2024-04-25T18:43:28.023 INFO:teuthology.orchestra.run.smithi045.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 3.6 MB/s | 21 MB 00:05 2024-04-25T18:43:28.106 INFO:teuthology.orchestra.run.smithi045.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 307 kB/s | 25 kB 00:00 2024-04-25T18:43:28.325 INFO:teuthology.orchestra.run.smithi062.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 51 kB/s | 19 kB 00:00 2024-04-25T18:43:28.467 INFO:teuthology.orchestra.run.smithi062.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 470 kB/s | 548 kB 00:01 2024-04-25T18:43:28.534 INFO:teuthology.orchestra.run.smithi062.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 259 kB/s | 309 kB 00:01 2024-04-25T18:43:28.576 INFO:teuthology.orchestra.run.smithi062.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-04-25T18:43:28.652 INFO:teuthology.orchestra.run.smithi062.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-04-25T18:43:28.677 INFO:teuthology.orchestra.run.smithi062.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-04-25T18:43:28.785 INFO:teuthology.orchestra.run.smithi062.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-04-25T18:43:28.886 INFO:teuthology.orchestra.run.smithi062.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-04-25T18:43:28.994 INFO:teuthology.orchestra.run.smithi062.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 402 kB/s | 43 kB 00:00 2024-04-25T18:43:29.036 INFO:teuthology.orchestra.run.smithi062.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.1 MB/s | 548 kB 00:00 2024-04-25T18:43:29.103 INFO:teuthology.orchestra.run.smithi062.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-25T18:43:29.170 INFO:teuthology.orchestra.run.smithi062.stdout:(116/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-04-25T18:43:29.195 INFO:teuthology.orchestra.run.smithi062.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 94 kB/s | 15 kB 00:00 2024-04-25T18:43:29.220 INFO:teuthology.orchestra.run.smithi062.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-25T18:43:29.277 INFO:teuthology.orchestra.run.smithi079.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 5.4 MB/s | 11 MB 00:01 2024-04-25T18:43:29.304 INFO:teuthology.orchestra.run.smithi062.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-25T18:43:29.329 INFO:teuthology.orchestra.run.smithi062.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 325 kB/s | 35 kB 00:00 2024-04-25T18:43:29.355 INFO:teuthology.orchestra.run.smithi062.stdout:(121/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-04-25T18:43:29.369 INFO:teuthology.orchestra.run.smithi079.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.4 MB/s | 225 kB 00:00 2024-04-25T18:43:29.430 INFO:teuthology.orchestra.run.smithi062.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-04-25T18:43:29.444 INFO:teuthology.orchestra.run.smithi079.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 361 kB/s | 27 kB 00:00 2024-04-25T18:43:29.463 INFO:teuthology.orchestra.run.smithi062.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-25T18:43:29.506 INFO:teuthology.orchestra.run.smithi045.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 3.4 MB/s | 17 MB 00:04 2024-04-25T18:43:29.519 INFO:teuthology.orchestra.run.smithi079.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 423 kB/s | 32 kB 00:00 2024-04-25T18:43:29.539 INFO:teuthology.orchestra.run.smithi062.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-04-25T18:43:29.581 INFO:teuthology.orchestra.run.smithi062.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-04-25T18:43:29.606 INFO:teuthology.orchestra.run.smithi062.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 124 kB/s | 18 kB 00:00 2024-04-25T18:43:29.644 INFO:teuthology.orchestra.run.smithi079.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 5.8 MB/s | 747 kB 00:00 2024-04-25T18:43:29.648 INFO:teuthology.orchestra.run.smithi062.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-25T18:43:29.672 INFO:teuthology.orchestra.run.smithi045.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.3 MB/s | 225 kB 00:00 2024-04-25T18:43:29.690 INFO:teuthology.orchestra.run.smithi062.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 178 kB/s | 19 kB 00:00 2024-04-25T18:43:29.724 INFO:teuthology.orchestra.run.smithi062.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 167 kB/s | 20 kB 00:00 2024-04-25T18:43:29.736 INFO:teuthology.orchestra.run.smithi079.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 163 kB 00:00 2024-04-25T18:43:29.756 INFO:teuthology.orchestra.run.smithi045.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 326 kB/s | 27 kB 00:00 2024-04-25T18:43:29.799 INFO:teuthology.orchestra.run.smithi062.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-04-25T18:43:29.820 INFO:teuthology.orchestra.run.smithi079.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-25T18:43:29.832 INFO:teuthology.orchestra.run.smithi062.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 728 kB/s | 79 kB 00:00 2024-04-25T18:43:29.840 INFO:teuthology.orchestra.run.smithi045.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 385 kB/s | 32 kB 00:00 2024-04-25T18:43:29.866 INFO:teuthology.orchestra.run.smithi062.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-04-25T18:43:29.908 INFO:teuthology.orchestra.run.smithi062.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-04-25T18:43:29.928 INFO:teuthology.orchestra.run.smithi079.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 4.5 MB/s | 500 kB 00:00 2024-04-25T18:43:29.942 INFO:teuthology.orchestra.run.smithi062.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-25T18:43:29.975 INFO:teuthology.orchestra.run.smithi062.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-04-25T18:43:30.017 INFO:teuthology.orchestra.run.smithi062.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 827 kB/s | 90 kB 00:00 2024-04-25T18:43:30.051 INFO:teuthology.orchestra.run.smithi062.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-04-25T18:43:30.084 INFO:teuthology.orchestra.run.smithi062.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 283 kB/s | 31 kB 00:00 2024-04-25T18:43:30.126 INFO:teuthology.orchestra.run.smithi062.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T18:43:30.169 INFO:teuthology.orchestra.run.smithi062.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 505 kB/s | 59 kB 00:00 2024-04-25T18:43:30.210 INFO:teuthology.orchestra.run.smithi062.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 284 kB/s | 36 kB 00:00 2024-04-25T18:43:30.224 INFO:teuthology.orchestra.run.smithi045.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.9 MB/s | 747 kB 00:00 2024-04-25T18:43:30.268 INFO:teuthology.orchestra.run.smithi062.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 358 kB/s | 51 kB 00:00 2024-04-25T18:43:30.311 INFO:teuthology.orchestra.run.smithi062.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-04-25T18:43:30.324 INFO:teuthology.orchestra.run.smithi045.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 163 kB 00:00 2024-04-25T18:43:30.344 INFO:teuthology.orchestra.run.smithi062.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 670 kB/s | 90 kB 00:00 2024-04-25T18:43:30.394 INFO:teuthology.orchestra.run.smithi062.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-25T18:43:30.411 INFO:teuthology.orchestra.run.smithi062.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 199 kB/s | 20 kB 00:00 2024-04-25T18:43:30.415 INFO:teuthology.orchestra.run.smithi045.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.4 MB/s | 126 kB 00:00 2024-04-25T18:43:30.453 INFO:teuthology.orchestra.run.smithi062.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T18:43:30.545 INFO:teuthology.orchestra.run.smithi062.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T18:43:30.579 INFO:teuthology.orchestra.run.smithi079.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 8.3 MB/s | 5.4 MB 00:00 2024-04-25T18:43:30.591 INFO:teuthology.orchestra.run.smithi045.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 2.8 MB/s | 500 kB 00:00 2024-04-25T18:43:30.603 INFO:teuthology.orchestra.run.smithi062.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-04-25T18:43:30.662 INFO:teuthology.orchestra.run.smithi079.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 542 kB/s | 45 kB 00:00 2024-04-25T18:43:30.695 INFO:teuthology.orchestra.run.smithi062.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-25T18:43:30.755 INFO:teuthology.orchestra.run.smithi079.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-25T18:43:30.838 INFO:teuthology.orchestra.run.smithi079.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.9 MB/s | 161 kB 00:00 2024-04-25T18:43:30.891 INFO:teuthology.orchestra.run.smithi045.stdout:(18/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 2.2 MB/s | 11 MB 00:04 2024-04-25T18:43:30.930 INFO:teuthology.orchestra.run.smithi079.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 3.4 MB/s | 320 kB 00:00 2024-04-25T18:43:30.975 INFO:teuthology.orchestra.run.smithi045.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 549 kB/s | 45 kB 00:00 2024-04-25T18:43:31.039 INFO:teuthology.orchestra.run.smithi079.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.7 MB/s | 298 kB 00:00 2024-04-25T18:43:31.067 INFO:teuthology.orchestra.run.smithi045.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-25T18:43:31.139 INFO:teuthology.orchestra.run.smithi079.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 996 kB/s | 99 kB 00:00 2024-04-25T18:43:31.208 INFO:teuthology.orchestra.run.smithi045.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.1 MB/s | 161 kB 00:00 2024-04-25T18:43:31.239 INFO:teuthology.orchestra.run.smithi079.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 853 kB/s | 85 kB 00:00 2024-04-25T18:43:31.320 INFO:teuthology.orchestra.run.smithi062.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 3.7 MB/s | 3.4 MB 00:00 2024-04-25T18:43:31.450 INFO:teuthology.orchestra.run.smithi045.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.3 MB/s | 320 kB 00:00 2024-04-25T18:43:31.601 INFO:teuthology.orchestra.run.smithi045.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.9 MB/s | 298 kB 00:00 2024-04-25T18:43:31.659 INFO:teuthology.orchestra.run.smithi045.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.7 MB/s | 99 kB 00:00 2024-04-25T18:43:31.718 INFO:teuthology.orchestra.run.smithi045.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T18:43:31.965 INFO:teuthology.orchestra.run.smithi079.stdout:(26/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 11 MB/s | 49 MB 00:04 2024-04-25T18:43:32.010 INFO:teuthology.orchestra.run.smithi045.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.8 MB/s | 5.4 MB 00:01 2024-04-25T18:43:32.032 INFO:teuthology.orchestra.run.smithi079.stdout:(27/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.9 MB/s | 3.1 MB 00:00 2024-04-25T18:43:32.057 INFO:teuthology.orchestra.run.smithi079.stdout:(28/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.8 MB/s | 170 kB 00:00 2024-04-25T18:43:32.068 INFO:teuthology.orchestra.run.smithi045.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.9 MB/s | 170 kB 00:00 2024-04-25T18:43:32.096 INFO:teuthology.orchestra.run.smithi062.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 1.9 MB/s | 3.1 MB 00:01 2024-04-25T18:43:32.099 INFO:teuthology.orchestra.run.smithi079.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 513 kB/s | 34 kB 00:00 2024-04-25T18:43:32.103 INFO:teuthology.orchestra.run.smithi062.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:43:32.103 INFO:teuthology.orchestra.run.smithi062.stdout:Total 16 MB/s | 222 MB 00:13 2024-04-25T18:43:32.119 INFO:teuthology.orchestra.run.smithi045.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 695 kB/s | 34 kB 00:00 2024-04-25T18:43:32.124 INFO:teuthology.orchestra.run.smithi079.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.1 MB/s | 142 kB 00:00 2024-04-25T18:43:32.177 INFO:teuthology.orchestra.run.smithi045.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.4 MB/s | 142 kB 00:00 2024-04-25T18:43:32.558 INFO:teuthology.orchestra.run.smithi079.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 7.7 MB/s | 3.5 MB 00:00 2024-04-25T18:43:32.569 INFO:teuthology.orchestra.run.smithi045.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.6 MB/s | 3.1 MB 00:00 2024-04-25T18:43:32.642 INFO:teuthology.orchestra.run.smithi079.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.9 MB/s | 249 kB 00:00 2024-04-25T18:43:32.725 INFO:teuthology.orchestra.run.smithi079.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 593 kB/s | 49 kB 00:00 2024-04-25T18:43:32.825 INFO:teuthology.orchestra.run.smithi079.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 165 kB/s | 16 kB 00:00 2024-04-25T18:43:32.917 INFO:teuthology.orchestra.run.smithi079.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 273 kB 00:00 2024-04-25T18:43:33.034 INFO:teuthology.orchestra.run.smithi079.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 6.4 MB/s | 762 kB 00:00 2024-04-25T18:43:33.126 INFO:teuthology.orchestra.run.smithi079.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 7.4 MB/s | 7.4 MB 00:01 2024-04-25T18:43:33.136 INFO:teuthology.orchestra.run.smithi045.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 3.7 MB/s | 3.5 MB 00:00 2024-04-25T18:43:33.151 INFO:teuthology.orchestra.run.smithi079.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 156 kB/s | 18 kB 00:00 2024-04-25T18:43:33.240 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction check 2024-04-25T18:43:33.260 INFO:teuthology.orchestra.run.smithi079.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 603 kB/s | 80 kB 00:00 2024-04-25T18:43:33.278 INFO:teuthology.orchestra.run.smithi045.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.7 MB/s | 249 kB 00:00 2024-04-25T18:43:33.310 INFO:teuthology.orchestra.run.smithi079.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-25T18:43:33.337 INFO:teuthology.orchestra.run.smithi045.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 856 kB/s | 49 kB 00:00 2024-04-25T18:43:33.346 INFO:teuthology.orchestra.run.smithi079.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-04-25T18:43:33.369 INFO:teuthology.orchestra.run.smithi079.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-04-25T18:43:33.387 INFO:teuthology.orchestra.run.smithi045.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 335 kB/s | 16 kB 00:00 2024-04-25T18:43:33.401 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction check succeeded. 2024-04-25T18:43:33.401 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction test 2024-04-25T18:43:33.419 INFO:teuthology.orchestra.run.smithi079.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-25T18:43:33.436 INFO:teuthology.orchestra.run.smithi079.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T18:43:33.461 INFO:teuthology.orchestra.run.smithi079.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T18:43:33.471 INFO:teuthology.orchestra.run.smithi045.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 273 kB 00:00 2024-04-25T18:43:33.487 INFO:teuthology.orchestra.run.smithi079.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T18:43:33.512 INFO:teuthology.orchestra.run.smithi079.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T18:43:33.538 INFO:teuthology.orchestra.run.smithi079.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T18:43:33.571 INFO:teuthology.orchestra.run.smithi079.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T18:43:33.596 INFO:teuthology.orchestra.run.smithi079.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T18:43:33.621 INFO:teuthology.orchestra.run.smithi045.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 5.0 MB/s | 762 kB 00:00 2024-04-25T18:43:33.621 INFO:teuthology.orchestra.run.smithi079.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-25T18:43:33.647 INFO:teuthology.orchestra.run.smithi079.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T18:43:33.672 INFO:teuthology.orchestra.run.smithi079.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 661 kB/s | 33 kB 00:00 2024-04-25T18:43:33.679 INFO:teuthology.orchestra.run.smithi045.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 318 kB/s | 18 kB 00:00 2024-04-25T18:43:33.714 INFO:teuthology.orchestra.run.smithi079.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 456 kB/s | 18 kB 00:00 2024-04-25T18:43:33.765 INFO:teuthology.orchestra.run.smithi079.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 25 MB/s | 3.0 MB 00:00 2024-04-25T18:43:33.790 INFO:teuthology.orchestra.run.smithi079.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 152 kB/s | 11 kB 00:00 2024-04-25T18:43:33.816 INFO:teuthology.orchestra.run.smithi079.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-25T18:43:33.832 INFO:teuthology.orchestra.run.smithi079.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T18:43:33.888 INFO:teuthology.orchestra.run.smithi045.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 387 kB/s | 80 kB 00:00 2024-04-25T18:43:33.988 INFO:teuthology.orchestra.run.smithi045.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.9 MB/s | 803 kB 00:00 2024-04-25T18:43:34.030 INFO:teuthology.orchestra.run.smithi045.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 195 kB 00:00 2024-04-25T18:43:34.041 INFO:teuthology.orchestra.run.smithi079.stdout:(59/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.2 MB/s | 21 MB 00:09 2024-04-25T18:43:34.075 INFO:teuthology.orchestra.run.smithi079.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-04-25T18:43:34.100 INFO:teuthology.orchestra.run.smithi079.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 2.1 MB/s | 582 kB 00:00 2024-04-25T18:43:34.117 INFO:teuthology.orchestra.run.smithi079.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 348 kB/s | 25 kB 00:00 2024-04-25T18:43:34.142 INFO:teuthology.orchestra.run.smithi079.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T18:43:34.164 INFO:teuthology.orchestra.run.smithi045.stdout:(41/152): ceph-mgr-diskprediction-local-19.0.0- 4.6 MB/s | 7.4 MB 00:01 2024-04-25T18:43:34.167 INFO:teuthology.orchestra.run.smithi079.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-25T18:43:34.192 INFO:teuthology.orchestra.run.smithi079.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-25T18:43:34.197 INFO:teuthology.orchestra.run.smithi045.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-04-25T18:43:34.217 INFO:teuthology.orchestra.run.smithi079.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 470 kB/s | 35 kB 00:00 2024-04-25T18:43:34.264 INFO:teuthology.orchestra.run.smithi045.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-25T18:43:34.290 INFO:teuthology.orchestra.run.smithi045.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-25T18:43:34.315 INFO:teuthology.orchestra.run.smithi045.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-04-25T18:43:34.326 INFO:teuthology.orchestra.run.smithi079.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 33 MB/s | 4.4 MB 00:00 2024-04-25T18:43:34.340 INFO:teuthology.orchestra.run.smithi045.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.5 MB/s | 125 kB 00:00 2024-04-25T18:43:34.368 INFO:teuthology.orchestra.run.smithi079.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T18:43:34.374 INFO:teuthology.orchestra.run.smithi045.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 215 kB 00:00 2024-04-25T18:43:34.424 INFO:teuthology.orchestra.run.smithi045.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 8.3 MB/s | 689 kB 00:00 2024-04-25T18:43:34.443 INFO:teuthology.orchestra.run.smithi079.stdout:(69/152): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-04-25T18:43:34.474 INFO:teuthology.orchestra.run.smithi045.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 9.9 MB/s | 943 kB 00:00 2024-04-25T18:43:34.485 INFO:teuthology.orchestra.run.smithi079.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.7 MB/s | 911 kB 00:00 2024-04-25T18:43:34.500 INFO:teuthology.orchestra.run.smithi045.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T18:43:34.510 INFO:teuthology.orchestra.run.smithi079.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 956 kB/s | 133 kB 00:00 2024-04-25T18:43:34.525 INFO:teuthology.orchestra.run.smithi045.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 445 kB/s | 21 kB 00:00 2024-04-25T18:43:34.560 INFO:teuthology.orchestra.run.smithi079.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.8 MB/s | 87 kB 00:00 2024-04-25T18:43:34.567 INFO:teuthology.orchestra.run.smithi045.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 4.7 MB/s | 313 kB 00:00 2024-04-25T18:43:34.592 INFO:teuthology.orchestra.run.smithi045.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 521 kB/s | 33 kB 00:00 2024-04-25T18:43:34.602 INFO:teuthology.orchestra.run.smithi079.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-25T18:43:34.651 INFO:teuthology.orchestra.run.smithi045.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 359 kB/s | 18 kB 00:00 2024-04-25T18:43:34.678 INFO:teuthology.orchestra.run.smithi079.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-04-25T18:43:34.726 INFO:teuthology.orchestra.run.smithi045.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-04-25T18:43:34.751 INFO:teuthology.orchestra.run.smithi045.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 123 kB/s | 11 kB 00:00 2024-04-25T18:43:34.770 INFO:teuthology.orchestra.run.smithi079.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 6.0 MB 00:00 2024-04-25T18:43:34.794 INFO:teuthology.orchestra.run.smithi079.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-04-25T18:43:34.820 INFO:teuthology.orchestra.run.smithi079.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-25T18:43:34.845 INFO:teuthology.orchestra.run.smithi079.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 345 kB/s | 26 kB 00:00 2024-04-25T18:43:34.870 INFO:teuthology.orchestra.run.smithi079.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-04-25T18:43:34.876 INFO:teuthology.orchestra.run.smithi045.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-25T18:43:34.893 INFO:teuthology.orchestra.run.smithi045.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 324 kB/s | 45 kB 00:00 2024-04-25T18:43:34.895 INFO:teuthology.orchestra.run.smithi079.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T18:43:34.921 INFO:teuthology.orchestra.run.smithi079.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 464 kB/s | 35 kB 00:00 2024-04-25T18:43:34.946 INFO:teuthology.orchestra.run.smithi079.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T18:43:34.952 INFO:teuthology.orchestra.run.smithi045.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.8 MB/s | 662 kB 00:00 2024-04-25T18:43:34.980 INFO:teuthology.orchestra.run.smithi079.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T18:43:34.994 INFO:teuthology.orchestra.run.smithi045.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.7 MB/s | 582 kB 00:00 2024-04-25T18:43:35.013 INFO:teuthology.orchestra.run.smithi079.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 77 kB 00:00 2024-04-25T18:43:35.036 INFO:teuthology.orchestra.run.smithi045.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 319 kB/s | 25 kB 00:00 2024-04-25T18:43:35.078 INFO:teuthology.orchestra.run.smithi045.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 576 kB/s | 47 kB 00:00 2024-04-25T18:43:35.080 INFO:teuthology.orchestra.run.smithi079.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-04-25T18:43:35.105 INFO:teuthology.orchestra.run.smithi079.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 346 kB/s | 42 kB 00:00 2024-04-25T18:43:35.114 INFO:teuthology.orchestra.run.smithi045.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-04-25T18:43:35.131 INFO:teuthology.orchestra.run.smithi079.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-04-25T18:43:35.156 INFO:teuthology.orchestra.run.smithi079.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T18:43:35.176 INFO:teuthology.orchestra.run.smithi045.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-25T18:43:35.181 INFO:teuthology.orchestra.run.smithi079.stdout:(89/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T18:43:35.207 INFO:teuthology.orchestra.run.smithi079.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 680 kB/s | 51 kB 00:00 2024-04-25T18:43:35.222 INFO:teuthology.orchestra.run.smithi045.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 332 kB/s | 35 kB 00:00 2024-04-25T18:43:35.232 INFO:teuthology.orchestra.run.smithi079.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 714 kB/s | 54 kB 00:00 2024-04-25T18:43:35.257 INFO:teuthology.orchestra.run.smithi079.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-04-25T18:43:35.271 INFO:teuthology.orchestra.run.smithi062.stdout:Transaction test succeeded. 2024-04-25T18:43:35.271 INFO:teuthology.orchestra.run.smithi062.stdout:Running transaction 2024-04-25T18:43:35.299 INFO:teuthology.orchestra.run.smithi079.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 663 kB/s | 44 kB 00:00 2024-04-25T18:43:35.358 INFO:teuthology.orchestra.run.smithi079.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-04-25T18:43:35.390 INFO:teuthology.orchestra.run.smithi045.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-04-25T18:43:35.458 INFO:teuthology.orchestra.run.smithi079.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 229 kB/s | 36 kB 00:00 2024-04-25T18:43:35.525 INFO:teuthology.orchestra.run.smithi079.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 288 kB/s | 47 kB 00:00 2024-04-25T18:43:35.617 INFO:teuthology.orchestra.run.smithi079.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 305 kB 00:00 2024-04-25T18:43:35.623 INFO:teuthology.orchestra.run.smithi045.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-25T18:43:35.690 INFO:teuthology.orchestra.run.smithi045.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-04-25T18:43:35.740 INFO:teuthology.orchestra.run.smithi045.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.8 MB/s | 911 kB 00:00 2024-04-25T18:43:35.765 INFO:teuthology.orchestra.run.smithi045.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T18:43:35.775 INFO:teuthology.orchestra.run.smithi079.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-04-25T18:43:35.801 INFO:teuthology.orchestra.run.smithi079.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 212 kB/s | 58 kB 00:00 2024-04-25T18:43:35.826 INFO:teuthology.orchestra.run.smithi079.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 451 kB/s | 94 kB 00:00 2024-04-25T18:43:35.851 INFO:teuthology.orchestra.run.smithi079.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-04-25T18:43:35.866 INFO:teuthology.orchestra.run.smithi045.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-25T18:43:35.876 INFO:teuthology.orchestra.run.smithi079.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 851 kB/s | 64 kB 00:00 2024-04-25T18:43:35.893 INFO:teuthology.orchestra.run.smithi079.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 333 kB/s | 22 kB 00:00 2024-04-25T18:43:35.927 INFO:teuthology.orchestra.run.smithi079.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 11 MB/s | 863 kB 00:00 2024-04-25T18:43:35.999 INFO:teuthology.orchestra.run.smithi045.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-04-25T18:43:36.025 INFO:teuthology.orchestra.run.smithi045.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 593 kB/s | 87 kB 00:00 2024-04-25T18:43:36.175 INFO:teuthology.orchestra.run.smithi045.stdout:(74/152): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-04-25T18:43:36.200 INFO:teuthology.orchestra.run.smithi045.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-25T18:43:36.202 INFO:teuthology.orchestra.run.smithi079.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-04-25T18:43:36.242 INFO:teuthology.orchestra.run.smithi045.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 6.0 MB/s | 248 kB 00:00 2024-04-25T18:43:36.284 INFO:teuthology.orchestra.run.smithi045.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 313 kB/s | 26 kB 00:00 2024-04-25T18:43:36.334 INFO:teuthology.orchestra.run.smithi045.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 392 kB/s | 19 kB 00:00 2024-04-25T18:43:36.368 INFO:teuthology.orchestra.run.smithi045.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-25T18:43:36.368 INFO:teuthology.orchestra.run.smithi079.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 652 kB/s | 309 kB 00:00 2024-04-25T18:43:36.394 INFO:teuthology.orchestra.run.smithi079.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-25T18:43:36.410 INFO:teuthology.orchestra.run.smithi045.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 476 kB/s | 35 kB 00:00 2024-04-25T18:43:36.452 INFO:teuthology.orchestra.run.smithi079.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-04-25T18:43:36.469 INFO:teuthology.orchestra.run.smithi079.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 646 kB/s | 49 kB 00:00 2024-04-25T18:43:36.493 INFO:teuthology.orchestra.run.smithi045.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-25T18:43:36.536 INFO:teuthology.orchestra.run.smithi079.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 805 kB/s | 67 kB 00:00 2024-04-25T18:43:36.562 INFO:teuthology.orchestra.run.smithi079.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-25T18:43:36.593 INFO:teuthology.orchestra.run.smithi045.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-25T18:43:36.627 INFO:teuthology.orchestra.run.smithi045.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-04-25T18:43:36.652 INFO:teuthology.orchestra.run.smithi045.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.4 MB/s | 77 kB 00:00 2024-04-25T18:43:36.669 INFO:teuthology.orchestra.run.smithi045.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T18:43:36.687 INFO:teuthology.orchestra.run.smithi079.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-25T18:43:36.711 INFO:teuthology.orchestra.run.smithi045.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 269 kB 00:00 2024-04-25T18:43:36.729 INFO:teuthology.orchestra.run.smithi079.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2024-04-25T18:43:36.745 INFO:teuthology.orchestra.run.smithi045.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T18:43:36.778 INFO:teuthology.orchestra.run.smithi045.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.2 MB/s | 279 kB 00:00 2024-04-25T18:43:36.804 INFO:teuthology.orchestra.run.smithi045.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 926 kB/s | 51 kB 00:00 2024-04-25T18:43:36.804 INFO:teuthology.orchestra.run.smithi079.stdout:(114/152): libarrow-9.0.0-10.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-04-25T18:43:36.829 INFO:teuthology.orchestra.run.smithi045.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T18:43:36.829 INFO:teuthology.orchestra.run.smithi079.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 305 kB/s | 43 kB 00:00 2024-04-25T18:43:36.854 INFO:teuthology.orchestra.run.smithi079.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 257 kB/s | 32 kB 00:00 2024-04-25T18:43:36.896 INFO:teuthology.orchestra.run.smithi045.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 838 kB/s | 42 kB 00:00 2024-04-25T18:43:36.896 INFO:teuthology.orchestra.run.smithi079.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-25T18:43:36.963 INFO:teuthology.orchestra.run.smithi079.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-25T18:43:36.971 INFO:teuthology.orchestra.run.smithi045.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 751 kB/s | 44 kB 00:00 2024-04-25T18:43:36.989 INFO:teuthology.orchestra.run.smithi079.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-04-25T18:43:37.031 INFO:teuthology.orchestra.run.smithi079.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-25T18:43:37.063 INFO:teuthology.orchestra.run.smithi045.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.7 MB/s | 221 kB 00:00 2024-04-25T18:43:37.073 INFO:teuthology.orchestra.run.smithi079.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 323 kB/s | 35 kB 00:00 2024-04-25T18:43:37.098 INFO:teuthology.orchestra.run.smithi079.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.8 MB/s | 206 kB 00:00 2024-04-25T18:43:37.165 INFO:teuthology.orchestra.run.smithi079.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-04-25T18:43:37.190 INFO:teuthology.orchestra.run.smithi079.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-25T18:43:37.216 INFO:teuthology.orchestra.run.smithi079.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-25T18:43:37.241 INFO:teuthology.orchestra.run.smithi079.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-04-25T18:43:37.264 INFO:teuthology.orchestra.run.smithi045.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-25T18:43:37.266 INFO:teuthology.orchestra.run.smithi079.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 306 kB/s | 23 kB 00:00 2024-04-25T18:43:37.289 INFO:teuthology.orchestra.run.smithi045.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 166 kB/s | 36 kB 00:00 2024-04-25T18:43:37.314 INFO:teuthology.orchestra.run.smithi045.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 977 kB/s | 47 kB 00:00 2024-04-25T18:43:37.373 INFO:teuthology.orchestra.run.smithi045.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-25T18:43:37.398 INFO:teuthology.orchestra.run.smithi045.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 725 kB/s | 58 kB 00:00 2024-04-25T18:43:37.423 INFO:teuthology.orchestra.run.smithi045.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-25T18:43:37.448 INFO:teuthology.orchestra.run.smithi045.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-25T18:43:37.450 INFO:teuthology.orchestra.run.smithi079.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 83 kB/s | 19 kB 00:00 2024-04-25T18:43:37.474 INFO:teuthology.orchestra.run.smithi045.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 920 kB/s | 45 kB 00:00 2024-04-25T18:43:37.475 INFO:teuthology.orchestra.run.smithi079.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 84 kB/s | 20 kB 00:00 2024-04-25T18:43:37.499 INFO:teuthology.orchestra.run.smithi045.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 457 kB/s | 22 kB 00:00 2024-04-25T18:43:37.509 INFO:teuthology.orchestra.run.smithi079.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-04-25T18:43:37.534 INFO:teuthology.orchestra.run.smithi079.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 553 kB/s | 46 kB 00:00 2024-04-25T18:43:37.558 INFO:teuthology.orchestra.run.smithi045.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T18:43:37.559 INFO:teuthology.orchestra.run.smithi079.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 940 kB/s | 79 kB 00:00 2024-04-25T18:43:37.584 INFO:teuthology.orchestra.run.smithi079.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-04-25T18:43:37.635 INFO:teuthology.orchestra.run.smithi079.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-25T18:43:37.660 INFO:teuthology.orchestra.run.smithi079.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-25T18:43:37.685 INFO:teuthology.orchestra.run.smithi079.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 896 kB/s | 90 kB 00:00 2024-04-25T18:43:37.710 INFO:teuthology.orchestra.run.smithi079.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-25T18:43:37.736 INFO:teuthology.orchestra.run.smithi079.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-25T18:43:37.761 INFO:teuthology.orchestra.run.smithi079.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-25T18:43:37.803 INFO:teuthology.orchestra.run.smithi079.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-04-25T18:43:37.820 INFO:teuthology.orchestra.run.smithi079.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 426 kB/s | 36 kB 00:00 2024-04-25T18:43:37.853 INFO:teuthology.orchestra.run.smithi079.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 553 kB/s | 51 kB 00:00 2024-04-25T18:43:37.895 INFO:teuthology.orchestra.run.smithi079.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-04-25T18:43:37.912 INFO:teuthology.orchestra.run.smithi079.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 972 kB/s | 90 kB 00:00 2024-04-25T18:43:37.946 INFO:teuthology.orchestra.run.smithi079.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-25T18:43:37.971 INFO:teuthology.orchestra.run.smithi079.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-04-25T18:43:38.004 INFO:teuthology.orchestra.run.smithi079.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-25T18:43:38.055 INFO:teuthology.orchestra.run.smithi079.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T18:43:38.113 INFO:teuthology.orchestra.run.smithi079.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-04-25T18:43:38.214 INFO:teuthology.orchestra.run.smithi079.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-04-25T18:43:38.233 INFO:teuthology.orchestra.run.smithi045.stdout:(104/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 4.8 MB/s | 49 MB 00:10 2024-04-25T18:43:38.358 INFO:teuthology.orchestra.run.smithi079.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 8.8 MB/s | 3.4 MB 00:00 2024-04-25T18:43:38.464 INFO:teuthology.orchestra.run.smithi079.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 6.8 MB/s | 3.1 MB 00:00 2024-04-25T18:43:38.472 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:43:38.472 INFO:teuthology.orchestra.run.smithi079.stdout:Total 15 MB/s | 222 MB 00:14 2024-04-25T18:43:38.508 INFO:teuthology.orchestra.run.smithi045.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 71 kB/s | 19 kB 00:00 2024-04-25T18:43:38.575 INFO:teuthology.orchestra.run.smithi045.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 304 kB/s | 309 kB 00:01 2024-04-25T18:43:38.625 INFO:teuthology.orchestra.run.smithi045.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 491 kB/s | 548 kB 00:01 2024-04-25T18:43:38.700 INFO:teuthology.orchestra.run.smithi045.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-04-25T18:43:38.751 INFO:teuthology.orchestra.run.smithi045.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 388 kB/s | 49 kB 00:00 2024-04-25T18:43:38.809 INFO:teuthology.orchestra.run.smithi045.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 624 kB/s | 67 kB 00:00 2024-04-25T18:43:38.980 INFO:teuthology.orchestra.run.smithi045.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 662 kB/s | 151 kB 00:00 2024-04-25T18:43:39.226 INFO:teuthology.orchestra.run.smithi045.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-04-25T18:43:39.326 INFO:teuthology.orchestra.run.smithi045.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.6 MB/s | 838 kB 00:00 2024-04-25T18:43:39.352 INFO:teuthology.orchestra.run.smithi045.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 349 kB/s | 43 kB 00:00 2024-04-25T18:43:39.469 INFO:teuthology.orchestra.run.smithi045.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 228 kB/s | 32 kB 00:00 2024-04-25T18:43:39.494 INFO:teuthology.orchestra.run.smithi045.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 106 kB/s | 15 kB 00:00 2024-04-25T18:43:39.568 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-04-25T18:43:39.603 INFO:teuthology.orchestra.run.smithi045.stdout:(117/152): libarrow-9.0.0-10.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-04-25T18:43:39.636 INFO:teuthology.orchestra.run.smithi045.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-04-25T18:43:39.686 INFO:teuthology.orchestra.run.smithi045.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-04-25T18:43:39.713 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-04-25T18:43:39.713 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-04-25T18:43:39.737 INFO:teuthology.orchestra.run.smithi045.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-04-25T18:43:39.762 INFO:teuthology.orchestra.run.smithi045.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 283 kB/s | 35 kB 00:00 2024-04-25T18:43:39.812 INFO:teuthology.orchestra.run.smithi045.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.6 MB/s | 206 kB 00:00 2024-04-25T18:43:40.087 INFO:teuthology.orchestra.run.smithi045.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 445 kB/s | 144 kB 00:00 2024-04-25T18:43:40.113 INFO:teuthology.orchestra.run.smithi045.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 36 kB/s | 11 kB 00:00 2024-04-25T18:43:40.188 INFO:teuthology.orchestra.run.smithi045.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-04-25T18:43:40.213 INFO:teuthology.orchestra.run.smithi045.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 142 kB/s | 18 kB 00:00 2024-04-25T18:43:40.230 INFO:teuthology.orchestra.run.smithi045.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2024-04-25T18:43:40.281 INFO:teuthology.orchestra.run.smithi045.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 211 kB/s | 19 kB 00:00 2024-04-25T18:43:40.306 INFO:teuthology.orchestra.run.smithi045.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-25T18:43:40.381 INFO:teuthology.orchestra.run.smithi045.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-25T18:43:40.415 INFO:teuthology.orchestra.run.smithi045.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 730 kB/s | 79 kB 00:00 2024-04-25T18:43:40.490 INFO:teuthology.orchestra.run.smithi045.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 536 kB/s | 58 kB 00:00 2024-04-25T18:43:40.549 INFO:teuthology.orchestra.run.smithi045.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-04-25T18:43:40.616 INFO:teuthology.orchestra.run.smithi045.stdout:(134/152): python3-kubernetes-26.1.0-2.el9.noar 2.7 MB/s | 1.0 MB 00:00 2024-04-25T18:43:40.641 INFO:teuthology.orchestra.run.smithi045.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 109 kB/s | 16 kB 00:00 2024-04-25T18:43:40.666 INFO:teuthology.orchestra.run.smithi045.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 772 kB/s | 90 kB 00:00 2024-04-25T18:43:40.725 INFO:teuthology.orchestra.run.smithi045.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 261 kB/s | 28 kB 00:00 2024-04-25T18:43:40.750 INFO:teuthology.orchestra.run.smithi045.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 285 kB/s | 31 kB 00:00 2024-04-25T18:43:40.800 INFO:teuthology.orchestra.run.smithi045.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-04-25T18:43:40.859 INFO:teuthology.orchestra.run.smithi045.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 332 kB/s | 36 kB 00:00 2024-04-25T18:43:40.884 INFO:teuthology.orchestra.run.smithi045.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 374 kB/s | 59 kB 00:00 2024-04-25T18:43:40.926 INFO:teuthology.orchestra.run.smithi045.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 409 kB/s | 51 kB 00:00 2024-04-25T18:43:41.001 INFO:teuthology.orchestra.run.smithi045.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 771 kB/s | 90 kB 00:00 2024-04-25T18:43:41.093 INFO:teuthology.orchestra.run.smithi045.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 984 kB/s | 230 kB 00:00 2024-04-25T18:43:41.118 INFO:teuthology.orchestra.run.smithi045.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 172 kB/s | 20 kB 00:00 2024-04-25T18:43:41.152 INFO:teuthology.orchestra.run.smithi045.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-04-25T18:43:41.394 INFO:teuthology.orchestra.run.smithi045.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 640 kB/s | 192 kB 00:00 2024-04-25T18:43:41.461 INFO:teuthology.orchestra.run.smithi045.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 4.6 MB/s | 1.6 MB 00:00 2024-04-25T18:43:41.473 INFO:teuthology.orchestra.run.smithi062.stdout: Preparing : 1/1 2024-04-25T18:43:41.483 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-04-25T18:43:41.483 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-04-25T18:43:41.536 INFO:teuthology.orchestra.run.smithi045.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T18:43:41.558 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T18:43:41.597 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T18:43:41.628 INFO:teuthology.orchestra.run.smithi045.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-25T18:43:41.651 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T18:43:41.701 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T18:43:41.726 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T18:43:41.770 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T18:43:41.811 INFO:teuthology.orchestra.run.smithi045.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 5.2 MB/s | 3.4 MB 00:00 2024-04-25T18:43:41.953 INFO:teuthology.orchestra.run.smithi045.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-25T18:43:41.961 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:43:41.962 INFO:teuthology.orchestra.run.smithi045.stdout:Total 11 MB/s | 222 MB 00:20 2024-04-25T18:43:42.114 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T18:43:42.226 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:43:42.351 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:43:42.378 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:43:42.470 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:43:42.499 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:43:42.658 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T18:43:42.713 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T18:43:42.740 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:43:42.845 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:43:43.035 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:43:43.079 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:43:43.100 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:43:43.123 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:43:43.285 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:43:43.317 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:43:43.428 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T18:43:43.532 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T18:43:43.565 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T18:43:43.601 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T18:43:43.677 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T18:43:43.725 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T18:43:43.769 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T18:43:43.801 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T18:43:43.829 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T18:43:43.902 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T18:43:43.953 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T18:43:43.982 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T18:43:44.026 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T18:43:44.105 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T18:43:44.144 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T18:43:44.188 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T18:43:44.216 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T18:43:44.291 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T18:43:44.340 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T18:43:44.417 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T18:43:44.564 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T18:43:44.626 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T18:43:44.858 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-25T18:43:45.115 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-25T18:43:45.115 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-25T18:43:45.193 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T18:43:45.282 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T18:43:45.316 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T18:43:45.359 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T18:43:45.397 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T18:43:45.439 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T18:43:45.603 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T18:43:45.675 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T18:43:45.731 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T18:43:45.806 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T18:43:45.842 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T18:43:45.879 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T18:43:45.926 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T18:43:45.963 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T18:43:46.005 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T18:43:46.047 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T18:43:46.192 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T18:43:46.235 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T18:43:46.265 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T18:43:46.307 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T18:43:46.406 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T18:43:46.440 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T18:43:46.472 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T18:43:46.505 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T18:43:46.530 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T18:43:46.609 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T18:43:46.739 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T18:43:46.770 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T18:43:46.799 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T18:43:46.841 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T18:43:46.872 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T18:43:46.912 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T18:43:46.941 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T18:43:47.161 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T18:43:47.294 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T18:43:47.337 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T18:43:47.370 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T18:43:47.465 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T18:43:47.493 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-04-25T18:43:47.517 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T18:43:47.556 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T18:43:47.584 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T18:43:47.606 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T18:43:47.622 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T18:43:47.675 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T18:43:47.710 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T18:43:47.716 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T18:43:47.735 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T18:43:47.775 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T18:43:47.778 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T18:43:47.872 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-25T18:43:47.873 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-25T18:43:48.116 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T18:43:48.219 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:43:48.332 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:43:48.362 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:43:48.453 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:43:48.491 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:43:48.551 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T18:43:48.603 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T18:43:48.637 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T18:43:48.646 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T18:43:48.688 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T18:43:48.716 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:43:48.743 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T18:43:48.805 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T18:43:48.811 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:43:48.834 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:43:48.879 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:43:48.901 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:43:48.904 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T18:43:48.920 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:43:48.948 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T18:43:49.093 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:43:49.126 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:43:49.132 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T18:43:49.161 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T18:43:49.232 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T18:43:49.244 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T18:43:49.329 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T18:43:49.367 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T18:43:49.401 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T18:43:49.467 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T18:43:49.509 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T18:43:49.553 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T18:43:49.585 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T18:43:49.613 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T18:43:49.683 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T18:43:49.744 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T18:43:49.774 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T18:43:49.818 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T18:43:49.894 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T18:43:49.928 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T18:43:49.930 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T18:43:49.971 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T18:43:50.001 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T18:43:50.047 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T18:43:50.070 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T18:43:50.123 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T18:43:50.197 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T18:43:50.332 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T18:43:50.384 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T18:43:50.920 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T18:43:51.021 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T18:43:51.050 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T18:43:51.093 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T18:43:51.131 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T18:43:51.172 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T18:43:51.335 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T18:43:51.358 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T18:43:51.392 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T18:43:51.423 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T18:43:51.446 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T18:43:51.451 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T18:43:51.484 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T18:43:51.519 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T18:43:51.551 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T18:43:51.587 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T18:43:51.634 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T18:43:51.669 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T18:43:51.729 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T18:43:51.772 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T18:43:51.812 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T18:43:51.855 INFO:teuthology.orchestra.run.smithi062.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:43:51.908 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T18:43:51.946 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:43:51.952 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T18:43:51.976 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T18:43:51.982 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T18:43:52.013 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T18:43:52.023 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T18:43:52.118 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T18:43:52.148 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T18:43:52.188 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T18:43:52.214 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T18:43:52.238 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T18:43:52.314 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T18:43:52.440 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T18:43:52.479 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T18:43:52.507 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T18:43:52.518 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T18:43:52.541 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T18:43:52.558 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:43:52.580 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T18:43:52.612 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T18:43:52.629 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:43:52.659 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T18:43:52.734 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T18:43:52.919 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T18:43:52.952 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T18:43:52.994 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T18:43:53.028 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T18:43:53.110 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T18:43:53.159 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T18:43:53.197 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T18:43:53.245 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T18:43:53.343 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T18:43:53.390 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T18:43:54.126 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T18:43:54.170 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T18:43:54.204 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T18:43:54.313 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T18:43:54.362 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T18:43:54.450 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T18:43:54.463 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-25T18:43:54.489 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T18:43:54.577 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T18:43:54.652 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T18:43:54.666 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T18:43:54.695 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T18:43:54.773 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T18:43:54.782 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T18:43:54.845 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T18:43:54.878 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T18:43:55.016 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T18:43:55.048 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:43:55.080 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:43:55.422 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T18:43:55.455 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T18:43:55.544 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T18:43:55.558 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:43:55.651 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:43:55.692 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:43:55.772 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:43:55.775 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:43:55.776 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T18:43:55.776 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T18:43:55.776 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:43:55.793 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:43:55.810 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:43:55.961 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:43:56.013 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:43:56.404 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T18:43:56.474 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T18:43:56.505 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:43:56.674 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:43:56.698 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:43:56.784 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T18:43:56.811 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T18:43:56.840 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:43:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T18:43:56.902 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T18:43:56.915 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T18:43:56.935 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T18:43:57.228 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:43:57.256 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T18:43:57.283 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T18:43:57.289 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:43:57.363 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:43:57.394 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T18:43:57.430 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T18:43:57.484 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T18:43:57.603 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T18:43:57.650 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T18:43:57.698 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T18:43:57.804 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T18:43:57.854 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T18:43:57.900 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T18:43:57.917 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T18:43:57.926 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:43:57.966 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T18:43:57.997 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:43:57.998 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T18:43:58.019 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T18:43:58.115 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T18:43:58.186 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T18:43:58.224 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T18:43:58.284 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T18:43:58.392 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T18:43:58.428 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T18:43:58.483 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T18:43:58.517 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T18:43:58.636 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T18:43:58.702 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T18:43:58.811 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T18:43:58.997 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T18:43:59.066 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T18:43:59.814 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T18:43:59.953 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T18:43:59.999 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T18:44:00.046 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T18:44:00.096 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T18:44:00.171 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T18:44:00.336 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:44:00.373 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:44:00.439 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T18:44:00.500 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T18:44:00.577 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T18:44:00.675 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T18:44:00.717 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T18:44:00.748 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T18:44:00.806 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T18:44:00.848 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T18:44:00.909 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T18:44:00.920 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:44:00.968 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T18:44:00.968 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:44:01.059 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:44:01.060 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T18:44:01.060 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T18:44:01.060 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:01.086 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:44:01.169 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T18:44:01.229 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T18:44:01.267 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T18:44:01.334 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T18:44:01.478 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T18:44:01.516 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T18:44:01.564 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T18:44:01.597 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T18:44:01.630 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T18:44:01.834 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T18:44:02.010 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T18:44:02.053 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T18:44:02.099 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T18:44:02.152 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T18:44:02.216 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T18:44:02.277 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T18:44:02.325 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T18:44:02.380 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T18:44:02.432 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T18:44:02.517 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T18:44:02.567 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T18:44:02.717 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T18:44:02.781 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T18:44:02.833 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T18:44:02.906 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T18:44:03.067 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T18:44:03.130 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T18:44:04.131 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T18:44:04.187 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T18:44:04.228 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T18:44:04.368 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T18:44:04.441 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T18:44:04.553 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T18:44:04.596 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T18:44:04.846 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T18:44:04.885 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T18:44:04.986 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T18:44:05.860 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T18:44:06.004 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T18:44:07.669 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:44:07.669 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /sys 2024-04-25T18:44:07.669 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /proc 2024-04-25T18:44:07.669 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /mnt 2024-04-25T18:44:07.669 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /var/tmp 2024-04-25T18:44:07.669 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /home 2024-04-25T18:44:07.669 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /root 2024-04-25T18:44:07.669 INFO:teuthology.orchestra.run.smithi062.stdout:skipping the directory /tmp 2024-04-25T18:44:07.669 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:07.676 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T18:44:07.773 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T18:44:07.834 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T18:44:07.845 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:44:07.861 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:44:07.862 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:07.862 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T18:44:07.862 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:07.877 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T18:44:08.289 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T18:44:08.327 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:44:08.449 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T18:44:08.484 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T18:44:08.541 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T18:44:09.145 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T18:44:09.171 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:44:09.315 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T18:44:09.340 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T18:44:09.390 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:44:09.417 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:44:09.418 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:09.418 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T18:44:09.418 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:44:09.418 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:44:09.418 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:09.649 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:44:09.673 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:44:09.673 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:09.673 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T18:44:09.673 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:44:09.674 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:44:09.674 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:10.133 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:44:10.157 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:44:10.157 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:10.157 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T18:44:10.157 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:44:10.157 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:44:10.157 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:10.272 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T18:44:10.331 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T18:44:10.395 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T18:44:10.593 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T18:44:10.654 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T18:44:10.732 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T18:44:10.831 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:44:10.976 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:44:11.026 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T18:44:11.057 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T18:44:11.101 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T18:44:11.538 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:44:11.558 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:44:11.582 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T18:44:11.646 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T18:44:11.672 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:44:12.254 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:44:12.294 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:44:12.779 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:44:12.816 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:44:12.839 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:44:12.839 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2024-04-25T18:44:12.839 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2024-04-25T18:44:12.839 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2024-04-25T18:44:12.840 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2024-04-25T18:44:12.840 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2024-04-25T18:44:12.840 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /root 2024-04-25T18:44:12.840 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2024-04-25T18:44:12.840 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:12.979 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T18:44:13.014 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:44:13.023 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:44:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:44:13.030 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:13.030 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T18:44:13.030 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T18:44:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T18:44:13.183 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T18:44:13.183 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:13.223 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:44:13.833 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:44:13.858 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:44:13.966 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:44:14.137 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T18:44:14.168 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:44:14.193 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:44:14.193 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:14.193 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T18:44:14.193 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:44:14.193 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:44:14.193 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:14.244 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:44:14.255 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:44:14.357 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T18:44:14.410 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T18:44:14.517 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:44:14.542 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:44:14.542 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:14.542 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T18:44:14.542 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:44:14.542 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:44:14.543 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:14.767 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:44:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:44:14.790 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:14.790 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T18:44:14.790 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:44:14.790 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:44:14.790 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:15.243 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:44:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:44:15.266 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:15.266 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T18:44:15.266 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:44:15.266 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:44:15.266 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:15.374 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T18:44:15.385 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T18:44:15.421 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:44:15.431 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T18:44:15.445 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:44:15.445 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:15.446 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T18:44:15.446 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:44:15.446 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:44:15.446 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:15.486 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T18:44:15.488 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:44:15.512 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:44:15.512 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:15.512 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T18:44:15.512 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:15.556 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:44:15.579 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:44:15.580 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:15.580 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T18:44:15.580 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:44:15.580 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:44:15.580 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:15.673 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T18:44:15.727 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T18:44:15.805 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T18:44:15.894 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:44:15.909 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:44:15.930 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:44:15.931 INFO:teuthology.orchestra.run.smithi062.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:15.931 INFO:teuthology.orchestra.run.smithi062.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T18:44:15.931 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:44:15.931 INFO:teuthology.orchestra.run.smithi062.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:44:15.931 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:16.026 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:44:16.069 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T18:44:16.101 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T18:44:16.129 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T18:44:16.547 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:44:16.577 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:44:16.601 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T18:44:16.663 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T18:44:16.692 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:44:17.729 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:44:17.760 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:44:18.725 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:44:18.752 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:44:18.854 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:44:19.026 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T18:44:19.054 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:44:19.077 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:44:19.077 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:19.077 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T18:44:19.077 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:44:19.077 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:44:19.078 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:19.122 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:44:19.132 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:44:19.223 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T18:44:19.278 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T18:44:20.205 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T18:44:20.233 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:44:20.255 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:44:20.255 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:20.255 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T18:44:20.255 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:44:20.255 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:44:20.255 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:20.300 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:44:20.321 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:44:20.322 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:20.322 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T18:44:20.322 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:20.367 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:44:20.389 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:44:20.389 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:20.389 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T18:44:20.390 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:44:20.390 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:44:20.390 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:20.693 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:44:20.714 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:44:20.715 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:20.715 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T18:44:20.715 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:44:20.715 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:44:20.715 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:20.802 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T18:44:20.844 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T18:44:20.884 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T18:44:20.916 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T18:44:20.943 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T18:44:21.144 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T18:44:21.189 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T18:44:21.218 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T18:44:21.858 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:44:21.899 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:44:21.920 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:44:22.823 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T18:44:22.912 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T18:44:22.954 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T18:44:23.038 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T18:44:23.074 INFO:teuthology.orchestra.run.smithi062.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T18:44:23.074 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:44:23.141 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:44:23.183 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:44:23.183 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:44:23.259 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:44:23.327 INFO:teuthology.orchestra.run.smithi062.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:44:25.306 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T18:44:25.347 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T18:44:25.389 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T18:44:25.420 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T18:44:25.448 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T18:44:25.638 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T18:44:25.677 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T18:44:25.707 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T18:44:26.308 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:44:26.347 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:44:26.366 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:44:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T18:44:26.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-04-25T18:44:26.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-04-25T18:44:26.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-04-25T18:44:26.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-04-25T18:44:26.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-04-25T18:44:26.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-04-25T18:44:26.674 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-04-25T18:44:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:44:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T18:44:26.969 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:26.969 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T18:44:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:27.203 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T18:44:27.277 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T18:44:27.318 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T18:44:27.401 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T18:44:27.440 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T18:44:27.441 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:44:27.532 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:44:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:44:27.573 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:44:27.662 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:44:27.739 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:44:28.653 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:44:28.738 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T18:44:28.738 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:28.738 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T18:44:28.738 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:44:28.738 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T18:44:28.738 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:29.003 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:44:29.076 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T18:44:29.076 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:29.076 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T18:44:29.076 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:44:29.076 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T18:44:29.076 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:29.099 INFO:teuthology.orchestra.run.smithi062.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:44:29.100 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T18:44:29.101 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T18:44:29.102 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T18:44:29.102 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T18:44:29.102 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T18:44:29.102 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T18:44:29.102 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T18:44:29.102 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T18:44:29.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T18:44:29.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T18:44:29.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T18:44:29.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T18:44:29.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T18:44:29.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T18:44:29.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T18:44:29.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T18:44:29.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T18:44:29.115 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T18:44:29.116 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T18:44:29.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T18:44:29.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T18:44:29.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T18:44:29.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T18:44:29.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T18:44:29.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T18:44:29.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T18:44:29.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T18:44:29.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T18:44:29.117 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T18:44:29.118 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T18:44:29.119 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T18:44:29.120 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T18:44:29.121 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T18:44:29.122 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T18:44:29.122 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T18:44:29.122 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T18:44:29.122 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T18:44:29.122 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T18:44:29.122 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T18:44:29.122 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T18:44:29.122 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T18:44:29.122 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T18:44:29.122 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T18:44:29.123 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T18:44:29.124 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T18:44:29.125 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T18:44:29.126 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T18:44:29.126 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:44:29.126 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T18:44:29.604 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:44:29.675 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T18:44:29.675 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:29.675 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T18:44:29.675 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:44:29.676 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T18:44:29.676 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:29.814 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T18:44:29.877 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T18:44:29.960 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T18:44:30.226 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T18:44:30.285 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T18:44:30.376 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T18:44:30.505 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:44:30.688 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:44:30.744 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T18:44:30.781 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T18:44:30.814 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T18:44:31.184 INFO:teuthology.orchestra.run.smithi062.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:44:31.184 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:31.184 INFO:teuthology.orchestra.run.smithi062.stdout:Upgraded: 2024-04-25T18:44:31.184 INFO:teuthology.orchestra.run.smithi062.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.184 INFO:teuthology.orchestra.run.smithi062.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.184 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout:Installed: 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.185 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.186 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.187 INFO:teuthology.orchestra.run.smithi062.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:31.187 INFO:teuthology.orchestra.run.smithi062.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:31.187 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.187 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.187 INFO:teuthology.orchestra.run.smithi062.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:31.187 INFO:teuthology.orchestra.run.smithi062.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T18:44:31.187 INFO:teuthology.orchestra.run.smithi062.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T18:44:31.187 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T18:44:31.187 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T18:44:31.187 INFO:teuthology.orchestra.run.smithi062.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T18:44:31.188 INFO:teuthology.orchestra.run.smithi062.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T18:44:31.189 INFO:teuthology.orchestra.run.smithi062.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T18:44:31.190 INFO:teuthology.orchestra.run.smithi062.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T18:44:31.191 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T18:44:31.192 INFO:teuthology.orchestra.run.smithi062.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T18:44:31.193 INFO:teuthology.orchestra.run.smithi062.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.194 INFO:teuthology.orchestra.run.smithi062.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T18:44:31.195 INFO:teuthology.orchestra.run.smithi062.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T18:44:31.196 INFO:teuthology.orchestra.run.smithi062.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T18:44:31.197 INFO:teuthology.orchestra.run.smithi062.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T18:44:31.197 INFO:teuthology.orchestra.run.smithi062.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T18:44:31.197 INFO:teuthology.orchestra.run.smithi062.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T18:44:31.197 INFO:teuthology.orchestra.run.smithi062.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T18:44:31.197 INFO:teuthology.orchestra.run.smithi062.stdout: 2024-04-25T18:44:31.197 INFO:teuthology.orchestra.run.smithi062.stdout:Complete! 2024-04-25T18:44:31.454 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:44:31.508 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T18:44:31.568 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T18:44:31.581 DEBUG:teuthology.parallel:result is None 2024-04-25T18:44:31.678 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T18:44:31.711 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:44:33.086 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T18:44:33.138 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:44:33.605 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T18:44:33.606 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T18:44:33.607 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T18:44:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T18:44:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T18:44:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T18:44:33.608 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T18:44:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T18:44:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T18:44:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T18:44:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T18:44:33.619 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T18:44:33.620 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T18:44:33.621 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T18:44:33.622 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T18:44:33.623 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T18:44:33.624 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T18:44:33.625 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T18:44:33.626 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:44:33.627 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T18:44:34.311 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T18:44:34.346 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:44:34.500 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T18:44:34.753 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T18:44:34.781 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:44:34.860 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T18:44:34.860 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:34.860 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T18:44:34.860 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:44:34.860 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T18:44:34.860 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:34.915 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:44:34.953 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T18:44:35.100 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T18:44:35.180 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T18:44:35.803 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:44:35.803 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:35.803 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2024-04-25T18:44:35.803 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.804 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:35.805 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T18:44:35.806 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T18:44:35.807 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.808 INFO:teuthology.orchestra.run.smithi079.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T18:44:35.808 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T18:44:35.808 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T18:44:35.808 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T18:44:35.808 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T18:44:35.808 INFO:teuthology.orchestra.run.smithi079.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T18:44:35.808 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T18:44:35.808 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T18:44:35.808 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T18:44:35.808 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T18:44:35.809 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T18:44:35.809 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T18:44:35.809 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T18:44:35.809 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T18:44:35.809 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T18:44:35.809 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T18:44:35.809 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T18:44:35.809 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T18:44:35.809 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T18:44:35.809 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T18:44:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T18:44:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T18:44:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T18:44:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T18:44:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T18:44:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T18:44:35.810 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T18:44:35.811 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T18:44:35.812 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T18:44:35.813 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T18:44:35.814 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T18:44:35.815 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T18:44:35.815 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.815 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.815 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:35.815 INFO:teuthology.orchestra.run.smithi079.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T18:44:35.815 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T18:44:35.815 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T18:44:35.815 INFO:teuthology.orchestra.run.smithi079.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T18:44:35.815 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T18:44:35.815 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T18:44:35.816 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T18:44:35.816 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:44:35.816 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-04-25T18:44:36.223 DEBUG:teuthology.parallel:result is None 2024-04-25T18:44:36.272 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T18:44:36.309 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:44:36.383 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T18:44:36.383 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:36.383 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T18:44:36.383 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:44:36.383 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T18:44:36.383 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:36.435 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:44:36.509 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T18:44:36.509 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:36.509 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T18:44:36.510 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:36.594 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:44:36.673 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T18:44:36.673 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:36.673 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T18:44:36.673 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:44:36.673 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T18:44:36.674 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:37.053 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:44:37.128 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T18:44:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T18:44:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T18:44:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:44:37.128 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T18:44:37.128 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:42.417 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T18:44:42.471 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T18:44:42.545 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T18:44:42.586 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T18:44:42.656 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T18:44:42.999 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T18:44:43.055 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T18:44:43.082 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T18:44:44.264 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:44:44.338 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:44:44.399 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T18:44:45.916 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T18:44:46.027 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T18:44:46.111 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T18:44:46.308 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T18:44:46.349 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T18:44:46.349 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:44:46.512 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T18:44:46.557 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:44:46.558 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:44:46.715 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T18:44:46.849 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:44:47.121 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:44:47.121 INFO:teuthology.orchestra.run.smithi045.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T18:44:47.121 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:52.929 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T18:44:52.929 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T18:44:52.930 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T18:44:52.931 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T18:44:52.944 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T18:44:52.945 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T18:44:52.946 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T18:44:52.946 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T18:44:52.946 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T18:44:52.946 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T18:44:52.946 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T18:44:52.946 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T18:44:52.946 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T18:44:52.946 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T18:44:52.946 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T18:44:52.946 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T18:44:52.947 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T18:44:52.948 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T18:44:52.949 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T18:44:52.950 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T18:44:52.951 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T18:44:52.952 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T18:44:52.953 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T18:44:55.389 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T18:44:55.389 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.390 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:55.391 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T18:44:55.392 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T18:44:55.393 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T18:44:55.394 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T18:44:55.395 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T18:44:55.396 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T18:44:55.397 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T18:44:55.398 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T18:44:55.399 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T18:44:55.400 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T18:44:55.401 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T18:44:55.401 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T18:44:55.401 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:44:55.401 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-25T18:44:55.951 DEBUG:teuthology.parallel:result is None 2024-04-25T18:44:55.952 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:44:56.172 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T18:44:56.231 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T18:44:56.232 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T18:44:56.232 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T18:44:56.234 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:44:56.372 DEBUG:teuthology.orchestra.run.smithi062:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T18:44:56.399 INFO:teuthology.orchestra.run.smithi062.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T18:44:56.399 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T18:44:56.399 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T18:44:56.400 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:44:56.541 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T18:44:56.567 INFO:teuthology.orchestra.run.smithi079.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T18:44:56.567 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T18:44:56.567 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T18:44:56.569 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T18:44:56.569 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:44:56.569 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T18:44:56.632 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:44:56.632 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T18:44:56.664 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:44:56.664 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T18:44:56.694 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T18:44:56.694 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:44:56.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T18:44:56.756 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T18:44:56.870 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:44:56.870 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T18:44:56.899 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T18:44:56.970 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:44:56.970 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T18:44:56.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T18:44:57.063 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T18:44:57.064 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:44:57.064 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T18:44:57.126 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T18:44:57.229 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:44:57.229 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T18:44:57.254 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T18:44:57.319 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:44:57.319 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T18:44:57.343 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T18:44:57.405 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T18:44:57.406 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:44:57.406 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T18:44:57.476 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T18:44:57.578 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:44:57.578 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T18:44:57.603 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T18:44:57.667 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:44:57.667 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T18:44:57.691 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T18:44:57.754 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T18:44:57.848 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T18:44:57.848 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:44:57.849 INFO:tasks.cephadm:Cluster fsid is ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:44:57.849 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T18:44:57.849 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.45', 'mon.b': '172.21.15.62', 'mon.c': '172.21.15.79'} 2024-04-25T18:44:57.849 INFO:tasks.cephadm:First mon is mon.a on smithi045 2024-04-25T18:44:57.850 INFO:tasks.cephadm:First mgr is x 2024-04-25T18:44:57.850 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T18:44:57.850 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-04-25T18:44:57.920 DEBUG:teuthology.orchestra.run.smithi062:> sudo hostname $(hostname -s) 2024-04-25T18:44:57.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo hostname $(hostname -s) 2024-04-25T18:44:57.967 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T18:44:57.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:44:58.106 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T18:44:58.375 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:44:58.377 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T18:44:58.377 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T18:45:00.318 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 18:45 /home/ubuntu/cephtest/cephadm 2024-04-25T18:45:00.319 DEBUG:teuthology.orchestra.run.smithi062:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T18:45:03.319 INFO:teuthology.orchestra.run.smithi062.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 18:45 /home/ubuntu/cephtest/cephadm 2024-04-25T18:45:03.319 DEBUG:teuthology.orchestra.run.smithi079:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T18:45:05.417 INFO:teuthology.orchestra.run.smithi079.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 18:45 /home/ubuntu/cephtest/cephadm 2024-04-25T18:45:05.418 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T18:45:05.477 DEBUG:teuthology.orchestra.run.smithi062:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T18:45:05.497 DEBUG:teuthology.orchestra.run.smithi079:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T18:45:05.534 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T18:45:05.534 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T18:45:05.542 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T18:45:05.544 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T18:45:07.034 INFO:teuthology.orchestra.run.smithi062.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:45:07.137 INFO:teuthology.orchestra.run.smithi079.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:45:08.507 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:45:45.937 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-25T18:45:45.937 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T18:45:45.937 INFO:teuthology.orchestra.run.smithi079.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T18:45:45.937 INFO:teuthology.orchestra.run.smithi079.stdout: "repo_digests": [ 2024-04-25T18:45:45.937 INFO:teuthology.orchestra.run.smithi079.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T18:45:45.937 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-04-25T18:45:45.937 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-25T18:45:49.996 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T18:45:49.996 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T18:45:49.996 INFO:teuthology.orchestra.run.smithi062.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T18:45:49.996 INFO:teuthology.orchestra.run.smithi062.stdout: "repo_digests": [ 2024-04-25T18:45:49.997 INFO:teuthology.orchestra.run.smithi062.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T18:45:49.997 INFO:teuthology.orchestra.run.smithi062.stdout: ] 2024-04-25T18:45:49.997 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T18:45:52.552 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T18:45:52.552 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T18:45:52.553 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T18:45:52.553 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-04-25T18:45:52.553 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T18:45:52.553 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T18:45:52.553 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T18:45:52.596 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-04-25T18:45:52.677 DEBUG:teuthology.orchestra.run.smithi062:> sudo mkdir -p /etc/ceph 2024-04-25T18:45:52.710 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph 2024-04-25T18:45:52.742 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-04-25T18:45:52.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo chmod 777 /etc/ceph 2024-04-25T18:45:52.854 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /etc/ceph 2024-04-25T18:45:52.882 INFO:tasks.cephadm:Writing seed config... 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T18:45:52.883 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T18:45:52.884 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T18:45:52.885 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T18:45:52.886 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T18:45:52.886 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T18:45:52.886 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T18:45:52.886 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T18:45:52.886 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:45:52.887 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T18:45:52.949 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ea2833ba-0333-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T18:45:52.950 DEBUG:teuthology.orchestra.run.smithi045:mon.a> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T18:45:52.998 DEBUG:teuthology.orchestra.run.smithi045:mgr.x> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T18:45:53.045 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T18:45:53.046 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid ea2833ba-0333-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.45 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:45:53.319 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-25T18:45:53.320 INFO:teuthology.orchestra.run.smithi045.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'ea2833ba-0333-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.45', '--skip-admin-label'] 2024-04-25T18:45:53.320 INFO:teuthology.orchestra.run.smithi045.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T18:45:53.320 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying podman|docker is present... 2024-04-25T18:45:53.379 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T18:45:53.379 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying lvm2 is present... 2024-04-25T18:45:53.379 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying time synchronization is in place... 2024-04-25T18:45:53.396 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T18:45:53.396 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T18:45:53.412 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T18:45:53.412 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T18:45:53.429 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-04-25T18:45:53.444 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-04-25T18:45:53.444 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-04-25T18:45:53.444 INFO:teuthology.orchestra.run.smithi045.stdout:Repeating the final host check... 2024-04-25T18:45:53.504 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T18:45:53.504 INFO:teuthology.orchestra.run.smithi045.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T18:45:53.504 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl is present 2024-04-25T18:45:53.504 INFO:teuthology.orchestra.run.smithi045.stdout:lvcreate is present 2024-04-25T18:45:53.522 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T18:45:53.522 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T18:45:53.544 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T18:45:53.544 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T18:45:53.570 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout enabled 2024-04-25T18:45:53.593 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout active 2024-04-25T18:45:53.594 INFO:teuthology.orchestra.run.smithi045.stdout:Unit chronyd.service is enabled and running 2024-04-25T18:45:53.594 INFO:teuthology.orchestra.run.smithi045.stdout:Host looks OK 2024-04-25T18:45:53.594 INFO:teuthology.orchestra.run.smithi045.stdout:Cluster fsid: ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:45:53.595 INFO:teuthology.orchestra.run.smithi045.stdout:Acquiring lock 140175243455648 on /run/cephadm/ea2833ba-0333-11ef-bc93-c7b262605968.lock 2024-04-25T18:45:53.595 INFO:teuthology.orchestra.run.smithi045.stdout:Lock 140175243455648 acquired on /run/cephadm/ea2833ba-0333-11ef-bc93-c7b262605968.lock 2024-04-25T18:45:53.596 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 3300 ... 2024-04-25T18:45:53.597 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying IP 172.21.15.45 port 6789 ... 2024-04-25T18:45:53.598 INFO:teuthology.orchestra.run.smithi045.stdout:Base mon IP(s) is [172.21.15.45:3300, 172.21.15.45:6789], mon addrv is [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-04-25T18:45:53.613 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.45 metric 100 2024-04-25T18:45:53.614 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.45 metric 100 2024-04-25T18:45:53.627 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T18:45:53.627 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T18:45:53.627 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T18:45:53.639 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T18:45:53.639 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T18:45:53.639 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T18:45:53.639 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T18:45:53.640 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-04-25T18:45:53.640 INFO:teuthology.orchestra.run.smithi045.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T18:45:53.642 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-04-25T18:45:53.642 INFO:teuthology.orchestra.run.smithi045.stdout:Mon IP `172.21.15.45` is in CIDR network `172.21.0.0/20` 2024-04-25T18:45:53.643 INFO:teuthology.orchestra.run.smithi045.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T18:45:53.646 INFO:teuthology.orchestra.run.smithi045.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T18:45:53.648 INFO:teuthology.orchestra.run.smithi045.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:45:57.529 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T18:45:57.529 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T18:45:57.529 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T18:45:57.529 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T18:45:57.530 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T18:45:57.530 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T18:45:57.530 INFO:teuthology.orchestra.run.smithi045.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T18:45:59.780 INFO:teuthology.orchestra.run.smithi045.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T18:45:59.780 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T18:45:59.780 INFO:teuthology.orchestra.run.smithi045.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T18:46:00.918 INFO:teuthology.orchestra.run.smithi045.stdout:stat: stdout 167 167 2024-04-25T18:46:00.918 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial keys... 2024-04-25T18:46:02.060 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDppCpmCK8DFxAAW+Br8gotBVXFhguDd3YENw== 2024-04-25T18:46:03.346 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDqpCpmdUqDLBAAcvw8ptVF1b+49zwjXy6FOA== 2024-04-25T18:46:04.373 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph-authtool: stdout AQDrpCpmXhAcLxAA9FHsdGIOtXlBuHJXHkVMmA== 2024-04-25T18:46:04.375 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial monmap... 2024-04-25T18:46:05.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T18:46:05.496 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T18:46:05.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:05.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T18:46:05.497 INFO:teuthology.orchestra.run.smithi045.stdout:monmaptool for a [v2:172.21.15.45:3300,v1:172.21.15.45:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T18:46:05.497 INFO:teuthology.orchestra.run.smithi045.stdout:setting min_mon_release = quincy 2024-04-25T18:46:05.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: set fsid to ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:05.497 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T18:46:05.497 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:46:05.498 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mon... 2024-04-25T18:46:06.710 INFO:teuthology.orchestra.run.smithi045.stdout:create mon.a on 2024-04-25T18:46:07.113 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T18:46:07.365 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T18:46:07.668 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ea2833ba-0333-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-ea2833ba-0333-11ef-bc93-c7b262605968.target. 2024-04-25T18:46:07.668 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ea2833ba-0333-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-ea2833ba-0333-11ef-bc93-c7b262605968.target. 2024-04-25T18:46:08.056 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.a 2024-04-25T18:46:08.056 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T18:46:08.363 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ea2833ba-0333-11ef-bc93-c7b262605968.target.wants/ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ea2833ba-0333-11ef-bc93-c7b262605968@.service. 2024-04-25T18:46:09.058 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:08 smithi045 podman[27390]: 2024-04-25 18:46:08.792757609 +0000 UTC m=+0.152590208 container create a8cfa50b2c7c65c5b7d8a289471c1f97afccf42fae08bb836759406a895b6dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T18:46:09.059 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:09 smithi045 podman[27390]: 2024-04-25 18:46:09.056013279 +0000 UTC m=+0.415845878 container init a8cfa50b2c7c65c5b7d8a289471c1f97afccf42fae08bb836759406a895b6dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T18:46:09.178 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:46:09.178 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-25T18:46:09.193 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:46:09.193 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T18:46:09.194 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-25T18:46:09.194 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T18:46:09.194 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon to start... 2024-04-25T18:46:09.194 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mon... 2024-04-25T18:46:09.344 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:09 smithi045 podman[27390]: 2024-04-25 18:46:09.067553072 +0000 UTC m=+0.427385667 container start a8cfa50b2c7c65c5b7d8a289471c1f97afccf42fae08bb836759406a895b6dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T18:46:09.345 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:09 smithi045 bash[27390]: a8cfa50b2c7c65c5b7d8a289471c1f97afccf42fae08bb836759406a895b6dfd 2024-04-25T18:46:09.345 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:09 smithi045 ceph-mon[27409]: mkfs ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:09.345 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:09 smithi045 systemd[1]: Started Ceph mon.a for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:46:09.345 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:09 smithi045 ceph-mon[27409]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T18:46:10.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T18:46:10.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: monmap epoch 1 2024-04-25T18:46:10.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:10.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: last_changed 2024-04-25T18:46:04.854399+0000 2024-04-25T18:46:10.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:46:10.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: min_mon_release 19 (squid) 2024-04-25T18:46:10.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: election_strategy: 1 2024-04-25T18:46:10.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:46:10.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: fsmap 2024-04-25T18:46:10.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T18:46:10.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: mgrmap e1: no daemons active 2024-04-25T18:46:10.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:10 smithi045 ceph-mon[27409]: from='client.? 172.21.15.45:0/195044578' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T18:46:10.643 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout id: ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout services: 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.786983s) 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout data: 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T18:46:10.645 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T18:46:10.651 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T18:46:10.651 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T18:46:10.651 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:10.651 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.838+0000 7ffaf3800700 1 Processor -- start 2024-04-25T18:46:10.651 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.839+0000 7ffaf3800700 1 -- start start 2024-04-25T18:46:10.651 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.839+0000 7ffaf3800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:10.651 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.839+0000 7ffaf3800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec109000 con 0x7ffaec108610 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.840+0000 7ffaf0e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.840+0000 7ffaf0e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56830/0 (socket says 172.21.15.45:56830) 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.840+0000 7ffaf0e00700 1 -- 172.21.15.45:0/3948193321 learned_addr learned my addr 172.21.15.45:0/3948193321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.841+0000 7ffaf0e00700 1 -- 172.21.15.45:0/3948193321 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec109810 con 0x7ffaec108610 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.841+0000 7ffaf0e00700 1 --2- 172.21.15.45:0/3948193321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec108a30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009e80 tx=0x7ffadc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedfcbf29211d2c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.842+0000 7ffaebe00700 1 -- 172.21.15.45:0/3948193321 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffadc017070 con 0x7ffaec108610 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.842+0000 7ffaebe00700 1 -- 172.21.15.45:0/3948193321 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffadc003a20 con 0x7ffaec108610 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.842+0000 7ffaebe00700 1 -- 172.21.15.45:0/3948193321 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffadc003d20 con 0x7ffaec108610 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.842+0000 7ffaf3800700 1 -- 172.21.15.45:0/3948193321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 msgr2=0x7ffaec108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.842+0000 7ffaf3800700 1 --2- 172.21.15.45:0/3948193321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec108a30 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009e80 tx=0x7ffadc00b330 comp rx=0 tx=0).stop 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.843+0000 7ffaf3800700 1 -- 172.21.15.45:0/3948193321 shutdown_connections 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.843+0000 7ffaf3800700 1 --2- 172.21.15.45:0/3948193321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec108a30 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.843+0000 7ffaf3800700 1 -- 172.21.15.45:0/3948193321 >> 172.21.15.45:0/3948193321 conn(0x7ffaec103c70 msgr2=0x7ffaec106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.843+0000 7ffaf3800700 1 -- 172.21.15.45:0/3948193321 shutdown_connections 2024-04-25T18:46:10.652 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.843+0000 7ffaf3800700 1 -- 172.21.15.45:0/3948193321 wait complete. 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.844+0000 7ffaf3800700 1 Processor -- start 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.844+0000 7ffaf3800700 1 -- start start 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.844+0000 7ffaf3800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.844+0000 7ffaf3800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec109d40 con 0x7ffaec108610 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.845+0000 7ffaf0e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.845+0000 7ffaf0e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec19df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56846/0 (socket says 172.21.15.45:56846) 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.845+0000 7ffaf0e00700 1 -- 172.21.15.45:0/195044578 learned_addr learned my addr 172.21.15.45:0/195044578 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.845+0000 7ffaf0e00700 1 -- 172.21.15.45:0/195044578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec19e4c0 con 0x7ffaec108610 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.845+0000 7ffaf0e00700 1 --2- 172.21.15.45:0/195044578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec19df80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffadc004000 tx=0x7ffadc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.846+0000 7ffaea000700 1 -- 172.21.15.45:0/195044578 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffadc022020 con 0x7ffaec108610 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.846+0000 7ffaea000700 1 -- 172.21.15.45:0/195044578 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffadc018040 con 0x7ffaec108610 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.846+0000 7ffaf3800700 1 -- 172.21.15.45:0/195044578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec19e750 con 0x7ffaec108610 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.846+0000 7ffaea000700 1 -- 172.21.15.45:0/195044578 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffadc017070 con 0x7ffaec108610 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.846+0000 7ffaf3800700 1 -- 172.21.15.45:0/195044578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec1a1440 con 0x7ffaec108610 2024-04-25T18:46:10.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.846+0000 7ffaea000700 1 -- 172.21.15.45:0/195044578 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffadc025020 con 0x7ffaec108610 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.846+0000 7ffaea000700 1 -- 172.21.15.45:0/195044578 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffadc00f050 con 0x7ffaec108610 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.847+0000 7ffaf3800700 1 -- 172.21.15.45:0/195044578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaec0640a0 con 0x7ffaec108610 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.851+0000 7ffaea000700 1 -- 172.21.15.45:0/195044578 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ffadc02c030 con 0x7ffaec108610 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.931+0000 7ffaf3800700 1 -- 172.21.15.45:0/195044578 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7ffaec1a1960 con 0x7ffaec108610 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.932+0000 7ffaea000700 1 -- 172.21.15.45:0/195044578 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7ffadc01e480 con 0x7ffaec108610 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.933+0000 7ffaf3800700 1 -- 172.21.15.45:0/195044578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 msgr2=0x7ffaec19df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.933+0000 7ffaf3800700 1 --2- 172.21.15.45:0/195044578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec19df80 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ffadc004000 tx=0x7ffadc004ad0 comp rx=0 tx=0).stop 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.933+0000 7ffaf3800700 1 -- 172.21.15.45:0/195044578 shutdown_connections 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.933+0000 7ffaf3800700 1 --2- 172.21.15.45:0/195044578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec108610 0x7ffaec19df80 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.933+0000 7ffaf3800700 1 -- 172.21.15.45:0/195044578 >> 172.21.15.45:0/195044578 conn(0x7ffaec103c70 msgr2=0x7ffaec07d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.934+0000 7ffaf3800700 1 -- 172.21.15.45:0/195044578 shutdown_connections 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:09.934+0000 7ffaf3800700 1 -- 172.21.15.45:0/195044578 wait complete. 2024-04-25T18:46:10.654 INFO:teuthology.orchestra.run.smithi045.stdout:mon is available 2024-04-25T18:46:10.655 INFO:teuthology.orchestra.run.smithi045.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T18:46:11.677 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:11 smithi045 ceph-mon[27409]: from='client.? 172.21.15.45:0/2363971732' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T18:46:11.678 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:11 smithi045 ceph-mon[27409]: from='client.? 172.21.15.45:0/2363971732' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T18:46:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T18:46:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T18:46:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.45:3300,v1:172.21.15.45:6789] 2024-04-25T18:46:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T18:46:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T18:46:11.959 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.269+0000 7f66c1600700 1 Processor -- start 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.270+0000 7f66c1600700 1 -- start start 2024-04-25T18:46:11.960 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.270+0000 7f66c1600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.270+0000 7f66c1600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc108fd0 con 0x7f66bc1085e0 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.270+0000 7f66baa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.271+0000 7f66baa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56850/0 (socket says 172.21.15.45:56850) 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.271+0000 7f66baa00700 1 -- 172.21.15.45:0/1117049149 learned_addr learned my addr 172.21.15.45:0/1117049149 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.271+0000 7f66baa00700 1 -- 172.21.15.45:0/1117049149 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc1097e0 con 0x7f66bc1085e0 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.271+0000 7f66baa00700 1 --2- 172.21.15.45:0/1117049149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc108a00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0098a0 tx=0x7f66ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07cb19a92f0cc81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.272+0000 7f66b9600700 1 -- 172.21.15.45:0/1117049149 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66ac004160 con 0x7f66bc1085e0 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.272+0000 7f66b9600700 1 -- 172.21.15.45:0/1117049149 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac004300 con 0x7f66bc1085e0 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.272+0000 7f66b9600700 1 -- 172.21.15.45:0/1117049149 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66ac0036a0 con 0x7f66bc1085e0 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.272+0000 7f66c1600700 1 -- 172.21.15.45:0/1117049149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 msgr2=0x7f66bc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.272+0000 7f66c1600700 1 --2- 172.21.15.45:0/1117049149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc108a00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f66ac0098a0 tx=0x7f66ac009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.272+0000 7f66c1600700 1 -- 172.21.15.45:0/1117049149 shutdown_connections 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.272+0000 7f66c1600700 1 --2- 172.21.15.45:0/1117049149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc108a00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.272+0000 7f66c1600700 1 -- 172.21.15.45:0/1117049149 >> 172.21.15.45:0/1117049149 conn(0x7f66bc103ba0 msgr2=0x7f66bc105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:11.961 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.273+0000 7f66c1600700 1 -- 172.21.15.45:0/1117049149 shutdown_connections 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.273+0000 7f66c1600700 1 -- 172.21.15.45:0/1117049149 wait complete. 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.274+0000 7f66c1600700 1 Processor -- start 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.274+0000 7f66c1600700 1 -- start start 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.274+0000 7f66c1600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.274+0000 7f66c1600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66bc109d10 con 0x7f66bc1085e0 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.274+0000 7f66baa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.274+0000 7f66baa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc19de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56852/0 (socket says 172.21.15.45:56852) 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.274+0000 7f66baa00700 1 -- 172.21.15.45:0/2363971732 learned_addr learned my addr 172.21.15.45:0/2363971732 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.275+0000 7f66baa00700 1 -- 172.21.15.45:0/2363971732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66bc19e3a0 con 0x7f66bc1085e0 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.275+0000 7f66baa00700 1 --2- 172.21.15.45:0/2363971732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc19de60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f66ac000c00 tx=0x7f66ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.275+0000 7f66b3400700 1 -- 172.21.15.45:0/2363971732 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66ac003db0 con 0x7f66bc1085e0 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.275+0000 7f66b3400700 1 -- 172.21.15.45:0/2363971732 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66ac011630 con 0x7f66bc1085e0 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.275+0000 7f66b3400700 1 -- 172.21.15.45:0/2363971732 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66ac011950 con 0x7f66bc1085e0 2024-04-25T18:46:11.962 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.275+0000 7f66c1600700 1 -- 172.21.15.45:0/2363971732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66bc19e630 con 0x7f66bc1085e0 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.276+0000 7f66c1600700 1 -- 172.21.15.45:0/2363971732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66bc1a1320 con 0x7f66bc1085e0 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.276+0000 7f66b3400700 1 -- 172.21.15.45:0/2363971732 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f66ac011af0 con 0x7f66bc1085e0 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.276+0000 7f66b3400700 1 -- 172.21.15.45:0/2363971732 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f66ac01c450 con 0x7f66bc1085e0 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.277+0000 7f66c1600700 1 -- 172.21.15.45:0/2363971732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66bc0640a0 con 0x7f66bc1085e0 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.280+0000 7f66b3400700 1 -- 172.21.15.45:0/2363971732 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f66ac032020 con 0x7f66bc1085e0 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.358+0000 7f66c1600700 1 -- 172.21.15.45:0/2363971732 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f66bc1a14f0 con 0x7f66bc1085e0 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.363+0000 7f66b3400700 1 -- 172.21.15.45:0/2363971732 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f66ac004260 con 0x7f66bc1085e0 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.364+0000 7f66b3400700 1 -- 172.21.15.45:0/2363971732 <== mon.0 v2:172.21.15.45:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f66ac01c650 con 0x7f66bc1085e0 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.365+0000 7f66c1600700 1 -- 172.21.15.45:0/2363971732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 msgr2=0x7f66bc19de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.365+0000 7f66c1600700 1 --2- 172.21.15.45:0/2363971732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc19de60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f66ac000c00 tx=0x7f66ac004750 comp rx=0 tx=0).stop 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.365+0000 7f66c1600700 1 -- 172.21.15.45:0/2363971732 shutdown_connections 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.365+0000 7f66c1600700 1 --2- 172.21.15.45:0/2363971732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66bc1085e0 0x7f66bc19de60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.365+0000 7f66c1600700 1 -- 172.21.15.45:0/2363971732 >> 172.21.15.45:0/2363971732 conn(0x7f66bc103ba0 msgr2=0x7f66bc07e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.365+0000 7f66c1600700 1 -- 172.21.15.45:0/2363971732 shutdown_connections 2024-04-25T18:46:11.963 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:11.365+0000 7f66c1600700 1 -- 172.21.15.45:0/2363971732 wait complete. 2024-04-25T18:46:11.964 INFO:teuthology.orchestra.run.smithi045.stdout:Generating new minimal ceph.conf... 2024-04-25T18:46:12.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:12 smithi045 ceph-mon[27409]: from='client.? 172.21.15.45:0/2481541900' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:46:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.537+0000 7f7980e00700 1 Processor -- start 2024-04-25T18:46:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.537+0000 7f7980e00700 1 -- start start 2024-04-25T18:46:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.538+0000 7f7980e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c10ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.538+0000 7f7980e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f797c10b220 con 0x7f797c10a830 2024-04-25T18:46:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.539+0000 7f7979e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c10ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.539+0000 7f7979e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c10ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56856/0 (socket says 172.21.15.45:56856) 2024-04-25T18:46:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.539+0000 7f7979e00700 1 -- 172.21.15.45:0/1683380702 learned_addr learned my addr 172.21.15.45:0/1683380702 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.539+0000 7f7979e00700 1 -- 172.21.15.45:0/1683380702 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c10ba00 con 0x7f797c10a830 2024-04-25T18:46:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.540+0000 7f7979e00700 1 --2- 172.21.15.45:0/1683380702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c10ac50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f796c0098a0 tx=0x7f796c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a271bdcca903366e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:13.321 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.540+0000 7f7978a00700 1 -- 172.21.15.45:0/1683380702 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f796c004160 con 0x7f797c10a830 2024-04-25T18:46:13.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.540+0000 7f7978a00700 1 -- 172.21.15.45:0/1683380702 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f796c004300 con 0x7f797c10a830 2024-04-25T18:46:13.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.540+0000 7f7978a00700 1 -- 172.21.15.45:0/1683380702 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f796c011af0 con 0x7f797c10a830 2024-04-25T18:46:13.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.540+0000 7f7980e00700 1 -- 172.21.15.45:0/1683380702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 msgr2=0x7f797c10ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:13.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.541+0000 7f7980e00700 1 --2- 172.21.15.45:0/1683380702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c10ac50 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f796c0098a0 tx=0x7f796c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:13.325 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.541+0000 7f7980e00700 1 -- 172.21.15.45:0/1683380702 shutdown_connections 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.541+0000 7f7980e00700 1 --2- 172.21.15.45:0/1683380702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c10ac50 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.541+0000 7f7980e00700 1 -- 172.21.15.45:0/1683380702 >> 172.21.15.45:0/1683380702 conn(0x7f797c07dbc0 msgr2=0x7f797c07dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.541+0000 7f7980e00700 1 -- 172.21.15.45:0/1683380702 shutdown_connections 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.541+0000 7f7980e00700 1 -- 172.21.15.45:0/1683380702 wait complete. 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.542+0000 7f7980e00700 1 Processor -- start 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.542+0000 7f7980e00700 1 -- start start 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.542+0000 7f7980e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.542+0000 7f7980e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f797c10ec30 con 0x7f797c10a830 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.543+0000 7f7979e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.543+0000 7f7979e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56862/0 (socket says 172.21.15.45:56862) 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.543+0000 7f7979e00700 1 -- 172.21.15.45:0/2481541900 learned_addr learned my addr 172.21.15.45:0/2481541900 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.543+0000 7f7979e00700 1 -- 172.21.15.45:0/2481541900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f797c19ecc0 con 0x7f797c10a830 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.543+0000 7f7979e00700 1 --2- 172.21.15.45:0/2481541900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c19e780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f796c0117b0 tx=0x7f796c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:13.326 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.544+0000 7f7972a00700 1 -- 172.21.15.45:0/2481541900 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f796c003af0 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.544+0000 7f7972a00700 1 -- 172.21.15.45:0/2481541900 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f796c020070 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.544+0000 7f7972a00700 1 -- 172.21.15.45:0/2481541900 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f796c019630 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.544+0000 7f7980e00700 1 -- 172.21.15.45:0/2481541900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797c19ef50 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.544+0000 7f7980e00700 1 -- 172.21.15.45:0/2481541900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f797c1a1c40 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.544+0000 7f7972a00700 1 -- 172.21.15.45:0/2481541900 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f796c019c30 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.544+0000 7f7972a00700 1 -- 172.21.15.45:0/2481541900 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f796c00b720 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.545+0000 7f7980e00700 1 -- 172.21.15.45:0/2481541900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f797c10edb0 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.548+0000 7f7972a00700 1 -- 172.21.15.45:0/2481541900 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f796c0042c0 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.627+0000 7f7980e00700 1 -- 172.21.15.45:0/2481541900 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f797c1a2170 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.628+0000 7f7972a00700 1 -- 172.21.15.45:0/2481541900 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f796c01a9c0 con 0x7f797c10a830 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.629+0000 7f7980e00700 1 -- 172.21.15.45:0/2481541900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 msgr2=0x7f797c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.629+0000 7f7980e00700 1 --2- 172.21.15.45:0/2481541900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c19e780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f796c0117b0 tx=0x7f796c011b70 comp rx=0 tx=0).stop 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.629+0000 7f7980e00700 1 -- 172.21.15.45:0/2481541900 shutdown_connections 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.629+0000 7f7980e00700 1 --2- 172.21.15.45:0/2481541900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f797c10a830 0x7f797c19e780 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:13.327 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.629+0000 7f7980e00700 1 -- 172.21.15.45:0/2481541900 >> 172.21.15.45:0/2481541900 conn(0x7f797c07dbc0 msgr2=0x7f797c1085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:13.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.629+0000 7f7980e00700 1 -- 172.21.15.45:0/2481541900 shutdown_connections 2024-04-25T18:46:13.328 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:12.629+0000 7f7980e00700 1 -- 172.21.15.45:0/2481541900 wait complete. 2024-04-25T18:46:13.328 INFO:teuthology.orchestra.run.smithi045.stdout:Restarting the monitor... 2024-04-25T18:46:13.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:13 smithi045 systemd[1]: Stopping Ceph mon.a for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:46:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:13 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a[27405]: 2024-04-25T18:46:13.847+0000 7f5e84800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T18:46:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:13 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a[27405]: 2024-04-25T18:46:13.847+0000 7f5e84800700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T18:46:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:13 smithi045 podman[27651]: 2024-04-25 18:46:13.86847891 +0000 UTC m=+0.149135822 container died a8cfa50b2c7c65c5b7d8a289471c1f97afccf42fae08bb836759406a895b6dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T18:46:14.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:14 smithi045 podman[27651]: 2024-04-25 18:46:14.480982744 +0000 UTC m=+0.761639685 container cleanup a8cfa50b2c7c65c5b7d8a289471c1f97afccf42fae08bb836759406a895b6dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T18:46:14.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:14 smithi045 bash[27651]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a 2024-04-25T18:46:15.161 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:14 smithi045 podman[27665]: 2024-04-25 18:46:14.748800347 +0000 UTC m=+0.862349653 container remove a8cfa50b2c7c65c5b7d8a289471c1f97afccf42fae08bb836759406a895b6dfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-25T18:46:15.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 systemd[1]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T18:46:15.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 systemd[1]: Stopped Ceph mon.a for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:46:15.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 systemd[1]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.908s CPU time. 2024-04-25T18:46:15.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 systemd[1]: Starting Ceph mon.a for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:46:15.865 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 podman[27757]: 2024-04-25 18:46:15.599801267 +0000 UTC m=+0.129794947 container create c3ffc389232cce19ad2773b68cccb69a8a68ecdc6b27de49c81eccefb3ad6d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T18:46:15.865 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 podman[27757]: 2024-04-25 18:46:15.506871462 +0000 UTC m=+0.036865124 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:46:15.865 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 podman[27757]: 2024-04-25 18:46:15.863258922 +0000 UTC m=+0.393252659 container init c3ffc389232cce19ad2773b68cccb69a8a68ecdc6b27de49c81eccefb3ad6d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T18:46:15.963 INFO:teuthology.orchestra.run.smithi045.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 podman[27757]: 2024-04-25 18:46:15.872740284 +0000 UTC m=+0.402733991 container start c3ffc389232cce19ad2773b68cccb69a8a68ecdc6b27de49c81eccefb3ad6d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: pidfile_write: ignore empty --pid-file 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: load: jerasure load: lrc 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: RocksDB version: 7.9.2 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Git sha 0 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: DB SUMMARY 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: DB Session ID: PWOEPFA328Y96KFMAPXF 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: CURRENT file: CURRENT 2024-04-25T18:46:16.210 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106034 ; 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.error_if_exists: 0 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.create_if_missing: 0 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.paranoid_checks: 1 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.env: 0x55ef3e96ecc0 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.info_log: 0x55ef48fadc40 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.statistics: (nil) 2024-04-25T18:46:16.211 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.use_fsync: 0 2024-04-25T18:46:16.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_log_file_size: 0 2024-04-25T18:46:16.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T18:46:16.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T18:46:16.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T18:46:16.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T18:46:16.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.allow_fallocate: 1 2024-04-25T18:46:16.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T18:46:16.212 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.use_direct_reads: 0 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.db_log_dir: 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.wal_dir: 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T18:46:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.write_buffer_manager: 0x55ef48f410e0 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.unordered_write: 0 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T18:46:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.row_cache: None 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.wal_filter: None 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.two_write_queues: 0 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.wal_compression: 0 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.atomic_flush: 0 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.log_readahead_size: 0 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T18:46:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_background_jobs: 2 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_background_compactions: -1 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_subcompactions: 1 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T18:46:16.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_open_files: -1 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_background_flushes: -1 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Compression algorithms supported: 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: kZSTD supported: 0 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: kXpressCompression supported: 0 2024-04-25T18:46:16.217 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T18:46:16.218 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: kZlibCompression supported: 1 2024-04-25T18:46:16.218 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: kSnappyCompression supported: 1 2024-04-25T18:46:16.218 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: kLZ4Compression supported: 1 2024-04-25T18:46:16.218 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: kBZip2Compression supported: 0 2024-04-25T18:46:16.218 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T18:46:16.218 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T18:46:16.218 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T18:46:16.218 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T18:46:16.218 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T18:46:16.218 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.merge_operator: 2024-04-25T18:46:16.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_filter: None 2024-04-25T18:46:16.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T18:46:16.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T18:46:16.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T18:46:16.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T18:46:16.219 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ef48fade00) 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks: 1 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: pin_top_level_index_and_filter: 1 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_type: 0 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_index_type: 0 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_shortening: 1 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: checksum: 4 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: no_block_cache: 0 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache: 0x55ef48f731f0 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_name: BinnedLRUCache 2024-04-25T18:46:16.220 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_options: 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: capacity : 536870912 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_shard_bits : 4 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: strict_capacity_limit : 0 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: high_pri_pool_ratio: 0.000 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_cache_compressed: (nil) 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: persistent_cache: (nil) 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size: 4096 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_size_deviation: 10 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_restart_interval: 16 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: index_block_restart_interval: 1 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: metadata_block_size: 4096 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: partition_filters: 0 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: use_delta_encoding: 1 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: filter_policy: bloomfilter 2024-04-25T18:46:16.221 INFO:journalctl@ceph.mon.a.smithi045.stdout: whole_key_filtering: 1 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout: verify_compression: 0 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout: read_amp_bytes_per_bit: 0 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout: format_version: 5 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_index_compression: 1 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout: block_align: 0 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout: max_auto_readahead_size: 262144 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout: prepopulate_block_cache: 0 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout: initial_auto_readahead_size: 8192 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compression: NoCompression 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T18:46:16.222 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.num_levels: 7 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T18:46:16.223 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T18:46:16.224 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T18:46:16.225 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.inplace_update_support: 0 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T18:46:16.226 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.bloom_locality: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.max_successive_merges: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.ttl: 2592000 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.enable_blob_files: false 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.min_blob_size: 0 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T18:46:16.227 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ba5d0a2b-a52b-44df-8870-23ad260d22f3 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070775934205, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070775937153, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101086, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98975, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 91154, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714070775, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ba5d0a2b-a52b-44df-8870-23ad260d22f3", "db_session_id": "PWOEPFA328Y96KFMAPXF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070775937404, "job": 1, "event": "recovery_finished"} 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T18:46:16.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ef49088000 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: DB pointer 0x55ef49072000 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** DB Stats ** 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-04-25T18:46:16.229 INFO:journalctl@ceph.mon.a.smithi045.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: L0 2/0 100.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 56.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: Sum 2/0 100.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 56.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 56.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** Compaction Stats [default] ** 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 56.7 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T18:46:16.230 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: Cumulative compaction: 0.00 GB write, 9.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: Interval compaction: 0.00 GB write, 9.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache BinnedLRUCache@0x55ef48f731f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: starting mon.a rank 0 at public addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] at bind addrs [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: mon.a@-1(???) e1 preinit fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: mon.a@-1(???).mds e1 new map 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: mon.a@-1(???).mds e1 print_map 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: e1 2024-04-25T18:46:16.231 INFO:journalctl@ceph.mon.a.smithi045.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout: legacy client fscid: -1 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout: 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout: No filesystems configured 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 ceph-mon[27776]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 bash[27757]: c3ffc389232cce19ad2773b68cccb69a8a68ecdc6b27de49c81eccefb3ad6d10 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:15 smithi045 systemd[1]: Started Ceph mon.a for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: monmap epoch 1 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: last_changed 2024-04-25T18:46:04.854399+0000 2024-04-25T18:46:16.232 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:46:16.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: min_mon_release 19 (squid) 2024-04-25T18:46:16.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: election_strategy: 1 2024-04-25T18:46:16.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:46:16.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: fsmap 2024-04-25T18:46:16.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T18:46:16.233 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:16 smithi045 ceph-mon[27776]: mgrmap e1: no daemons active 2024-04-25T18:46:17.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.537+0000 7fbd2b000700 1 Processor -- start 2024-04-25T18:46:17.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.537+0000 7fbd2b000700 1 -- start start 2024-04-25T18:46:17.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.538+0000 7fbd2b000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd24108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:17.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.538+0000 7fbd2b000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbd24109060 con 0x7fbd24108670 2024-04-25T18:46:17.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.538+0000 7fbd23e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd24108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:17.314 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.538+0000 7fbd23e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd24108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55124/0 (socket says 172.21.15.45:55124) 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.538+0000 7fbd23e00700 1 -- 172.21.15.45:0/214086731 learned_addr learned my addr 172.21.15.45:0/214086731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.539+0000 7fbd23e00700 1 -- 172.21.15.45:0/214086731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd24109820 con 0x7fbd24108670 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.539+0000 7fbd23e00700 1 --2- 172.21.15.45:0/214086731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd24108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbd140098a0 tx=0x7fbd14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7f424bd9b9d5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.540+0000 7fbd22a00700 1 -- 172.21.15.45:0/214086731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd14004160 con 0x7fbd24108670 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.540+0000 7fbd22a00700 1 -- 172.21.15.45:0/214086731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd14004300 con 0x7fbd24108670 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.540+0000 7fbd22a00700 1 -- 172.21.15.45:0/214086731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd14011af0 con 0x7fbd24108670 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.540+0000 7fbd2b000700 1 -- 172.21.15.45:0/214086731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 msgr2=0x7fbd24108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.540+0000 7fbd2b000700 1 --2- 172.21.15.45:0/214086731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd24108a90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbd140098a0 tx=0x7fbd14009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.541+0000 7fbd2b000700 1 -- 172.21.15.45:0/214086731 shutdown_connections 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.541+0000 7fbd2b000700 1 --2- 172.21.15.45:0/214086731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd24108a90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.541+0000 7fbd2b000700 1 -- 172.21.15.45:0/214086731 >> 172.21.15.45:0/214086731 conn(0x7fbd24103c30 msgr2=0x7fbd24106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.541+0000 7fbd2b000700 1 -- 172.21.15.45:0/214086731 shutdown_connections 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.541+0000 7fbd2b000700 1 -- 172.21.15.45:0/214086731 wait complete. 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.542+0000 7fbd2b000700 1 Processor -- start 2024-04-25T18:46:17.315 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.542+0000 7fbd2b000700 1 -- start start 2024-04-25T18:46:17.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.542+0000 7fbd2b000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd2419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:17.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.543+0000 7fbd2b000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2410ca50 con 0x7fbd24108670 2024-04-25T18:46:17.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.543+0000 7fbd23e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd2419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:17.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.543+0000 7fbd23e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd2419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55136/0 (socket says 172.21.15.45:55136) 2024-04-25T18:46:17.316 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.543+0000 7fbd23e00700 1 -- 172.21.15.45:0/1667439229 learned_addr learned my addr 172.21.15.45:0/1667439229 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:17.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.543+0000 7fbd23e00700 1 -- 172.21.15.45:0/1667439229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2419ecf0 con 0x7fbd24108670 2024-04-25T18:46:17.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.544+0000 7fbd23e00700 1 --2- 172.21.15.45:0/1667439229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd2419e7b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbd140117b0 tx=0x7fbd14011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:17.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.544+0000 7fbd20c00700 1 -- 172.21.15.45:0/1667439229 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd14003af0 con 0x7fbd24108670 2024-04-25T18:46:17.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.544+0000 7fbd20c00700 1 -- 172.21.15.45:0/1667439229 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbd14020070 con 0x7fbd24108670 2024-04-25T18:46:17.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.544+0000 7fbd20c00700 1 -- 172.21.15.45:0/1667439229 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd14019630 con 0x7fbd24108670 2024-04-25T18:46:17.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.544+0000 7fbd2b000700 1 -- 172.21.15.45:0/1667439229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2419ef80 con 0x7fbd24108670 2024-04-25T18:46:17.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.544+0000 7fbd2b000700 1 -- 172.21.15.45:0/1667439229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd241a1c70 con 0x7fbd24108670 2024-04-25T18:46:17.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.545+0000 7fbd20c00700 1 -- 172.21.15.45:0/1667439229 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd14019c80 con 0x7fbd24108670 2024-04-25T18:46:17.318 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.545+0000 7fbd20c00700 1 -- 172.21.15.45:0/1667439229 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbd1400b720 con 0x7fbd24108670 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.545+0000 7fbd2b000700 1 -- 172.21.15.45:0/1667439229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd2410cbd0 con 0x7fbd24108670 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.549+0000 7fbd20c00700 1 -- 172.21.15.45:0/1667439229 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fbd140042a0 con 0x7fbd24108670 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.628+0000 7fbd2b000700 1 -- 172.21.15.45:0/1667439229 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fbd241a2300 con 0x7fbd24108670 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.631+0000 7fbd20c00700 1 -- 172.21.15.45:0/1667439229 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fbd1401a9c0 con 0x7fbd24108670 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.633+0000 7fbd2b000700 1 -- 172.21.15.45:0/1667439229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 msgr2=0x7fbd2419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.633+0000 7fbd2b000700 1 --2- 172.21.15.45:0/1667439229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd2419e7b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbd140117b0 tx=0x7fbd14011b70 comp rx=0 tx=0).stop 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.633+0000 7fbd2b000700 1 -- 172.21.15.45:0/1667439229 shutdown_connections 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.633+0000 7fbd2b000700 1 --2- 172.21.15.45:0/1667439229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd24108670 0x7fbd2419e7b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.633+0000 7fbd2b000700 1 -- 172.21.15.45:0/1667439229 >> 172.21.15.45:0/1667439229 conn(0x7fbd24103c30 msgr2=0x7fbd241046f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.633+0000 7fbd2b000700 1 -- 172.21.15.45:0/1667439229 shutdown_connections 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:16.633+0000 7fbd2b000700 1 -- 172.21.15.45:0/1667439229 wait complete. 2024-04-25T18:46:17.319 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T18:46:17.342 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:46:17.343 INFO:teuthology.orchestra.run.smithi045.stdout:Creating mgr... 2024-04-25T18:46:17.344 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T18:46:17.345 INFO:teuthology.orchestra.run.smithi045.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T18:46:17.781 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.x 2024-04-25T18:46:17.781 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T18:46:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:17 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1667439229' entity='client.admin' 2024-04-25T18:46:18.041 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ea2833ba-0333-11ef-bc93-c7b262605968.target.wants/ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ea2833ba-0333-11ef-bc93-c7b262605968@.service. 2024-04-25T18:46:18.446 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:18 smithi045 systemd[1]: Starting Ceph mgr.x for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:46:18.715 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:18 smithi045 podman[28004]: 2024-04-25 18:46:18.444478309 +0000 UTC m=+0.123925683 container create ffacc8ad311ab730175a2f6385858bbc0add9a209f7f335b2cee9d66ba55bf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T18:46:18.716 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:18 smithi045 podman[28004]: 2024-04-25 18:46:18.352112626 +0000 UTC m=+0.031559958 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T18:46:19.077 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:46:19.077 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-25T18:46:19.092 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:46:19.093 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T18:46:19.093 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-25T18:46:19.093 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T18:46:19.114 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:46:19.115 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-25T18:46:19.134 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:46:19.135 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T18:46:19.135 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-25T18:46:19.135 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T18:46:19.135 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr to start... 2024-04-25T18:46:19.135 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr... 2024-04-25T18:46:19.202 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:18 smithi045 podman[28004]: 2024-04-25 18:46:18.948415575 +0000 UTC m=+0.627862972 container init ffacc8ad311ab730175a2f6385858bbc0add9a209f7f335b2cee9d66ba55bf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T18:46:19.225 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:18 smithi045 podman[28004]: 2024-04-25 18:46:18.960154166 +0000 UTC m=+0.639601526 container start ffacc8ad311ab730175a2f6385858bbc0add9a209f7f335b2cee9d66ba55bf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T18:46:19.225 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:19 smithi045 bash[28004]: ffacc8ad311ab730175a2f6385858bbc0add9a209f7f335b2cee9d66ba55bf8f 2024-04-25T18:46:19.225 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:19 smithi045 systemd[1]: Started Ceph mgr.x for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:46:19.456 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:19.201+0000 7fb49fb72200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:46:19.456 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:19.454+0000 7fb49fb72200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:46:19.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:19.597+0000 7fb49fb72200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:46:19.968 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:19 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3739011672' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:46:20.357 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:20 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:20.026+0000 7fb49fb72200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:46:20.357 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:20 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:20.143+0000 7fb49fb72200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:46:20.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:20.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:46:20.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:46:20.653 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:20.654 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:46:20.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:46:20.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:20.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:46:20.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:20.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:46:20.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:46:20.655 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:46:20.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:46:20.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:46:20.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:46:20.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:20.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:46:20.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:46:20.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:46:20.662 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:46:20.663 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:46:09.148241+0000", 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.803+0000 7f617ee00700 1 Processor -- start 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.803+0000 7f617ee00700 1 -- start start 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.803+0000 7f617ee00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617810b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:20.664 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.804+0000 7f617ee00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6178063d00 con 0x7f6178109110 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.804+0000 7f6177e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617810b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.804+0000 7f6177e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617810b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55152/0 (socket says 172.21.15.45:55152) 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.804+0000 7f6177e00700 1 -- 172.21.15.45:0/294683060 learned_addr learned my addr 172.21.15.45:0/294683060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.805+0000 7f6177e00700 1 -- 172.21.15.45:0/294683060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617810ba40 con 0x7f6178109110 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.805+0000 7f6177e00700 1 --2- 172.21.15.45:0/294683060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617810b500 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61680098a0 tx=0x7f6168009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3928699c366ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.805+0000 7f6176a00700 1 -- 172.21.15.45:0/294683060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6168004160 con 0x7f6178109110 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.805+0000 7f6176a00700 1 -- 172.21.15.45:0/294683060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6168004300 con 0x7f6178109110 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.805+0000 7f6176a00700 1 -- 172.21.15.45:0/294683060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6168011af0 con 0x7f6178109110 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.806+0000 7f617ee00700 1 -- 172.21.15.45:0/294683060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 msgr2=0x7f617810b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.806+0000 7f617ee00700 1 --2- 172.21.15.45:0/294683060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617810b500 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61680098a0 tx=0x7f6168009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.806+0000 7f617ee00700 1 -- 172.21.15.45:0/294683060 shutdown_connections 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.806+0000 7f617ee00700 1 --2- 172.21.15.45:0/294683060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617810b500 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.806+0000 7f617ee00700 1 -- 172.21.15.45:0/294683060 >> 172.21.15.45:0/294683060 conn(0x7f6178102b80 msgr2=0x7f6178104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.806+0000 7f617ee00700 1 -- 172.21.15.45:0/294683060 shutdown_connections 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.806+0000 7f617ee00700 1 -- 172.21.15.45:0/294683060 wait complete. 2024-04-25T18:46:20.665 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.807+0000 7f617ee00700 1 Processor -- start 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.807+0000 7f617ee00700 1 -- start start 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.808+0000 7f617ee00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.808+0000 7f617ee00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f617810c940 con 0x7f6178109110 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.808+0000 7f6177e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.808+0000 7f6177e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617819c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55166/0 (socket says 172.21.15.45:55166) 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.808+0000 7f6177e00700 1 -- 172.21.15.45:0/3739011672 learned_addr learned my addr 172.21.15.45:0/3739011672 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.808+0000 7f6177e00700 1 -- 172.21.15.45:0/3739011672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f617819ca60 con 0x7f6178109110 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.809+0000 7f6177e00700 1 --2- 172.21.15.45:0/3739011672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617819c520 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6168009fb0 tx=0x7f6168011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.809+0000 7f6174c00700 1 -- 172.21.15.45:0/3739011672 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61680036a0 con 0x7f6178109110 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.809+0000 7f6174c00700 1 -- 172.21.15.45:0/3739011672 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f616801a430 con 0x7f6178109110 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.809+0000 7f617ee00700 1 -- 172.21.15.45:0/3739011672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f617819ccf0 con 0x7f6178109110 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.809+0000 7f6174c00700 1 -- 172.21.15.45:0/3739011672 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6168019550 con 0x7f6178109110 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.809+0000 7f617ee00700 1 -- 172.21.15.45:0/3739011672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f617819f9e0 con 0x7f6178109110 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.810+0000 7f6174c00700 1 -- 172.21.15.45:0/3739011672 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6168019b90 con 0x7f6178109110 2024-04-25T18:46:20.666 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.810+0000 7f6174c00700 1 -- 172.21.15.45:0/3739011672 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6168024730 con 0x7f6178109110 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.810+0000 7f617ee00700 1 -- 172.21.15.45:0/3739011672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61780640a0 con 0x7f6178109110 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.813+0000 7f6174c00700 1 -- 172.21.15.45:0/3739011672 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6168043e50 con 0x7f6178109110 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.893+0000 7f617ee00700 1 -- 172.21.15.45:0/3739011672 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f617819ff30 con 0x7f6178109110 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.894+0000 7f6174c00700 1 -- 172.21.15.45:0/3739011672 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6168032030 con 0x7f6178109110 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.895+0000 7f617ee00700 1 -- 172.21.15.45:0/3739011672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 msgr2=0x7f617819c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.895+0000 7f617ee00700 1 --2- 172.21.15.45:0/3739011672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617819c520 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6168009fb0 tx=0x7f6168011940 comp rx=0 tx=0).stop 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.895+0000 7f617ee00700 1 -- 172.21.15.45:0/3739011672 shutdown_connections 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.895+0000 7f617ee00700 1 --2- 172.21.15.45:0/3739011672 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6178109110 0x7f617819c520 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.895+0000 7f617ee00700 1 -- 172.21.15.45:0/3739011672 >> 172.21.15.45:0/3739011672 conn(0x7f6178102b80 msgr2=0x7f61781036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.895+0000 7f617ee00700 1 -- 172.21.15.45:0/3739011672 shutdown_connections 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:19.895+0000 7f617ee00700 1 -- 172.21.15.45:0/3739011672 wait complete. 2024-04-25T18:46:20.667 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (1/15)... 2024-04-25T18:46:22.213 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:21 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:21.767+0000 7fb49fb72200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:46:22.213 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:21 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:21.998+0000 7fb49fb72200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:46:22.610 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:22 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:22.216+0000 7fb49fb72200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:46:22.610 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:22 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:22.608+0000 7fb49fb72200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:46:22.960 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:22 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:22.739+0000 7fb49fb72200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:46:23.213 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:23 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:23.000+0000 7fb49fb72200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:46:23.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:23 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/252136363' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:46:23.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:23 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:23.267+0000 7fb49fb72200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:46:24.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:24.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:46:24.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:46:24.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:46:24.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:46:24.028 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:46:24.030 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:46:24.030 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:24.030 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:46:24.030 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:46:24.031 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:24.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:46:24.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:46:24.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:46:24.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:46:24.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:46:24.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:46:24.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:24.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:46:24.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:46:24.033 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:46:24.035 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T18:46:24.036 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:46:24.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:46:24.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:46:24.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:46:24.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:46:24.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:24.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:46:24.037 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:46:09.148241+0000", 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.279+0000 7ffbd8200700 1 Processor -- start 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.280+0000 7ffbd8200700 1 -- start start 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.280+0000 7ffbd8200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd010b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.280+0000 7ffbd8200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd0063ca0 con 0x7ffbd0109170 2024-04-25T18:46:24.041 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.280+0000 7ffbd5800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd010b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.280+0000 7ffbd5800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd010b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55176/0 (socket says 172.21.15.45:55176) 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.280+0000 7ffbd5800700 1 -- 172.21.15.45:0/4014660068 learned_addr learned my addr 172.21.15.45:0/4014660068 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.281+0000 7ffbd5800700 1 -- 172.21.15.45:0/4014660068 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd010baa0 con 0x7ffbd0109170 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.281+0000 7ffbd5800700 1 --2- 172.21.15.45:0/4014660068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd010b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40098a0 tx=0x7ffbc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1b5f41d5ac1574 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.281+0000 7ffbcfe00700 1 -- 172.21.15.45:0/4014660068 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbc4004160 con 0x7ffbd0109170 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.282+0000 7ffbcfe00700 1 -- 172.21.15.45:0/4014660068 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffbc4004300 con 0x7ffbd0109170 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.282+0000 7ffbcfe00700 1 -- 172.21.15.45:0/4014660068 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbc4011af0 con 0x7ffbd0109170 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.282+0000 7ffbd8200700 1 -- 172.21.15.45:0/4014660068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 msgr2=0x7ffbd010b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.282+0000 7ffbd8200700 1 --2- 172.21.15.45:0/4014660068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd010b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40098a0 tx=0x7ffbc4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.282+0000 7ffbd8200700 1 -- 172.21.15.45:0/4014660068 shutdown_connections 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.282+0000 7ffbd8200700 1 --2- 172.21.15.45:0/4014660068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd010b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.282+0000 7ffbd8200700 1 -- 172.21.15.45:0/4014660068 >> 172.21.15.45:0/4014660068 conn(0x7ffbd0102b80 msgr2=0x7ffbd0104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.282+0000 7ffbd8200700 1 -- 172.21.15.45:0/4014660068 shutdown_connections 2024-04-25T18:46:24.042 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.283+0000 7ffbd8200700 1 -- 172.21.15.45:0/4014660068 wait complete. 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.284+0000 7ffbd8200700 1 Processor -- start 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.284+0000 7ffbd8200700 1 -- start start 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.285+0000 7ffbd8200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.285+0000 7ffbd8200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd010c940 con 0x7ffbd0109170 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.285+0000 7ffbd5800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.285+0000 7ffbd5800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55178/0 (socket says 172.21.15.45:55178) 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.285+0000 7ffbd5800700 1 -- 172.21.15.45:0/252136363 learned_addr learned my addr 172.21.15.45:0/252136363 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.285+0000 7ffbd5800700 1 -- 172.21.15.45:0/252136363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd019ca60 con 0x7ffbd0109170 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.286+0000 7ffbd5800700 1 --2- 172.21.15.45:0/252136363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd019c520 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4011ac0 tx=0x7ffbc4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.286+0000 7ffbce000700 1 -- 172.21.15.45:0/252136363 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbc4003970 con 0x7ffbd0109170 2024-04-25T18:46:24.043 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.286+0000 7ffbce000700 1 -- 172.21.15.45:0/252136363 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffbc4003b10 con 0x7ffbd0109170 2024-04-25T18:46:24.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.286+0000 7ffbd8200700 1 -- 172.21.15.45:0/252136363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd019ccf0 con 0x7ffbd0109170 2024-04-25T18:46:24.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.286+0000 7ffbce000700 1 -- 172.21.15.45:0/252136363 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbc4024610 con 0x7ffbd0109170 2024-04-25T18:46:24.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.286+0000 7ffbd8200700 1 -- 172.21.15.45:0/252136363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd019f9e0 con 0x7ffbd0109170 2024-04-25T18:46:24.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.287+0000 7ffbce000700 1 -- 172.21.15.45:0/252136363 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffbc4024bd0 con 0x7ffbd0109170 2024-04-25T18:46:24.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.287+0000 7ffbce000700 1 -- 172.21.15.45:0/252136363 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffbc4019700 con 0x7ffbd0109170 2024-04-25T18:46:24.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.287+0000 7ffbd8200700 1 -- 172.21.15.45:0/252136363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbd00640a0 con 0x7ffbd0109170 2024-04-25T18:46:24.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.290+0000 7ffbce000700 1 -- 172.21.15.45:0/252136363 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ffbc4043e50 con 0x7ffbd0109170 2024-04-25T18:46:24.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.368+0000 7ffbd8200700 1 -- 172.21.15.45:0/252136363 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffbd019fbb0 con 0x7ffbd0109170 2024-04-25T18:46:24.045 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.369+0000 7ffbce000700 1 -- 172.21.15.45:0/252136363 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ffbc4032030 con 0x7ffbd0109170 2024-04-25T18:46:24.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.370+0000 7ffbd8200700 1 -- 172.21.15.45:0/252136363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 msgr2=0x7ffbd019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:24.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.370+0000 7ffbd8200700 1 --2- 172.21.15.45:0/252136363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd019c520 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4011ac0 tx=0x7ffbc4011ba0 comp rx=0 tx=0).stop 2024-04-25T18:46:24.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.370+0000 7ffbd8200700 1 -- 172.21.15.45:0/252136363 shutdown_connections 2024-04-25T18:46:24.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.370+0000 7ffbd8200700 1 --2- 172.21.15.45:0/252136363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffbd0109170 0x7ffbd019c520 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:24.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.370+0000 7ffbd8200700 1 -- 172.21.15.45:0/252136363 >> 172.21.15.45:0/252136363 conn(0x7ffbd0102b80 msgr2=0x7ffbd01036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:24.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.370+0000 7ffbd8200700 1 -- 172.21.15.45:0/252136363 shutdown_connections 2024-04-25T18:46:24.046 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:23.370+0000 7ffbd8200700 1 -- 172.21.15.45:0/252136363 wait complete. 2024-04-25T18:46:24.046 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (2/15)... 2024-04-25T18:46:24.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:24.088+0000 7fb49fb72200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:46:24.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:24.659+0000 7fb49fb72200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:46:24.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:24.788+0000 7fb49fb72200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:46:25.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:25 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:25.061+0000 7fb49fb72200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:46:26.188 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:25 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:25.724+0000 7fb49fb72200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:46:26.188 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:25 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:25.851+0000 7fb49fb72200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:46:26.188 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:26 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:26.186+0000 7fb49fb72200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:46:26.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:26 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:26.374+0000 7fb49fb72200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:46:26.740 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:26 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:26.504+0000 7fb49fb72200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:46:26.740 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:26 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:26.665+0000 7fb49fb72200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:46:27.047 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:26 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1288565147' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:46:27.395 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:27 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:27.045+0000 7fb49fb72200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:46:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:46:27.428 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:27.428 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:46:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:46:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-25T18:46:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:46:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:46:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:46:27.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:27.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:46:27.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:27.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:46:27.430 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:46:27.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:46:27.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:46:27.438 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:46:27.439 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:46:27.440 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:46:09.148241+0000", 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.645+0000 7f1068000700 1 Processor -- start 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.645+0000 7f1068000700 1 -- start start 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.646+0000 7f1068000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f10601072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.646+0000 7f1068000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1060063d60 con 0x7f1060106ea0 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.646+0000 7f1065600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f10601072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.646+0000 7f1065600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f10601072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60876/0 (socket says 172.21.15.45:60876) 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.646+0000 7f1065600700 1 -- 172.21.15.45:0/3438817242 learned_addr learned my addr 172.21.15.45:0/3438817242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.647+0000 7f1065600700 1 -- 172.21.15.45:0/3438817242 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1060107800 con 0x7f1060106ea0 2024-04-25T18:46:27.441 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.647+0000 7f1065600700 1 --2- 172.21.15.45:0/3438817242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f10601072c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f10540098a0 tx=0x7f1054009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b47de05096516db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.647+0000 7f105fe00700 1 -- 172.21.15.45:0/3438817242 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1054004160 con 0x7f1060106ea0 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.648+0000 7f105fe00700 1 -- 172.21.15.45:0/3438817242 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1054004300 con 0x7f1060106ea0 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.648+0000 7f105fe00700 1 -- 172.21.15.45:0/3438817242 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1054011af0 con 0x7f1060106ea0 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.648+0000 7f1068000700 1 -- 172.21.15.45:0/3438817242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 msgr2=0x7f10601072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.648+0000 7f1068000700 1 --2- 172.21.15.45:0/3438817242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f10601072c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f10540098a0 tx=0x7f1054009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.648+0000 7f1068000700 1 -- 172.21.15.45:0/3438817242 shutdown_connections 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.648+0000 7f1068000700 1 --2- 172.21.15.45:0/3438817242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f10601072c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.648+0000 7f1068000700 1 -- 172.21.15.45:0/3438817242 >> 172.21.15.45:0/3438817242 conn(0x7f1060102b80 msgr2=0x7f1060104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.649+0000 7f1068000700 1 -- 172.21.15.45:0/3438817242 shutdown_connections 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.649+0000 7f1068000700 1 -- 172.21.15.45:0/3438817242 wait complete. 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.650+0000 7f1068000700 1 Processor -- start 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.650+0000 7f1068000700 1 -- start start 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.650+0000 7f1068000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f106019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:27.442 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.650+0000 7f1068000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10601087c0 con 0x7f1060106ea0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.651+0000 7f1065600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f106019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.651+0000 7f1065600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f106019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60880/0 (socket says 172.21.15.45:60880) 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.651+0000 7f1065600700 1 -- 172.21.15.45:0/1288565147 learned_addr learned my addr 172.21.15.45:0/1288565147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.651+0000 7f1065600700 1 -- 172.21.15.45:0/1288565147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f106019ec60 con 0x7f1060106ea0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.651+0000 7f1065600700 1 --2- 172.21.15.45:0/1288565147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f106019e720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1054011ac0 tx=0x7f1054011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.652+0000 7f105e000700 1 -- 172.21.15.45:0/1288565147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1054003970 con 0x7f1060106ea0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.652+0000 7f105e000700 1 -- 172.21.15.45:0/1288565147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1054003b10 con 0x7f1060106ea0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.652+0000 7f1068000700 1 -- 172.21.15.45:0/1288565147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106019eef0 con 0x7f1060106ea0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.652+0000 7f105e000700 1 -- 172.21.15.45:0/1288565147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1054024610 con 0x7f1060106ea0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.652+0000 7f1068000700 1 -- 172.21.15.45:0/1288565147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10601a1be0 con 0x7f1060106ea0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.652+0000 7f105e000700 1 -- 172.21.15.45:0/1288565147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1054024bd0 con 0x7f1060106ea0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.652+0000 7f105e000700 1 -- 172.21.15.45:0/1288565147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1054019700 con 0x7f1060106ea0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.653+0000 7f1068000700 1 -- 172.21.15.45:0/1288565147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10600640a0 con 0x7f1060106ea0 2024-04-25T18:46:27.443 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.656+0000 7f105e000700 1 -- 172.21.15.45:0/1288565147 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1054043e50 con 0x7f1060106ea0 2024-04-25T18:46:27.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.735+0000 7f1068000700 1 -- 172.21.15.45:0/1288565147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f10600008d0 con 0x7f1060106ea0 2024-04-25T18:46:27.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.736+0000 7f105e000700 1 -- 172.21.15.45:0/1288565147 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f1054032030 con 0x7f1060106ea0 2024-04-25T18:46:27.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.737+0000 7f1068000700 1 -- 172.21.15.45:0/1288565147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 msgr2=0x7f106019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:27.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.737+0000 7f1068000700 1 --2- 172.21.15.45:0/1288565147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f106019e720 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1054011ac0 tx=0x7f1054011ba0 comp rx=0 tx=0).stop 2024-04-25T18:46:27.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.737+0000 7f1068000700 1 -- 172.21.15.45:0/1288565147 shutdown_connections 2024-04-25T18:46:27.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.737+0000 7f1068000700 1 --2- 172.21.15.45:0/1288565147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1060106ea0 0x7f106019e720 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:27.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.737+0000 7f1068000700 1 -- 172.21.15.45:0/1288565147 >> 172.21.15.45:0/1288565147 conn(0x7f1060102b80 msgr2=0x7f1060195400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:27.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.738+0000 7f1068000700 1 -- 172.21.15.45:0/1288565147 shutdown_connections 2024-04-25T18:46:27.444 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:26.738+0000 7f1068000700 1 -- 172.21.15.45:0/1288565147 wait complete. 2024-04-25T18:46:27.444 INFO:teuthology.orchestra.run.smithi045.stdout:mgr not available, waiting (3/15)... 2024-04-25T18:46:28.144 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:27 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:27.848+0000 7fb49fb72200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:46:28.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:28.143+0000 7fb49fb72200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:46:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: Activating manager daemon x 2024-04-25T18:46:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: mgrmap e2: x(active, starting, since 0.0126501s) 2024-04-25T18:46:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:46:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:46:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:46:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:46:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:46:28.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:46:28.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:46:28.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:46:28.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: Manager daemon x is now available 2024-04-25T18:46:28.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:46:28.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' 2024-04-25T18:46:28.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:28 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:46:29.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:29 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' 2024-04-25T18:46:29.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:29 smithi045 ceph-mon[27776]: from='mgr.14100 172.21.15.45:0/2247466313' entity='mgr.x' 2024-04-25T18:46:29.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:29 smithi045 ceph-mon[27776]: mgrmap e3: x(active, since 1.02132s) 2024-04-25T18:46:31.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:31.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:46:31.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:46:31.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T18:46:31.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T18:46:31.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T18:46:31.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T18:46:31.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:31.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T18:46:31.105 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 0 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:31.106 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T18:46:31.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T18:46:31.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T18:46:31.107 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T18:46:31.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T18:46:31.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T18:46:31.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:31.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T18:46:31.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T18:46:31.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T18:46:31.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T18:46:31.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ], 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T18:46:09.148241+0000", 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout }, 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.114+0000 7feb1f200700 1 Processor -- start 2024-04-25T18:46:31.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.114+0000 7feb1f200700 1 -- start start 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.115+0000 7feb1f200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb20082860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.115+0000 7feb1f200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feb20082e30 con 0x7feb20082440 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.115+0000 7feb1de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb20082860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.115+0000 7feb1de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb20082860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60940/0 (socket says 172.21.15.45:60940) 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.115+0000 7feb1de00700 1 -- 172.21.15.45:0/1661429731 learned_addr learned my addr 172.21.15.45:0/1661429731 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.116+0000 7feb1de00700 1 -- 172.21.15.45:0/1661429731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb2010fcd0 con 0x7feb20082440 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.116+0000 7feb1de00700 1 --2- 172.21.15.45:0/1661429731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb20082860 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb100098a0 tx=0x7feb10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ef5213cc2f6804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.116+0000 7feb1d400700 1 -- 172.21.15.45:0/1661429731 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb10004160 con 0x7feb20082440 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.116+0000 7feb1d400700 1 -- 172.21.15.45:0/1661429731 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feb10004300 con 0x7feb20082440 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.116+0000 7feb1d400700 1 -- 172.21.15.45:0/1661429731 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb10011af0 con 0x7feb20082440 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.117+0000 7feb1f200700 1 -- 172.21.15.45:0/1661429731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 msgr2=0x7feb20082860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.117+0000 7feb1f200700 1 --2- 172.21.15.45:0/1661429731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb20082860 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb100098a0 tx=0x7feb10009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.117+0000 7feb1f200700 1 -- 172.21.15.45:0/1661429731 shutdown_connections 2024-04-25T18:46:31.118 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.117+0000 7feb1f200700 1 --2- 172.21.15.45:0/1661429731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb20082860 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.117+0000 7feb1f200700 1 -- 172.21.15.45:0/1661429731 >> 172.21.15.45:0/1661429731 conn(0x7feb2007da50 msgr2=0x7feb2007fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.117+0000 7feb1f200700 1 -- 172.21.15.45:0/1661429731 shutdown_connections 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.117+0000 7feb1f200700 1 -- 172.21.15.45:0/1661429731 wait complete. 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.118+0000 7feb1f200700 1 Processor -- start 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.119+0000 7feb1f200700 1 -- start start 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.119+0000 7feb1f200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb201a29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.119+0000 7feb1f200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feb20112f00 con 0x7feb20082440 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.119+0000 7feb1de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb201a29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.119+0000 7feb1de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb201a29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60952/0 (socket says 172.21.15.45:60952) 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.119+0000 7feb1de00700 1 -- 172.21.15.45:0/873051342 learned_addr learned my addr 172.21.15.45:0/873051342 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.119+0000 7feb1de00700 1 -- 172.21.15.45:0/873051342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb201a2ef0 con 0x7feb20082440 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.120+0000 7feb1de00700 1 --2- 172.21.15.45:0/873051342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb201a29b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb10011ac0 tx=0x7feb10011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.120+0000 7feb16a00700 1 -- 172.21.15.45:0/873051342 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb10003970 con 0x7feb20082440 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.120+0000 7feb16a00700 1 -- 172.21.15.45:0/873051342 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feb10003b10 con 0x7feb20082440 2024-04-25T18:46:31.119 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.120+0000 7feb1f200700 1 -- 172.21.15.45:0/873051342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb201a3180 con 0x7feb20082440 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.120+0000 7feb16a00700 1 -- 172.21.15.45:0/873051342 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb10024580 con 0x7feb20082440 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.120+0000 7feb1f200700 1 -- 172.21.15.45:0/873051342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb201a5e70 con 0x7feb20082440 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.122+0000 7feb16a00700 1 -- 172.21.15.45:0/873051342 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 3) ==== 46441+0+0 (secure 0 0 0) 0x7feb1001a430 con 0x7feb20082440 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.122+0000 7feb1f200700 1 -- 172.21.15.45:0/873051342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb20113080 con 0x7feb20082440 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.122+0000 7feb16a00700 1 --2- 172.21.15.45:0/873051342 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7feb04039490 0x7feb0403b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.122+0000 7feb16a00700 1 -- 172.21.15.45:0/873051342 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7feb1004d5a0 con 0x7feb20082440 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.122+0000 7feb17e00700 1 --2- 172.21.15.45:0/873051342 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7feb04039490 0x7feb0403b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.123+0000 7feb17e00700 1 --2- 172.21.15.45:0/873051342 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7feb04039490 0x7feb0403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb08006fd0 tx=0x7feb08006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.129+0000 7feb16a00700 1 -- 172.21.15.45:0/873051342 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb1002ddb0 con 0x7feb20082440 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.208+0000 7feb16a00700 1 -- 172.21.15.45:0/873051342 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7feb10024b20 con 0x7feb20082440 2024-04-25T18:46:31.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.451+0000 7feb1f200700 1 -- 172.21.15.45:0/873051342 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7feb201a6420 con 0x7feb20082440 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.451+0000 7feb16a00700 1 -- 172.21.15.45:0/873051342 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7feb1000b3e0 con 0x7feb20082440 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.454+0000 7feb1f200700 1 -- 172.21.15.45:0/873051342 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7feb04039490 msgr2=0x7feb0403b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.454+0000 7feb1f200700 1 --2- 172.21.15.45:0/873051342 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7feb04039490 0x7feb0403b950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb08006fd0 tx=0x7feb08006e40 comp rx=0 tx=0).stop 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.454+0000 7feb1f200700 1 -- 172.21.15.45:0/873051342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 msgr2=0x7feb201a29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.454+0000 7feb1f200700 1 --2- 172.21.15.45:0/873051342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb201a29b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feb10011ac0 tx=0x7feb10011ba0 comp rx=0 tx=0).stop 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.455+0000 7feb1f200700 1 -- 172.21.15.45:0/873051342 shutdown_connections 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.455+0000 7feb1f200700 1 --2- 172.21.15.45:0/873051342 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7feb04039490 0x7feb0403b950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.455+0000 7feb1f200700 1 --2- 172.21.15.45:0/873051342 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb20082440 0x7feb201a29b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.455+0000 7feb1f200700 1 -- 172.21.15.45:0/873051342 >> 172.21.15.45:0/873051342 conn(0x7feb2007da50 msgr2=0x7feb2007e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.455+0000 7feb1f200700 1 -- 172.21.15.45:0/873051342 shutdown_connections 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:30.455+0000 7feb1f200700 1 -- 172.21.15.45:0/873051342 wait complete. 2024-04-25T18:46:31.121 INFO:teuthology.orchestra.run.smithi045.stdout:mgr is available 2024-04-25T18:46:31.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:31 smithi045 ceph-mon[27776]: mgrmap e4: x(active, since 2s) 2024-04-25T18:46:31.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:31 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/873051342' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T18:46:32.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:32 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1538987824' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T18:46:32.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:32.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T18:46:32.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout fsid = ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:32.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T18:46:32.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T18:46:32.725 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T18:46:32.726 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.729+0000 7f98ea800700 1 Processor -- start 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.729+0000 7f98ea800700 1 -- start start 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.730+0000 7f98ea800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e4108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.730+0000 7f98ea800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4108ff0 con 0x7f98e4108600 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.730+0000 7f98e3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e4108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.730+0000 7f98e3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e4108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60960/0 (socket says 172.21.15.45:60960) 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.730+0000 7f98e3400700 1 -- 172.21.15.45:0/3190385432 learned_addr learned my addr 172.21.15.45:0/3190385432 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.731+0000 7f98e3400700 1 -- 172.21.15.45:0/3190385432 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e4109800 con 0x7f98e4108600 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.731+0000 7f98e3400700 1 --2- 172.21.15.45:0/3190385432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e4108a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98d40098a0 tx=0x7f98d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1261b3c8cf3e279c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.731+0000 7f98e2a00700 1 -- 172.21.15.45:0/3190385432 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98d4004160 con 0x7f98e4108600 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.731+0000 7f98e2a00700 1 -- 172.21.15.45:0/3190385432 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f98d4004300 con 0x7f98e4108600 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.731+0000 7f98e2a00700 1 -- 172.21.15.45:0/3190385432 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98d4011af0 con 0x7f98e4108600 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.732+0000 7f98ea800700 1 -- 172.21.15.45:0/3190385432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 msgr2=0x7f98e4108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.732+0000 7f98ea800700 1 --2- 172.21.15.45:0/3190385432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e4108a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f98d40098a0 tx=0x7f98d4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:32.727 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.732+0000 7f98ea800700 1 -- 172.21.15.45:0/3190385432 shutdown_connections 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.732+0000 7f98ea800700 1 --2- 172.21.15.45:0/3190385432 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e4108a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.732+0000 7f98ea800700 1 -- 172.21.15.45:0/3190385432 >> 172.21.15.45:0/3190385432 conn(0x7f98e4103be0 msgr2=0x7f98e4106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.732+0000 7f98ea800700 1 -- 172.21.15.45:0/3190385432 shutdown_connections 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.732+0000 7f98ea800700 1 -- 172.21.15.45:0/3190385432 wait complete. 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.733+0000 7f98ea800700 1 Processor -- start 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.733+0000 7f98ea800700 1 -- start start 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.734+0000 7f98ea800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.734+0000 7f98ea800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f98e410ca30 con 0x7f98e4108600 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.734+0000 7f98e3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.734+0000 7f98e3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60974/0 (socket says 172.21.15.45:60974) 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.734+0000 7f98e3400700 1 -- 172.21.15.45:0/1538987824 learned_addr learned my addr 172.21.15.45:0/1538987824 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.734+0000 7f98e3400700 1 -- 172.21.15.45:0/1538987824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e419cac0 con 0x7f98e4108600 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.735+0000 7f98e3400700 1 --2- 172.21.15.45:0/1538987824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e419c580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f98d40117b0 tx=0x7f98d4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:32.728 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.735+0000 7f98e0c00700 1 -- 172.21.15.45:0/1538987824 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98d4003af0 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.735+0000 7f98e0c00700 1 -- 172.21.15.45:0/1538987824 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f98d4020070 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.735+0000 7f98e0c00700 1 -- 172.21.15.45:0/1538987824 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f98d4019630 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.735+0000 7f98ea800700 1 -- 172.21.15.45:0/1538987824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e419cd50 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.735+0000 7f98ea800700 1 -- 172.21.15.45:0/1538987824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e419fa40 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.736+0000 7f98e0c00700 1 -- 172.21.15.45:0/1538987824 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f98d401a9c0 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.736+0000 7f98ea800700 1 -- 172.21.15.45:0/1538987824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e410cbb0 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.737+0000 7f98e0c00700 1 --2- 172.21.15.45:0/1538987824 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f98c40395b0 0x7f98c403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.737+0000 7f98e0c00700 1 -- 172.21.15.45:0/1538987824 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f98d404dae0 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.737+0000 7f98dbe00700 1 --2- 172.21.15.45:0/1538987824 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f98c40395b0 0x7f98c403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.738+0000 7f98dbe00700 1 --2- 172.21.15.45:0/1538987824 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f98c40395b0 0x7f98c403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98cc006fd0 tx=0x7f98cc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.743+0000 7f98e0c00700 1 -- 172.21.15.45:0/1538987824 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98d40042a0 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.973+0000 7f98ea800700 1 -- 172.21.15.45:0/1538987824 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f98e419fc10 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.975+0000 7f98e0c00700 1 -- 172.21.15.45:0/1538987824 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f98d400e260 con 0x7f98e4108600 2024-04-25T18:46:32.729 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.978+0000 7f98ea800700 1 -- 172.21.15.45:0/1538987824 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f98c40395b0 msgr2=0x7f98c403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:32.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.978+0000 7f98ea800700 1 --2- 172.21.15.45:0/1538987824 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f98c40395b0 0x7f98c403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98cc006fd0 tx=0x7f98cc006e40 comp rx=0 tx=0).stop 2024-04-25T18:46:32.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.978+0000 7f98ea800700 1 -- 172.21.15.45:0/1538987824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 msgr2=0x7f98e419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:32.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.978+0000 7f98ea800700 1 --2- 172.21.15.45:0/1538987824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e419c580 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f98d40117b0 tx=0x7f98d4011b70 comp rx=0 tx=0).stop 2024-04-25T18:46:32.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.978+0000 7f98ea800700 1 -- 172.21.15.45:0/1538987824 shutdown_connections 2024-04-25T18:46:32.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.978+0000 7f98ea800700 1 --2- 172.21.15.45:0/1538987824 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f98c40395b0 0x7f98c403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:32.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.978+0000 7f98ea800700 1 --2- 172.21.15.45:0/1538987824 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f98e4108600 0x7f98e419c580 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:32.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.978+0000 7f98ea800700 1 -- 172.21.15.45:0/1538987824 >> 172.21.15.45:0/1538987824 conn(0x7f98e4103be0 msgr2=0x7f98e4104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:32.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.978+0000 7f98ea800700 1 -- 172.21.15.45:0/1538987824 shutdown_connections 2024-04-25T18:46:32.730 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:31.978+0000 7f98ea800700 1 -- 172.21.15.45:0/1538987824 wait complete. 2024-04-25T18:46:32.730 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling cephadm module... 2024-04-25T18:46:33.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:33 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1475517673' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T18:46:33.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:33 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: ignoring --setuser ceph since I am not root 2024-04-25T18:46:33.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:33 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: ignoring --setgroup ceph since I am not root 2024-04-25T18:46:34.217 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:33 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:33.963+0000 7f3e173ea200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:46:34.217 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:34 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:34.214+0000 7f3e173ea200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:46:34.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.313+0000 7f4815800700 1 Processor -- start 2024-04-25T18:46:34.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.314+0000 7f4815800700 1 -- start start 2024-04-25T18:46:34.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.314+0000 7f4815800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f4810107740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:34.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.314+0000 7f4815800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4810107d10 con 0x7f4810107320 2024-04-25T18:46:34.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.314+0000 7f480fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f4810107740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:34.356 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.315+0000 7f480fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f4810107740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60976/0 (socket says 172.21.15.45:60976) 2024-04-25T18:46:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.315+0000 7f480fe00700 1 -- 172.21.15.45:0/2246284297 learned_addr learned my addr 172.21.15.45:0/2246284297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.315+0000 7f480fe00700 1 -- 172.21.15.45:0/2246284297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4810108520 con 0x7f4810107320 2024-04-25T18:46:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.315+0000 7f480fe00700 1 --2- 172.21.15.45:0/2246284297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f4810107740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f48000098a0 tx=0x7f4800009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d75941a4edad01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.316+0000 7f480f400700 1 -- 172.21.15.45:0/2246284297 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4800004160 con 0x7f4810107320 2024-04-25T18:46:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.316+0000 7f480f400700 1 -- 172.21.15.45:0/2246284297 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4800004300 con 0x7f4810107320 2024-04-25T18:46:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.316+0000 7f480f400700 1 -- 172.21.15.45:0/2246284297 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4800011af0 con 0x7f4810107320 2024-04-25T18:46:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.317+0000 7f4815800700 1 -- 172.21.15.45:0/2246284297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 msgr2=0x7f4810107740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.317+0000 7f4815800700 1 --2- 172.21.15.45:0/2246284297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f4810107740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f48000098a0 tx=0x7f4800009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.357 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.317+0000 7f4815800700 1 -- 172.21.15.45:0/2246284297 shutdown_connections 2024-04-25T18:46:34.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.317+0000 7f4815800700 1 --2- 172.21.15.45:0/2246284297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f4810107740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.317+0000 7f4815800700 1 -- 172.21.15.45:0/2246284297 >> 172.21.15.45:0/2246284297 conn(0x7f4810102920 msgr2=0x7f4810104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:34.361 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.317+0000 7f4815800700 1 -- 172.21.15.45:0/2246284297 shutdown_connections 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.317+0000 7f4815800700 1 -- 172.21.15.45:0/2246284297 wait complete. 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.318+0000 7f4815800700 1 Processor -- start 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.318+0000 7f4815800700 1 -- start start 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.318+0000 7f4815800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f481019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.318+0000 7f4815800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f481010b750 con 0x7f4810107320 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.319+0000 7f480fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f481019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.319+0000 7f480fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f481019c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:60992/0 (socket says 172.21.15.45:60992) 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.319+0000 7f480fe00700 1 -- 172.21.15.45:0/1475517673 learned_addr learned my addr 172.21.15.45:0/1475517673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.319+0000 7f480fe00700 1 -- 172.21.15.45:0/1475517673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481019c8f0 con 0x7f4810107320 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.319+0000 7f480fe00700 1 --2- 172.21.15.45:0/1475517673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f481019c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f48000117b0 tx=0x7f4800011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.320+0000 7f480d600700 1 -- 172.21.15.45:0/1475517673 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4800003af0 con 0x7f4810107320 2024-04-25T18:46:34.362 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.320+0000 7f480d600700 1 -- 172.21.15.45:0/1475517673 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4800020070 con 0x7f4810107320 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.320+0000 7f480d600700 1 -- 172.21.15.45:0/1475517673 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4800019630 con 0x7f4810107320 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.320+0000 7f4815800700 1 -- 172.21.15.45:0/1475517673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481019cb80 con 0x7f4810107320 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.320+0000 7f4815800700 1 -- 172.21.15.45:0/1475517673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481019f870 con 0x7f4810107320 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.321+0000 7f480d600700 1 -- 172.21.15.45:0/1475517673 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f480001a9c0 con 0x7f4810107320 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.321+0000 7f4815800700 1 -- 172.21.15.45:0/1475517673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481010b8d0 con 0x7f4810107320 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.322+0000 7f480d600700 1 --2- 172.21.15.45:0/1475517673 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f47f4039600 0x7f47f403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.322+0000 7f480d600700 1 -- 172.21.15.45:0/1475517673 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f480004dae0 con 0x7f4810107320 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.322+0000 7f4807400700 1 --2- 172.21.15.45:0/1475517673 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f47f4039600 0x7f47f403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.323+0000 7f4807400700 1 --2- 172.21.15.45:0/1475517673 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f47f4039600 0x7f47f403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47f8006fd0 tx=0x7f47f8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.327+0000 7f480d600700 1 -- 172.21.15.45:0/1475517673 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f48000042a0 con 0x7f4810107320 2024-04-25T18:46:34.363 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.621+0000 7f4815800700 1 -- 172.21.15.45:0/1475517673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f481002e360 con 0x7f4810107320 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.676+0000 7f480d600700 1 -- 172.21.15.45:0/1475517673 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f480000e260 con 0x7f4810107320 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.678+0000 7f480d600700 1 -- 172.21.15.45:0/1475517673 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f480001a430 con 0x7f4810107320 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.679+0000 7f4815800700 1 -- 172.21.15.45:0/1475517673 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f47f4039600 msgr2=0x7f47f403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.679+0000 7f4815800700 1 --2- 172.21.15.45:0/1475517673 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f47f4039600 0x7f47f403bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f47f8006fd0 tx=0x7f47f8006e40 comp rx=0 tx=0).stop 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.679+0000 7f4815800700 1 -- 172.21.15.45:0/1475517673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 msgr2=0x7f481019c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.679+0000 7f4815800700 1 --2- 172.21.15.45:0/1475517673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f481019c3b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f48000117b0 tx=0x7f4800011b70 comp rx=0 tx=0).stop 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.681+0000 7f4815800700 1 -- 172.21.15.45:0/1475517673 shutdown_connections 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.681+0000 7f4815800700 1 --2- 172.21.15.45:0/1475517673 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7f47f4039600 0x7f47f403bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.681+0000 7f4815800700 1 --2- 172.21.15.45:0/1475517673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4810107320 0x7f481019c3b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.681+0000 7f4815800700 1 -- 172.21.15.45:0/1475517673 >> 172.21.15.45:0/1475517673 conn(0x7f4810102920 msgr2=0x7f4810103870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.681+0000 7f4815800700 1 -- 172.21.15.45:0/1475517673 shutdown_connections 2024-04-25T18:46:34.364 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:33.681+0000 7f4815800700 1 -- 172.21.15.45:0/1475517673 wait complete. 2024-04-25T18:46:34.512 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:34 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:34.353+0000 7f3e173ea200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:46:34.769 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:34 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1475517673' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T18:46:34.770 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:34 smithi045 ceph-mon[27776]: mgrmap e5: x(active, since 5s) 2024-04-25T18:46:35.162 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:34 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:34.767+0000 7f3e173ea200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:46:35.162 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:34 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:34.884+0000 7f3e173ea200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:46:35.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:35 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3611140676' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T18:46:35.931 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:46:35.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T18:46:35.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T18:46:35.932 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T18:46:35.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T18:46:35.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:46:35.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.950+0000 7fefd4c00700 1 Processor -- start 2024-04-25T18:46:35.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.950+0000 7fefd4c00700 1 -- start start 2024-04-25T18:46:35.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.951+0000 7fefd4c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd01072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:35.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.951+0000 7fefd4c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0063d40 con 0x7fefd0106ec0 2024-04-25T18:46:35.933 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.951+0000 7fefcde00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd01072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:35.938 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.951+0000 7fefcde00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd01072e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58198/0 (socket says 172.21.15.45:58198) 2024-04-25T18:46:35.938 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.951+0000 7fefcde00700 1 -- 172.21.15.45:0/1362735591 learned_addr learned my addr 172.21.15.45:0/1362735591 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:35.938 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.952+0000 7fefcde00700 1 -- 172.21.15.45:0/1362735591 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd0107820 con 0x7fefd0106ec0 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.952+0000 7fefcde00700 1 --2- 172.21.15.45:0/1362735591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd01072e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fefc00098a0 tx=0x7fefc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04b385953978cee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.952+0000 7fefcca00700 1 -- 172.21.15.45:0/1362735591 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefc0004160 con 0x7fefd0106ec0 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.952+0000 7fefcca00700 1 -- 172.21.15.45:0/1362735591 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fefc0004300 con 0x7fefd0106ec0 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.952+0000 7fefcca00700 1 -- 172.21.15.45:0/1362735591 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefc0011af0 con 0x7fefd0106ec0 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.953+0000 7fefd4c00700 1 -- 172.21.15.45:0/1362735591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 msgr2=0x7fefd01072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.953+0000 7fefd4c00700 1 --2- 172.21.15.45:0/1362735591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd01072e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fefc00098a0 tx=0x7fefc0009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.953+0000 7fefd4c00700 1 -- 172.21.15.45:0/1362735591 shutdown_connections 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.953+0000 7fefd4c00700 1 --2- 172.21.15.45:0/1362735591 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd01072e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.953+0000 7fefd4c00700 1 -- 172.21.15.45:0/1362735591 >> 172.21.15.45:0/1362735591 conn(0x7fefd0102b80 msgr2=0x7fefd0104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.953+0000 7fefd4c00700 1 -- 172.21.15.45:0/1362735591 shutdown_connections 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.953+0000 7fefd4c00700 1 -- 172.21.15.45:0/1362735591 wait complete. 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.954+0000 7fefd4c00700 1 Processor -- start 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.955+0000 7fefd4c00700 1 -- start start 2024-04-25T18:46:35.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.955+0000 7fefd4c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.955+0000 7fefd4c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefd01087e0 con 0x7fefd0106ec0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.955+0000 7fefcde00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.955+0000 7fefcde00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd019e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58206/0 (socket says 172.21.15.45:58206) 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.955+0000 7fefcde00700 1 -- 172.21.15.45:0/3611140676 learned_addr learned my addr 172.21.15.45:0/3611140676 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.956+0000 7fefcde00700 1 -- 172.21.15.45:0/3611140676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd019ec50 con 0x7fefd0106ec0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.956+0000 7fefcde00700 1 --2- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd019e710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009fb0 tx=0x7fefc0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.956+0000 7fefc6a00700 1 -- 172.21.15.45:0/3611140676 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefc00036a0 con 0x7fefd0106ec0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.956+0000 7fefc6a00700 1 -- 172.21.15.45:0/3611140676 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fefc001a430 con 0x7fefd0106ec0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.956+0000 7fefc6a00700 1 -- 172.21.15.45:0/3611140676 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fefc0019550 con 0x7fefd0106ec0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.956+0000 7fefd4c00700 1 -- 172.21.15.45:0/3611140676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd019eee0 con 0x7fefd0106ec0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.956+0000 7fefd4c00700 1 -- 172.21.15.45:0/3611140676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd01a1bd0 con 0x7fefd0106ec0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.957+0000 7fefc6a00700 1 -- 172.21.15.45:0/3611140676 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fefc001a8e0 con 0x7fefd0106ec0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.957+0000 7fefd4c00700 1 -- 172.21.15.45:0/3611140676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd00640a0 con 0x7fefd0106ec0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.958+0000 7fefc6a00700 1 --2- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fefb4039560 0x7fefb403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.958+0000 7fefc6a00700 1 -- 172.21.15.45:0/3611140676 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fefc004d8e0 con 0x7fefd0106ec0 2024-04-25T18:46:35.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.958+0000 7fefcd400700 1 -- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fefb4039560 msgr2=0x7fefb403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1404432740 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.958+0000 7fefcd400700 1 --2- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fefb4039560 0x7fefb403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:34.967+0000 7fefc6a00700 1 -- 172.21.15.45:0/3611140676 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fefc001a690 con 0x7fefd0106ec0 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.159+0000 7fefcd400700 1 -- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fefb4039560 msgr2=0x7fefb403ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1404432740 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.159+0000 7fefcd400700 1 --2- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fefb4039560 0x7fefb403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.262+0000 7fefd4c00700 1 -- 172.21.15.45:0/3611140676 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fefd00008d0 con 0x7fefd0106ec0 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.263+0000 7fefc6a00700 1 -- 172.21.15.45:0/3611140676 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fefc004d300 con 0x7fefd0106ec0 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.267+0000 7fefd4c00700 1 -- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fefb4039560 msgr2=0x7fefb403ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.267+0000 7fefd4c00700 1 --2- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fefb4039560 0x7fefb403ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.267+0000 7fefd4c00700 1 -- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 msgr2=0x7fefd019e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.267+0000 7fefd4c00700 1 --2- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd019e710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009fb0 tx=0x7fefc0011940 comp rx=0 tx=0).stop 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.267+0000 7fefd4c00700 1 -- 172.21.15.45:0/3611140676 shutdown_connections 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.267+0000 7fefd4c00700 1 --2- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fefb4039560 0x7fefb403ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.267+0000 7fefd4c00700 1 --2- 172.21.15.45:0/3611140676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefd0106ec0 0x7fefd019e710 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.267+0000 7fefd4c00700 1 -- 172.21.15.45:0/3611140676 >> 172.21.15.45:0/3611140676 conn(0x7fefd0102b80 msgr2=0x7fefd01953d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:35.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.267+0000 7fefd4c00700 1 -- 172.21.15.45:0/3611140676 shutdown_connections 2024-04-25T18:46:35.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:35.267+0000 7fefd4c00700 1 -- 172.21.15.45:0/3611140676 wait complete. 2024-04-25T18:46:35.942 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-04-25T18:46:35.942 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 5... 2024-04-25T18:46:36.956 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:36 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:36.500+0000 7f3e173ea200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:46:36.956 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:36 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:36.735+0000 7f3e173ea200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:46:36.956 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:36 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:36.953+0000 7f3e173ea200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:46:37.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:37.347+0000 7f3e173ea200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:46:37.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:37.473+0000 7f3e173ea200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:46:37.996 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:37.731+0000 7f3e173ea200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:46:38.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:37.993+0000 7f3e173ea200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:46:39.213 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:38 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:38.800+0000 7f3e173ea200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:46:39.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:39.369+0000 7f3e173ea200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:46:39.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:39.499+0000 7f3e173ea200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:46:40.213 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:39.773+0000 7f3e173ea200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:46:40.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:40.436+0000 7f3e173ea200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:46:40.714 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:40.564+0000 7f3e173ea200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:46:41.211 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:40.894+0000 7f3e173ea200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:46:41.211 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:41.078+0000 7f3e173ea200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:46:41.211 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:41.208+0000 7f3e173ea200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:46:41.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:41.363+0000 7f3e173ea200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:46:42.213 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:41.740+0000 7f3e173ea200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:46:42.821 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:42.524+0000 7f3e173ea200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:46:43.213 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:46:42.818+0000 7f3e173ea200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: Active manager daemon x restarted 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: Activating manager daemon x 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: mgrmap e6: x(active, starting, since 0.00883887s) 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: Manager daemon x is now available 2024-04-25T18:46:43.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:42 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:46:44.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:43 smithi045 ceph-mon[27776]: Found migration_current of "None". Setting to last migration. 2024-04-25T18:46:44.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:43 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:46:44.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:43 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:46:44.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:43 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:46:44.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:43 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:46:44.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:43 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:46:44.185 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:43 smithi045 ceph-mon[27776]: mgrmap e7: x(active, since 1.01678s) 2024-04-25T18:46:44.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:46:44.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T18:46:44.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T18:46:44.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:46:44.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.521+0000 7fee38c00700 1 Processor -- start 2024-04-25T18:46:44.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.521+0000 7fee38c00700 1 -- start start 2024-04-25T18:46:44.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.522+0000 7fee38c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3410ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:44.487 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.522+0000 7fee38c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee3410b260 con 0x7fee3410a870 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.522+0000 7fee31e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3410ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.522+0000 7fee31e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3410ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58222/0 (socket says 172.21.15.45:58222) 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.522+0000 7fee31e00700 1 -- 172.21.15.45:0/2002987361 learned_addr learned my addr 172.21.15.45:0/2002987361 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.523+0000 7fee31e00700 1 -- 172.21.15.45:0/2002987361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3410ba60 con 0x7fee3410a870 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.523+0000 7fee31e00700 1 --2- 172.21.15.45:0/2002987361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3410ac90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee240098a0 tx=0x7fee24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbbda27d8f039c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.523+0000 7fee30a00700 1 -- 172.21.15.45:0/2002987361 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee24004160 con 0x7fee3410a870 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.523+0000 7fee30a00700 1 -- 172.21.15.45:0/2002987361 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fee24004300 con 0x7fee3410a870 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.523+0000 7fee30a00700 1 -- 172.21.15.45:0/2002987361 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee24011af0 con 0x7fee3410a870 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.524+0000 7fee38c00700 1 -- 172.21.15.45:0/2002987361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 msgr2=0x7fee3410ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.524+0000 7fee38c00700 1 --2- 172.21.15.45:0/2002987361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3410ac90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fee240098a0 tx=0x7fee24009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.524+0000 7fee38c00700 1 -- 172.21.15.45:0/2002987361 shutdown_connections 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.524+0000 7fee38c00700 1 --2- 172.21.15.45:0/2002987361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3410ac90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.524+0000 7fee38c00700 1 -- 172.21.15.45:0/2002987361 >> 172.21.15.45:0/2002987361 conn(0x7fee3407dc50 msgr2=0x7fee3407e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:44.488 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.524+0000 7fee38c00700 1 -- 172.21.15.45:0/2002987361 shutdown_connections 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.524+0000 7fee38c00700 1 -- 172.21.15.45:0/2002987361 wait complete. 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.526+0000 7fee38c00700 1 Processor -- start 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.526+0000 7fee38c00700 1 -- start start 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.526+0000 7fee38c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.526+0000 7fee38c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fee3410ec90 con 0x7fee3410a870 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.526+0000 7fee31e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.526+0000 7fee31e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58236/0 (socket says 172.21.15.45:58236) 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.526+0000 7fee31e00700 1 -- 172.21.15.45:0/2086810969 learned_addr learned my addr 172.21.15.45:0/2086810969 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.527+0000 7fee31e00700 1 -- 172.21.15.45:0/2086810969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3419ec70 con 0x7fee3410a870 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.527+0000 7fee31e00700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3419e730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fee24011ac0 tx=0x7fee24011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.527+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee24003970 con 0x7fee3410a870 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.527+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fee24003b10 con 0x7fee3410a870 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.527+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee24024610 con 0x7fee3410a870 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.528+0000 7fee38c00700 1 -- 172.21.15.45:0/2086810969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3419ef00 con 0x7fee3410a870 2024-04-25T18:46:44.489 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.528+0000 7fee38c00700 1 -- 172.21.15.45:0/2086810969 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee341a1bf0 con 0x7fee3410a870 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.529+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fee2401a9e0 con 0x7fee3410a870 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.529+0000 7fee2aa00700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 0x7fee1803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.529+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 --> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fee1803c180 con 0x7fee180395b0 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.529+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee2404d920 con 0x7fee3410a870 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.529+0000 7fee31400700 1 -- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 msgr2=0x7fee1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1404432740 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.530+0000 7fee31400700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 0x7fee1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.731+0000 7fee31400700 1 -- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 msgr2=0x7fee1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1404432740 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:36.731+0000 7fee31400700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 0x7fee1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:37.132+0000 7fee31400700 1 -- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 msgr2=0x7fee1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1404432740 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:37.132+0000 7fee31400700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 0x7fee1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:37.933+0000 7fee31400700 1 -- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 msgr2=0x7fee1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1404432740 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:37.933+0000 7fee31400700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 0x7fee1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:39.535+0000 7fee31400700 1 -- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 msgr2=0x7fee1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1404432740 2024-04-25T18:46:44.490 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:39.535+0000 7fee31400700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 0x7fee1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:42.738+0000 7fee31400700 1 -- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 msgr2=0x7fee1803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/1404432740 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:42.739+0000 7fee31400700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 0x7fee1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:42.827+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fee2401a430 con 0x7fee3410a870 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:42.828+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 msgr2=0x7fee1803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:42.828+0000 7fee2aa00700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 0x7fee1803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.833+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fee2404f450 con 0x7fee3410a870 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.834+0000 7fee2aa00700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fee1803cea0 0x7fee1803f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.834+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 --> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fee1803c180 con 0x7fee1803cea0 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.836+0000 7fee31400700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fee1803cea0 0x7fee1803f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.840+0000 7fee31400700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fee1803cea0 0x7fee1803f290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fee20003a10 tx=0x7fee20009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.842+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 <== mgr.14120 v2:172.21.15.45:6800/3065212608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fee1803c180 con 0x7fee1803cea0 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.850+0000 7fee38c00700 1 -- 172.21.15.45:0/2086810969 --> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fee3410ee10 con 0x7fee1803cea0 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.851+0000 7fee2aa00700 1 -- 172.21.15.45:0/2086810969 <== mgr.14120 v2:172.21.15.45:6800/3065212608 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fee3410ee10 con 0x7fee1803cea0 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.851+0000 7fee38c00700 1 -- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fee1803cea0 msgr2=0x7fee1803f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:44.491 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.851+0000 7fee38c00700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fee1803cea0 0x7fee1803f290 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fee20003a10 tx=0x7fee20009290 comp rx=0 tx=0).stop 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.851+0000 7fee38c00700 1 -- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 msgr2=0x7fee3419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.851+0000 7fee38c00700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3419e730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fee24011ac0 tx=0x7fee24011ba0 comp rx=0 tx=0).stop 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.851+0000 7fee38c00700 1 -- 172.21.15.45:0/2086810969 shutdown_connections 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.851+0000 7fee38c00700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fee1803cea0 0x7fee1803f290 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.851+0000 7fee38c00700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fee3410a870 0x7fee3419e730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.851+0000 7fee38c00700 1 --2- 172.21.15.45:0/2086810969 >> [v2:172.21.15.45:6800/1404432740,v1:172.21.15.45:6801/1404432740] conn(0x7fee180395b0 0x7fee1803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.851+0000 7fee38c00700 1 -- 172.21.15.45:0/2086810969 >> 172.21.15.45:0/2086810969 conn(0x7fee3407dc50 msgr2=0x7fee34108540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.852+0000 7fee38c00700 1 -- 172.21.15.45:0/2086810969 shutdown_connections 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:43.852+0000 7fee38c00700 1 -- 172.21.15.45:0/2086810969 wait complete. 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 5 is available 2024-04-25T18:46:44.492 INFO:teuthology.orchestra.run.smithi045.stdout:Setting orchestrator backend to cephadm... 2024-04-25T18:46:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:45 smithi045 ceph-mon[27776]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T18:46:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:45 smithi045 ceph-mon[27776]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T18:46:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:45 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:46:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:45 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:46:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:45 smithi045 ceph-mon[27776]: [25/Apr/2024:18:46:44] ENGINE Bus STARTING 2024-04-25T18:46:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:45 smithi045 ceph-mon[27776]: [25/Apr/2024:18:46:44] ENGINE Serving on http://172.21.15.45:8765 2024-04-25T18:46:45.330 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:45 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:46:46.146 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.075+0000 7fd833200700 1 Processor -- start 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.076+0000 7fd833200700 1 -- start start 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.076+0000 7fd833200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c07c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.076+0000 7fd833200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c07c7a0 con 0x7fd82c07dd70 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.077+0000 7fd831e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c07c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.077+0000 7fd831e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c07c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37550/0 (socket says 172.21.15.45:37550) 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.077+0000 7fd831e00700 1 -- 172.21.15.45:0/3782821603 learned_addr learned my addr 172.21.15.45:0/3782821603 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.077+0000 7fd831e00700 1 -- 172.21.15.45:0/3782821603 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82c07c920 con 0x7fd82c07dd70 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.078+0000 7fd831e00700 1 --2- 172.21.15.45:0/3782821603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c07c1d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd8200098a0 tx=0x7fd820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa563a96dceaf090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.078+0000 7fd830a00700 1 -- 172.21.15.45:0/3782821603 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820004160 con 0x7fd82c07dd70 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.078+0000 7fd830a00700 1 -- 172.21.15.45:0/3782821603 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd820004300 con 0x7fd82c07dd70 2024-04-25T18:46:46.147 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.078+0000 7fd830a00700 1 -- 172.21.15.45:0/3782821603 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820011af0 con 0x7fd82c07dd70 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.079+0000 7fd833200700 1 -- 172.21.15.45:0/3782821603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 msgr2=0x7fd82c07c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.079+0000 7fd833200700 1 --2- 172.21.15.45:0/3782821603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c07c1d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd8200098a0 tx=0x7fd820009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.079+0000 7fd833200700 1 -- 172.21.15.45:0/3782821603 shutdown_connections 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.079+0000 7fd833200700 1 --2- 172.21.15.45:0/3782821603 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c07c1d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.079+0000 7fd833200700 1 -- 172.21.15.45:0/3782821603 >> 172.21.15.45:0/3782821603 conn(0x7fd82c103a10 msgr2=0x7fd82c105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.079+0000 7fd833200700 1 -- 172.21.15.45:0/3782821603 shutdown_connections 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.079+0000 7fd833200700 1 -- 172.21.15.45:0/3782821603 wait complete. 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.080+0000 7fd833200700 1 Processor -- start 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.080+0000 7fd833200700 1 -- start start 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.081+0000 7fd833200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c19e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.081+0000 7fd833200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c10e9a0 con 0x7fd82c07dd70 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.081+0000 7fd831e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c19e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.081+0000 7fd831e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c19e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37564/0 (socket says 172.21.15.45:37564) 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.081+0000 7fd831e00700 1 -- 172.21.15.45:0/1829462404 learned_addr learned my addr 172.21.15.45:0/1829462404 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:46.148 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.081+0000 7fd831e00700 1 -- 172.21.15.45:0/1829462404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82c19eab0 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.081+0000 7fd831e00700 1 --2- 172.21.15.45:0/1829462404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c19e570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd820011ac0 tx=0x7fd820011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.082+0000 7fd826a00700 1 -- 172.21.15.45:0/1829462404 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820003970 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.082+0000 7fd826a00700 1 -- 172.21.15.45:0/1829462404 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd820003b10 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.082+0000 7fd833200700 1 -- 172.21.15.45:0/1829462404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82c19ed40 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.082+0000 7fd826a00700 1 -- 172.21.15.45:0/1829462404 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd820024610 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.082+0000 7fd833200700 1 -- 172.21.15.45:0/1829462404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82c1a1a30 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.083+0000 7fd826a00700 1 -- 172.21.15.45:0/1829462404 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fd82001a9e0 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.083+0000 7fd833200700 1 -- 172.21.15.45:0/1829462404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82c0640a0 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.083+0000 7fd826a00700 1 --2- 172.21.15.45:0/1829462404 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd8140394e0 0x7fd81403b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.084+0000 7fd826a00700 1 -- 172.21.15.45:0/1829462404 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd82004d7a0 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.084+0000 7fd831400700 1 --2- 172.21.15.45:0/1829462404 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd8140394e0 0x7fd81403b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.084+0000 7fd831400700 1 --2- 172.21.15.45:0/1829462404 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd8140394e0 0x7fd81403b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd81c006fd0 tx=0x7fd81c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.090+0000 7fd826a00700 1 -- 172.21.15.45:0/1829462404 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd82000b3e0 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.108+0000 7fd826a00700 1 -- 172.21.15.45:0/1829462404 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd82002c350 con 0x7fd82c07dd70 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.328+0000 7fd833200700 1 -- 172.21.15.45:0/1829462404 --> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd82c104ce0 con 0x7fd8140394e0 2024-04-25T18:46:46.149 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.341+0000 7fd826a00700 1 -- 172.21.15.45:0/1829462404 <== mgr.14120 v2:172.21.15.45:6800/3065212608 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd82c104ce0 con 0x7fd8140394e0 2024-04-25T18:46:46.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.343+0000 7fd833200700 1 -- 172.21.15.45:0/1829462404 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd8140394e0 msgr2=0x7fd81403b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:46.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.343+0000 7fd833200700 1 --2- 172.21.15.45:0/1829462404 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd8140394e0 0x7fd81403b9a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd81c006fd0 tx=0x7fd81c006e40 comp rx=0 tx=0).stop 2024-04-25T18:46:46.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.344+0000 7fd833200700 1 -- 172.21.15.45:0/1829462404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 msgr2=0x7fd82c19e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:46.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.344+0000 7fd833200700 1 --2- 172.21.15.45:0/1829462404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c19e570 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd820011ac0 tx=0x7fd820011ba0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.344+0000 7fd833200700 1 -- 172.21.15.45:0/1829462404 shutdown_connections 2024-04-25T18:46:46.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.344+0000 7fd833200700 1 --2- 172.21.15.45:0/1829462404 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd8140394e0 0x7fd81403b9a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.344+0000 7fd833200700 1 --2- 172.21.15.45:0/1829462404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd82c07dd70 0x7fd82c19e570 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:46.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.344+0000 7fd833200700 1 -- 172.21.15.45:0/1829462404 >> 172.21.15.45:0/1829462404 conn(0x7fd82c103a10 msgr2=0x7fd82c1045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:46.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.344+0000 7fd833200700 1 -- 172.21.15.45:0/1829462404 shutdown_connections 2024-04-25T18:46:46.150 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:45.344+0000 7fd833200700 1 -- 172.21.15.45:0/1829462404 wait complete. 2024-04-25T18:46:46.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:46 smithi045 ceph-mon[27776]: [25/Apr/2024:18:46:44] ENGINE Serving on https://172.21.15.45:7150 2024-04-25T18:46:46.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:46 smithi045 ceph-mon[27776]: [25/Apr/2024:18:46:44] ENGINE Bus STARTED 2024-04-25T18:46:46.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:46 smithi045 ceph-mon[27776]: mgrmap e8: x(active, since 2s) 2024-04-25T18:46:46.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:46 smithi045 ceph-mon[27776]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:46.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:46 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:46:46.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:46 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:46:47.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T18:46:47.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.754+0000 7fd23d000700 1 Processor -- start 2024-04-25T18:46:47.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.754+0000 7fd23d000700 1 -- start start 2024-04-25T18:46:47.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.754+0000 7fd23d000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd238108850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:47.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.755+0000 7fd23d000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd238108e20 con 0x7fd238108430 2024-04-25T18:46:47.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.755+0000 7fd237400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd238108850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:47.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.755+0000 7fd237400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd238108850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37570/0 (socket says 172.21.15.45:37570) 2024-04-25T18:46:47.634 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.755+0000 7fd237400700 1 -- 172.21.15.45:0/2980430008 learned_addr learned my addr 172.21.15.45:0/2980430008 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.756+0000 7fd237400700 1 -- 172.21.15.45:0/2980430008 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd238109630 con 0x7fd238108430 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.756+0000 7fd237400700 1 --2- 172.21.15.45:0/2980430008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd238108850 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd2280098a0 tx=0x7fd228009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233e1809764076b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.756+0000 7fd236000700 1 -- 172.21.15.45:0/2980430008 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd228004160 con 0x7fd238108430 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.756+0000 7fd236000700 1 -- 172.21.15.45:0/2980430008 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd228004300 con 0x7fd238108430 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.756+0000 7fd236000700 1 -- 172.21.15.45:0/2980430008 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd228011af0 con 0x7fd238108430 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.757+0000 7fd23d000700 1 -- 172.21.15.45:0/2980430008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 msgr2=0x7fd238108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.757+0000 7fd23d000700 1 --2- 172.21.15.45:0/2980430008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd238108850 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd2280098a0 tx=0x7fd228009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.757+0000 7fd23d000700 1 -- 172.21.15.45:0/2980430008 shutdown_connections 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.757+0000 7fd23d000700 1 --2- 172.21.15.45:0/2980430008 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd238108850 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.757+0000 7fd23d000700 1 -- 172.21.15.45:0/2980430008 >> 172.21.15.45:0/2980430008 conn(0x7fd238103a10 msgr2=0x7fd238105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.757+0000 7fd23d000700 1 -- 172.21.15.45:0/2980430008 shutdown_connections 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.757+0000 7fd23d000700 1 -- 172.21.15.45:0/2980430008 wait complete. 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.758+0000 7fd23d000700 1 Processor -- start 2024-04-25T18:46:47.635 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.758+0000 7fd23d000700 1 -- start start 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.759+0000 7fd23d000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd23819c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.759+0000 7fd23d000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd23810c860 con 0x7fd238108430 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.759+0000 7fd237400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd23819c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.759+0000 7fd237400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd23819c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37578/0 (socket says 172.21.15.45:37578) 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.759+0000 7fd237400700 1 -- 172.21.15.45:0/4151340914 learned_addr learned my addr 172.21.15.45:0/4151340914 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.759+0000 7fd237400700 1 -- 172.21.15.45:0/4151340914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23819c8f0 con 0x7fd238108430 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.760+0000 7fd237400700 1 --2- 172.21.15.45:0/4151340914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd23819c3b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd2280117b0 tx=0x7fd228011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.760+0000 7fd22fe00700 1 -- 172.21.15.45:0/4151340914 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd228003af0 con 0x7fd238108430 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.760+0000 7fd22fe00700 1 -- 172.21.15.45:0/4151340914 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd228020070 con 0x7fd238108430 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.760+0000 7fd23d000700 1 -- 172.21.15.45:0/4151340914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23819cb80 con 0x7fd238108430 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.760+0000 7fd22fe00700 1 -- 172.21.15.45:0/4151340914 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd228019630 con 0x7fd238108430 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.760+0000 7fd23d000700 1 -- 172.21.15.45:0/4151340914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23819f870 con 0x7fd238108430 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.761+0000 7fd22fe00700 1 -- 172.21.15.45:0/4151340914 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd2280198a0 con 0x7fd238108430 2024-04-25T18:46:47.636 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.762+0000 7fd23d000700 1 -- 172.21.15.45:0/4151340914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd23810c9e0 con 0x7fd238108430 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.762+0000 7fd22fe00700 1 --2- 172.21.15.45:0/4151340914 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd21c039600 0x7fd21c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.762+0000 7fd22fe00700 1 -- 172.21.15.45:0/4151340914 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd22804dd30 con 0x7fd238108430 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.762+0000 7fd236a00700 1 --2- 172.21.15.45:0/4151340914 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd21c039600 0x7fd21c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.763+0000 7fd236a00700 1 --2- 172.21.15.45:0/4151340914 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd21c039600 0x7fd21c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd224006fd0 tx=0x7fd224006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.768+0000 7fd22fe00700 1 -- 172.21.15.45:0/4151340914 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd228023370 con 0x7fd238108430 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.995+0000 7fd23d000700 1 -- 172.21.15.45:0/4151340914 --> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fd238104ce0 con 0x7fd21c039600 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.996+0000 7fd22fe00700 1 -- 172.21.15.45:0/4151340914 <== mgr.14120 v2:172.21.15.45:6800/3065212608 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fd238104ce0 con 0x7fd21c039600 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.999+0000 7fd23d000700 1 -- 172.21.15.45:0/4151340914 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd21c039600 msgr2=0x7fd21c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.999+0000 7fd23d000700 1 --2- 172.21.15.45:0/4151340914 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd21c039600 0x7fd21c03bac0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd224006fd0 tx=0x7fd224006e40 comp rx=0 tx=0).stop 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.999+0000 7fd23d000700 1 -- 172.21.15.45:0/4151340914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 msgr2=0x7fd23819c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.999+0000 7fd23d000700 1 --2- 172.21.15.45:0/4151340914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd23819c3b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd2280117b0 tx=0x7fd228011b70 comp rx=0 tx=0).stop 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.999+0000 7fd23d000700 1 -- 172.21.15.45:0/4151340914 shutdown_connections 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.999+0000 7fd23d000700 1 --2- 172.21.15.45:0/4151340914 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fd21c039600 0x7fd21c03bac0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:47.637 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.999+0000 7fd23d000700 1 --2- 172.21.15.45:0/4151340914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238108430 0x7fd23819c3b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:47.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.999+0000 7fd23d000700 1 -- 172.21.15.45:0/4151340914 >> 172.21.15.45:0/4151340914 conn(0x7fd238103a10 msgr2=0x7fd2381045d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:47.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.999+0000 7fd23d000700 1 -- 172.21.15.45:0/4151340914 shutdown_connections 2024-04-25T18:46:47.638 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:46.999+0000 7fd23d000700 1 -- 172.21.15.45:0/4151340914 wait complete. 2024-04-25T18:46:47.638 INFO:teuthology.orchestra.run.smithi045.stdout:Generating ssh key... 2024-04-25T18:46:47.918 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:47 smithi045 ceph-mon[27776]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:48.965 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: Generating public/private rsa key pair. 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: Your identification has been saved in /tmp/tmpm0ffvg9r/key. 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: Your public key has been saved in /tmp/tmpm0ffvg9r/key.pub. 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: The key fingerprint is: 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: SHA256:fd/7B36jrYsBbQrb7HQKSmuR1nlobL0ozp27r/OU3Qw ceph-ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: The key's randomart image is: 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: +---[RSA 3072]----+ 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: | | 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: | | 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: | | 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: | .. | 2024-04-25T18:46:48.966 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: | +.S..E. | 2024-04-25T18:46:48.967 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: | + B=o*.+... | 2024-04-25T18:46:48.967 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: | ..+ooB.+ +...| 2024-04-25T18:46:48.967 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: | oo+o*.o o o.+| 2024-04-25T18:46:48.967 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: | o=.*B= . +++=| 2024-04-25T18:46:48.967 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:46:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: +----[SHA256]-----+ 2024-04-25T18:46:49.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.220+0000 7efdc6800700 1 Processor -- start 2024-04-25T18:46:49.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.221+0000 7efdc6800700 1 -- start start 2024-04-25T18:46:49.305 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.221+0000 7efdc6800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc0108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.221+0000 7efdc6800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efdc0109000 con 0x7efdc0108610 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.221+0000 7efdbf400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc0108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.221+0000 7efdbf400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc0108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37590/0 (socket says 172.21.15.45:37590) 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.221+0000 7efdbf400700 1 -- 172.21.15.45:0/2933025319 learned_addr learned my addr 172.21.15.45:0/2933025319 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.222+0000 7efdbf400700 1 -- 172.21.15.45:0/2933025319 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc0109810 con 0x7efdc0108610 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.222+0000 7efdbf400700 1 --2- 172.21.15.45:0/2933025319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc0108a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efdb00098a0 tx=0x7efdb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d12c6ce232a99ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.223+0000 7efdbea00700 1 -- 172.21.15.45:0/2933025319 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb0004160 con 0x7efdc0108610 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.223+0000 7efdbea00700 1 -- 172.21.15.45:0/2933025319 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efdb0004300 con 0x7efdc0108610 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.223+0000 7efdbea00700 1 -- 172.21.15.45:0/2933025319 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb0011af0 con 0x7efdc0108610 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.223+0000 7efdc6800700 1 -- 172.21.15.45:0/2933025319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 msgr2=0x7efdc0108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:49.306 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.223+0000 7efdc6800700 1 --2- 172.21.15.45:0/2933025319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc0108a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efdb00098a0 tx=0x7efdb0009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:49.307 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.224+0000 7efdc6800700 1 -- 172.21.15.45:0/2933025319 shutdown_connections 2024-04-25T18:46:49.307 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.224+0000 7efdc6800700 1 --2- 172.21.15.45:0/2933025319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc0108a30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:49.307 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.224+0000 7efdc6800700 1 -- 172.21.15.45:0/2933025319 >> 172.21.15.45:0/2933025319 conn(0x7efdc0103c70 msgr2=0x7efdc0106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:49.307 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.224+0000 7efdc6800700 1 -- 172.21.15.45:0/2933025319 shutdown_connections 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.224+0000 7efdc6800700 1 -- 172.21.15.45:0/2933025319 wait complete. 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.225+0000 7efdc6800700 1 Processor -- start 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.225+0000 7efdc6800700 1 -- start start 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.225+0000 7efdc6800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc019c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.225+0000 7efdc6800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efdc010ca40 con 0x7efdc0108610 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.226+0000 7efdbf400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc019c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.226+0000 7efdbf400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc019c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37596/0 (socket says 172.21.15.45:37596) 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.226+0000 7efdbf400700 1 -- 172.21.15.45:0/1889673633 learned_addr learned my addr 172.21.15.45:0/1889673633 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.226+0000 7efdbf400700 1 -- 172.21.15.45:0/1889673633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc019ca80 con 0x7efdc0108610 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.226+0000 7efdbf400700 1 --2- 172.21.15.45:0/1889673633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc019c540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009fb0 tx=0x7efdb0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.227+0000 7efdbcc00700 1 -- 172.21.15.45:0/1889673633 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb00036a0 con 0x7efdc0108610 2024-04-25T18:46:49.311 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.227+0000 7efdbcc00700 1 -- 172.21.15.45:0/1889673633 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efdb001a430 con 0x7efdc0108610 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.227+0000 7efdbcc00700 1 -- 172.21.15.45:0/1889673633 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdb0019550 con 0x7efdc0108610 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.227+0000 7efdc6800700 1 -- 172.21.15.45:0/1889673633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc019cd10 con 0x7efdc0108610 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.227+0000 7efdc6800700 1 -- 172.21.15.45:0/1889673633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc019fa00 con 0x7efdc0108610 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.228+0000 7efdbcc00700 1 -- 172.21.15.45:0/1889673633 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7efdb001a8e0 con 0x7efdc0108610 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.228+0000 7efdc6800700 1 -- 172.21.15.45:0/1889673633 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc010cbc0 con 0x7efdc0108610 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.229+0000 7efdbcc00700 1 --2- 172.21.15.45:0/1889673633 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7efda00395b0 0x7efda003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.229+0000 7efdbcc00700 1 -- 172.21.15.45:0/1889673633 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efdb004d990 con 0x7efdc0108610 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.229+0000 7efdb6a00700 1 --2- 172.21.15.45:0/1889673633 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7efda00395b0 0x7efda003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.230+0000 7efdb6a00700 1 --2- 172.21.15.45:0/1889673633 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7efda00395b0 0x7efda003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efda8006fd0 tx=0x7efda8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.235+0000 7efdbcc00700 1 -- 172.21.15.45:0/1889673633 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efdb001a690 con 0x7efdc0108610 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.467+0000 7efdc6800700 1 -- 172.21.15.45:0/1889673633 --> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7efdc0104ec0 con 0x7efda00395b0 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.621+0000 7efdbcc00700 1 -- 172.21.15.45:0/1889673633 <== mgr.14120 v2:172.21.15.45:6800/3065212608 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efdc0104ec0 con 0x7efda00395b0 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.625+0000 7efdc6800700 1 -- 172.21.15.45:0/1889673633 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7efda00395b0 msgr2=0x7efda003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:49.312 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.625+0000 7efdc6800700 1 --2- 172.21.15.45:0/1889673633 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7efda00395b0 0x7efda003ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efda8006fd0 tx=0x7efda8006e40 comp rx=0 tx=0).stop 2024-04-25T18:46:49.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.625+0000 7efdc6800700 1 -- 172.21.15.45:0/1889673633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 msgr2=0x7efdc019c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:49.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.625+0000 7efdc6800700 1 --2- 172.21.15.45:0/1889673633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc019c540 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efdb0009fb0 tx=0x7efdb0011940 comp rx=0 tx=0).stop 2024-04-25T18:46:49.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.625+0000 7efdc6800700 1 -- 172.21.15.45:0/1889673633 shutdown_connections 2024-04-25T18:46:49.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.625+0000 7efdc6800700 1 --2- 172.21.15.45:0/1889673633 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7efda00395b0 0x7efda003ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:49.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.625+0000 7efdc6800700 1 --2- 172.21.15.45:0/1889673633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdc0108610 0x7efdc019c540 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:49.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.625+0000 7efdc6800700 1 -- 172.21.15.45:0/1889673633 >> 172.21.15.45:0/1889673633 conn(0x7efdc0103c70 msgr2=0x7efdc01047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:49.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.625+0000 7efdc6800700 1 -- 172.21.15.45:0/1889673633 shutdown_connections 2024-04-25T18:46:49.313 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:48.625+0000 7efdc6800700 1 -- 172.21.15.45:0/1889673633 wait complete. 2024-04-25T18:46:49.914 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:49 smithi045 ceph-mon[27776]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:49.920 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:49 smithi045 ceph-mon[27776]: Generating ssh key... 2024-04-25T18:46:49.920 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:49 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:46:49.920 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:49 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:46:50.762 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDExa0soJz1qBmgK4LdK81Z5ovpCPaZP9w0Inkioej3e6VrKUDAgw3+bC4M9zgf9u8n0+Fa4zV9SNAl9qzkF0tropb4sa/J3uUHculs5lNGNd2AAFgMns/ZDaoXHGHKH7mGyKAg+Db8LxfIubbAvqwPleQ9CjSpmMCrBbVWDFjmZPUpBd5tP9Z2sXzi218CyCj4P7KhADYH3jiMZlhOzuvEFUVyAVee/qJT3AaHGyqbHXl0ZhJqrjJxF0elBOtAhDUiU+0XHwFyv0L2vY8//2WrL17pmdfuVKwVT4yAjZXB5MdkbpjWl8QcLs3PKvAwxk3kQ8iObiErcIFsdZkzferLlHn245mx1fth5yHKn6qswARHNZTWUZq5WID37f4yav/RXu6v1o2DdwlJDNLDTwSU27vfsC6Ke4t+7cCUSuAvUu+8g+Hc4mI5olrG5UknHiEhVH3fN2W2IGNq/7LQjWwhJ+pFqo6oLEQm5BTD6EVf2n00Tmf204Dv7R7I142bHSM= ceph-ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:46:50.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.911+0000 7f99b8400700 1 Processor -- start 2024-04-25T18:46:50.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.912+0000 7f99b8400700 1 -- start start 2024-04-25T18:46:50.763 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.912+0000 7f99b8400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b0108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:50.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.912+0000 7f99b8400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99b0108ff0 con 0x7f99b0108600 2024-04-25T18:46:50.764 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.913+0000 7f99b5a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b0108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:50.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.913+0000 7f99b5a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b0108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37598/0 (socket says 172.21.15.45:37598) 2024-04-25T18:46:50.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.913+0000 7f99b5a00700 1 -- 172.21.15.45:0/3530677882 learned_addr learned my addr 172.21.15.45:0/3530677882 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:50.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.914+0000 7f99b5a00700 1 -- 172.21.15.45:0/3530677882 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b0109800 con 0x7f99b0108600 2024-04-25T18:46:50.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.914+0000 7f99b5a00700 1 --2- 172.21.15.45:0/3530677882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b0108a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99a40098a0 tx=0x7f99a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbf184312a78fce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:50.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.914+0000 7f99afe00700 1 -- 172.21.15.45:0/3530677882 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a4004160 con 0x7f99b0108600 2024-04-25T18:46:50.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.914+0000 7f99afe00700 1 -- 172.21.15.45:0/3530677882 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f99a4004300 con 0x7f99b0108600 2024-04-25T18:46:50.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.914+0000 7f99afe00700 1 -- 172.21.15.45:0/3530677882 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a4011af0 con 0x7f99b0108600 2024-04-25T18:46:50.768 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.915+0000 7f99b8400700 1 -- 172.21.15.45:0/3530677882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 msgr2=0x7f99b0108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.915+0000 7f99b8400700 1 --2- 172.21.15.45:0/3530677882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b0108a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f99a40098a0 tx=0x7f99a4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.915+0000 7f99b8400700 1 -- 172.21.15.45:0/3530677882 shutdown_connections 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.915+0000 7f99b8400700 1 --2- 172.21.15.45:0/3530677882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b0108a20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.915+0000 7f99b8400700 1 -- 172.21.15.45:0/3530677882 >> 172.21.15.45:0/3530677882 conn(0x7f99b0103be0 msgr2=0x7f99b0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.915+0000 7f99b8400700 1 -- 172.21.15.45:0/3530677882 shutdown_connections 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.915+0000 7f99b8400700 1 -- 172.21.15.45:0/3530677882 wait complete. 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.917+0000 7f99b8400700 1 Processor -- start 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.917+0000 7f99b8400700 1 -- start start 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.917+0000 7f99b8400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.918+0000 7f99b8400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f99b010ca30 con 0x7f99b0108600 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.918+0000 7f99b5a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.918+0000 7f99b5a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37612/0 (socket says 172.21.15.45:37612) 2024-04-25T18:46:50.769 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.918+0000 7f99b5a00700 1 -- 172.21.15.45:0/705816768 learned_addr learned my addr 172.21.15.45:0/705816768 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.918+0000 7f99b5a00700 1 -- 172.21.15.45:0/705816768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99b019cac0 con 0x7f99b0108600 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.919+0000 7f99b5a00700 1 --2- 172.21.15.45:0/705816768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b019c580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009fb0 tx=0x7f99a4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.919+0000 7f99ae000700 1 -- 172.21.15.45:0/705816768 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a40036a0 con 0x7f99b0108600 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.919+0000 7f99ae000700 1 -- 172.21.15.45:0/705816768 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f99a401a430 con 0x7f99b0108600 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.919+0000 7f99b8400700 1 -- 172.21.15.45:0/705816768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99b019cd50 con 0x7f99b0108600 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.919+0000 7f99ae000700 1 -- 172.21.15.45:0/705816768 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99a4019550 con 0x7f99b0108600 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.919+0000 7f99b8400700 1 -- 172.21.15.45:0/705816768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99b019fa40 con 0x7f99b0108600 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.920+0000 7f99ae000700 1 -- 172.21.15.45:0/705816768 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f99a4019800 con 0x7f99b0108600 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.921+0000 7f99a3400700 1 -- 172.21.15.45:0/705816768 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f998c005330 con 0x7f99b0108600 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.921+0000 7f99ae000700 1 --2- 172.21.15.45:0/705816768 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f99980395b0 0x7f999803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.921+0000 7f99ae000700 1 -- 172.21.15.45:0/705816768 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f99a404da80 con 0x7f99b0108600 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.922+0000 7f99b5000700 1 --2- 172.21.15.45:0/705816768 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f99980395b0 0x7f999803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.922+0000 7f99b5000700 1 --2- 172.21.15.45:0/705816768 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f99980395b0 0x7f999803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f999c006fd0 tx=0x7f999c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:50.770 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:49.927+0000 7f99ae000700 1 -- 172.21.15.45:0/705816768 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99a4019af0 con 0x7f99b0108600 2024-04-25T18:46:50.771 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.158+0000 7f99a3400700 1 -- 172.21.15.45:0/705816768 --> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f998c000bc0 con 0x7f99980395b0 2024-04-25T18:46:50.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.160+0000 7f99ae000700 1 -- 172.21.15.45:0/705816768 <== mgr.14120 v2:172.21.15.45:6800/3065212608 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f998c000bc0 con 0x7f99980395b0 2024-04-25T18:46:50.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.163+0000 7f99a3400700 1 -- 172.21.15.45:0/705816768 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f99980395b0 msgr2=0x7f999803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:50.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.163+0000 7f99a3400700 1 --2- 172.21.15.45:0/705816768 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f99980395b0 0x7f999803ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f999c006fd0 tx=0x7f999c006e40 comp rx=0 tx=0).stop 2024-04-25T18:46:50.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.163+0000 7f99a3400700 1 -- 172.21.15.45:0/705816768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 msgr2=0x7f99b019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:50.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.163+0000 7f99a3400700 1 --2- 172.21.15.45:0/705816768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b019c580 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f99a4009fb0 tx=0x7f99a4011940 comp rx=0 tx=0).stop 2024-04-25T18:46:50.772 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.163+0000 7f99a3400700 1 -- 172.21.15.45:0/705816768 shutdown_connections 2024-04-25T18:46:50.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.163+0000 7f99a3400700 1 --2- 172.21.15.45:0/705816768 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f99980395b0 0x7f999803ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.163+0000 7f99a3400700 1 --2- 172.21.15.45:0/705816768 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f99b0108600 0x7f99b019c580 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:50.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.163+0000 7f99a3400700 1 -- 172.21.15.45:0/705816768 >> 172.21.15.45:0/705816768 conn(0x7f99b0103be0 msgr2=0x7f99b0104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:50.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.164+0000 7f99a3400700 1 -- 172.21.15.45:0/705816768 shutdown_connections 2024-04-25T18:46:50.773 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:50.164+0000 7f99a3400700 1 -- 172.21.15.45:0/705816768 wait complete. 2024-04-25T18:46:50.773 INFO:teuthology.orchestra.run.smithi045.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T18:46:50.773 INFO:teuthology.orchestra.run.smithi045.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T18:46:50.778 INFO:teuthology.orchestra.run.smithi045.stdout:Adding host smithi045... 2024-04-25T18:46:51.096 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:50 smithi045 ceph-mon[27776]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:52.074 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:51 smithi045 ceph-mon[27776]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:46:53.951 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:53 smithi045 ceph-mon[27776]: Deploying cephadm binary to smithi045 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Added host 'smithi045' with addr '172.21.15.45' 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.407+0000 7ff6e9000700 1 Processor -- start 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.408+0000 7ff6e9000700 1 -- start start 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.408+0000 7ff6e9000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e410ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.408+0000 7ff6e9000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e410b210 con 0x7ff6e410a820 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.409+0000 7ff6e1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e410ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.409+0000 7ff6e1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e410ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37622/0 (socket says 172.21.15.45:37622) 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.409+0000 7ff6e1e00700 1 -- 172.21.15.45:0/2795239105 learned_addr learned my addr 172.21.15.45:0/2795239105 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.409+0000 7ff6e1e00700 1 -- 172.21.15.45:0/2795239105 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e410ba20 con 0x7ff6e410a820 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.410+0000 7ff6e1e00700 1 --2- 172.21.15.45:0/2795239105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e410ac40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40098a0 tx=0x7ff6d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789d3b8f52ef69f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:58.712 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.410+0000 7ff6e1400700 1 -- 172.21.15.45:0/2795239105 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4004160 con 0x7ff6e410a820 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.410+0000 7ff6e1400700 1 -- 172.21.15.45:0/2795239105 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff6d4004300 con 0x7ff6e410a820 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.410+0000 7ff6e1400700 1 -- 172.21.15.45:0/2795239105 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4011af0 con 0x7ff6e410a820 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.411+0000 7ff6e9000700 1 -- 172.21.15.45:0/2795239105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 msgr2=0x7ff6e410ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.411+0000 7ff6e9000700 1 --2- 172.21.15.45:0/2795239105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e410ac40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40098a0 tx=0x7ff6d4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.411+0000 7ff6e9000700 1 -- 172.21.15.45:0/2795239105 shutdown_connections 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.411+0000 7ff6e9000700 1 --2- 172.21.15.45:0/2795239105 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e410ac40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.411+0000 7ff6e9000700 1 -- 172.21.15.45:0/2795239105 >> 172.21.15.45:0/2795239105 conn(0x7ff6e407dbe0 msgr2=0x7ff6e41083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.411+0000 7ff6e9000700 1 -- 172.21.15.45:0/2795239105 shutdown_connections 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.411+0000 7ff6e9000700 1 -- 172.21.15.45:0/2795239105 wait complete. 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.412+0000 7ff6e9000700 1 Processor -- start 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.412+0000 7ff6e9000700 1 -- start start 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.412+0000 7ff6e9000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.412+0000 7ff6e9000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff6e410ec50 con 0x7ff6e410a820 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.413+0000 7ff6e1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:58.713 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.413+0000 7ff6e1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e419e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37636/0 (socket says 172.21.15.45:37636) 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.413+0000 7ff6e1e00700 1 -- 172.21.15.45:0/502560076 learned_addr learned my addr 172.21.15.45:0/502560076 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.413+0000 7ff6e1e00700 1 -- 172.21.15.45:0/502560076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6e419ecf0 con 0x7ff6e410a820 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.413+0000 7ff6e1e00700 1 --2- 172.21.15.45:0/502560076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e419e7b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40117b0 tx=0x7ff6d4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.414+0000 7ff6db400700 1 -- 172.21.15.45:0/502560076 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4003af0 con 0x7ff6e410a820 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.414+0000 7ff6db400700 1 -- 172.21.15.45:0/502560076 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff6d4020070 con 0x7ff6e410a820 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.414+0000 7ff6e9000700 1 -- 172.21.15.45:0/502560076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6e419ef80 con 0x7ff6e410a820 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.414+0000 7ff6db400700 1 -- 172.21.15.45:0/502560076 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6d4019630 con 0x7ff6e410a820 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.414+0000 7ff6e9000700 1 -- 172.21.15.45:0/502560076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6e41a1c70 con 0x7ff6e410a820 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.415+0000 7ff6db400700 1 -- 172.21.15.45:0/502560076 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff6d401a9c0 con 0x7ff6e410a820 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.415+0000 7ff6e9000700 1 -- 172.21.15.45:0/502560076 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6e410edd0 con 0x7ff6e410a820 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.415+0000 7ff6db400700 1 --2- 172.21.15.45:0/502560076 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7ff6c8039600 0x7ff6c803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.416+0000 7ff6db400700 1 -- 172.21.15.45:0/502560076 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff6d404dbf0 con 0x7ff6e410a820 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.416+0000 7ff6d9400700 1 --2- 172.21.15.45:0/502560076 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7ff6c8039600 0x7ff6c803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.416+0000 7ff6d9400700 1 --2- 172.21.15.45:0/502560076 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7ff6c8039600 0x7ff6c803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc006fd0 tx=0x7ff6cc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:46:58.714 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.421+0000 7ff6db400700 1 -- 172.21.15.45:0/502560076 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6d40042a0 con 0x7ff6e410a820 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:51.656+0000 7ff6e9000700 1 -- 172.21.15.45:0/502560076 --> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "addr": "172.21.15.45", "target": ["mon-mgr", ""]}) -- 0x7ff6e4108290 con 0x7ff6c8039600 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.935+0000 7ff6db400700 1 -- 172.21.15.45:0/502560076 <== mgr.14120 v2:172.21.15.45:6800/3065212608 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff6e4108290 con 0x7ff6c8039600 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.940+0000 7ff6e9000700 1 -- 172.21.15.45:0/502560076 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7ff6c8039600 msgr2=0x7ff6c803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.940+0000 7ff6e9000700 1 --2- 172.21.15.45:0/502560076 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7ff6c8039600 0x7ff6c803bac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc006fd0 tx=0x7ff6cc006e40 comp rx=0 tx=0).stop 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.940+0000 7ff6e9000700 1 -- 172.21.15.45:0/502560076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 msgr2=0x7ff6e419e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.940+0000 7ff6e9000700 1 --2- 172.21.15.45:0/502560076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e419e7b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff6d40117b0 tx=0x7ff6d4011b70 comp rx=0 tx=0).stop 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.940+0000 7ff6e9000700 1 -- 172.21.15.45:0/502560076 shutdown_connections 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.940+0000 7ff6e9000700 1 --2- 172.21.15.45:0/502560076 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7ff6c8039600 0x7ff6c803bac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.940+0000 7ff6e9000700 1 --2- 172.21.15.45:0/502560076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff6e410a820 0x7ff6e419e7b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.940+0000 7ff6e9000700 1 -- 172.21.15.45:0/502560076 >> 172.21.15.45:0/502560076 conn(0x7ff6e407dbe0 msgr2=0x7ff6e4107b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.941+0000 7ff6e9000700 1 -- 172.21.15.45:0/502560076 shutdown_connections 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:57.941+0000 7ff6e9000700 1 -- 172.21.15.45:0/502560076 wait complete. 2024-04-25T18:46:58.715 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mon service... 2024-04-25T18:46:58.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:58 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:46:58.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:58 smithi045 ceph-mon[27776]: Added host smithi045 2024-04-25T18:46:58.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:46:58 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:00.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T18:47:00.391 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.368+0000 7f2ab7a00700 1 Processor -- start 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.368+0000 7f2ab7a00700 1 -- start start 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.369+0000 7f2ab7a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab0108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.369+0000 7f2ab7a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab0109000 con 0x7f2ab0108610 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.369+0000 7f2ab5000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab0108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.369+0000 7f2ab5000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab0108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56484/0 (socket says 172.21.15.45:56484) 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.369+0000 7f2ab5000700 1 -- 172.21.15.45:0/3488837193 learned_addr learned my addr 172.21.15.45:0/3488837193 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.370+0000 7f2ab5000700 1 -- 172.21.15.45:0/3488837193 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab0109810 con 0x7f2ab0108610 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.370+0000 7f2ab5000700 1 --2- 172.21.15.45:0/3488837193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab0108a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40098a0 tx=0x7f2aa4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f175e838a1edd3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.370+0000 7f2aaf400700 1 -- 172.21.15.45:0/3488837193 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa4004160 con 0x7f2ab0108610 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.371+0000 7f2aaf400700 1 -- 172.21.15.45:0/3488837193 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2aa4004300 con 0x7f2ab0108610 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.371+0000 7f2aaf400700 1 -- 172.21.15.45:0/3488837193 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa4011af0 con 0x7f2ab0108610 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.371+0000 7f2ab7a00700 1 -- 172.21.15.45:0/3488837193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 msgr2=0x7f2ab0108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.371+0000 7f2ab7a00700 1 --2- 172.21.15.45:0/3488837193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab0108a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40098a0 tx=0x7f2aa4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.371+0000 7f2ab7a00700 1 -- 172.21.15.45:0/3488837193 shutdown_connections 2024-04-25T18:47:00.392 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.371+0000 7f2ab7a00700 1 --2- 172.21.15.45:0/3488837193 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab0108a30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.371+0000 7f2ab7a00700 1 -- 172.21.15.45:0/3488837193 >> 172.21.15.45:0/3488837193 conn(0x7f2ab0103c70 msgr2=0x7f2ab0106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.372+0000 7f2ab7a00700 1 -- 172.21.15.45:0/3488837193 shutdown_connections 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.372+0000 7f2ab7a00700 1 -- 172.21.15.45:0/3488837193 wait complete. 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.373+0000 7f2ab7a00700 1 Processor -- start 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.373+0000 7f2ab7a00700 1 -- start start 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.373+0000 7f2ab7a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.373+0000 7f2ab7a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab010ca40 con 0x7f2ab0108610 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.374+0000 7f2ab5000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.374+0000 7f2ab5000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab019e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56496/0 (socket says 172.21.15.45:56496) 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.374+0000 7f2ab5000700 1 -- 172.21.15.45:0/2749550090 learned_addr learned my addr 172.21.15.45:0/2749550090 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.374+0000 7f2ab5000700 1 -- 172.21.15.45:0/2749550090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab019ecb0 con 0x7f2ab0108610 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.374+0000 7f2ab5000700 1 --2- 172.21.15.45:0/2749550090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab019e770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40117b0 tx=0x7f2aa4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.374+0000 7f2aad600700 1 -- 172.21.15.45:0/2749550090 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa4003af0 con 0x7f2ab0108610 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.375+0000 7f2aad600700 1 -- 172.21.15.45:0/2749550090 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2aa4020070 con 0x7f2ab0108610 2024-04-25T18:47:00.393 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.375+0000 7f2aad600700 1 -- 172.21.15.45:0/2749550090 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa4019630 con 0x7f2ab0108610 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.375+0000 7f2ab7a00700 1 -- 172.21.15.45:0/2749550090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab019ef40 con 0x7f2ab0108610 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.375+0000 7f2ab7a00700 1 -- 172.21.15.45:0/2749550090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab01a1c30 con 0x7f2ab0108610 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.376+0000 7f2aad600700 1 -- 172.21.15.45:0/2749550090 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2aa401a9c0 con 0x7f2ab0108610 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.376+0000 7f2ab7a00700 1 -- 172.21.15.45:0/2749550090 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab010cbc0 con 0x7f2ab0108610 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.376+0000 7f2aad600700 1 --2- 172.21.15.45:0/2749550090 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f2a98039600 0x7f2a9803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.376+0000 7f2aad600700 1 -- 172.21.15.45:0/2749550090 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2aa404db30 con 0x7f2ab0108610 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.377+0000 7f2aafe00700 1 --2- 172.21.15.45:0/2749550090 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f2a98039600 0x7f2a9803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.377+0000 7f2aafe00700 1 --2- 172.21.15.45:0/2749550090 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f2a98039600 0x7f2a9803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c006fd0 tx=0x7f2a9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.382+0000 7f2aad600700 1 -- 172.21.15.45:0/2749550090 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2aa400e260 con 0x7f2ab0108610 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.630+0000 7f2ab7a00700 1 -- 172.21.15.45:0/2749550090 --> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2ab0104f10 con 0x7f2a98039600 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.642+0000 7f2aad600700 1 -- 172.21.15.45:0/2749550090 <== mgr.14120 v2:172.21.15.45:6800/3065212608 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2ab0104f10 con 0x7f2a98039600 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.646+0000 7f2ab7a00700 1 -- 172.21.15.45:0/2749550090 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f2a98039600 msgr2=0x7f2a9803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:00.394 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.646+0000 7f2ab7a00700 1 --2- 172.21.15.45:0/2749550090 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f2a98039600 0x7f2a9803bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c006fd0 tx=0x7f2a9c006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:00.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.646+0000 7f2ab7a00700 1 -- 172.21.15.45:0/2749550090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 msgr2=0x7f2ab019e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:00.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.646+0000 7f2ab7a00700 1 --2- 172.21.15.45:0/2749550090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab019e770 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40117b0 tx=0x7f2aa4011b70 comp rx=0 tx=0).stop 2024-04-25T18:47:00.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.646+0000 7f2ab7a00700 1 -- 172.21.15.45:0/2749550090 shutdown_connections 2024-04-25T18:47:00.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.646+0000 7f2ab7a00700 1 --2- 172.21.15.45:0/2749550090 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f2a98039600 0x7f2a9803bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:00.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.646+0000 7f2ab7a00700 1 --2- 172.21.15.45:0/2749550090 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ab0108610 0x7f2ab019e770 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:00.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.646+0000 7f2ab7a00700 1 -- 172.21.15.45:0/2749550090 >> 172.21.15.45:0/2749550090 conn(0x7f2ab0103c70 msgr2=0x7f2ab0104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:00.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.647+0000 7f2ab7a00700 1 -- 172.21.15.45:0/2749550090 shutdown_connections 2024-04-25T18:47:00.395 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:46:59.647+0000 7f2ab7a00700 1 -- 172.21.15.45:0/2749550090 wait complete. 2024-04-25T18:47:00.395 INFO:teuthology.orchestra.run.smithi045.stdout:Deploying unmanaged mgr service... 2024-04-25T18:47:00.901 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:00 smithi045 ceph-mon[27776]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:00.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:00 smithi045 ceph-mon[27776]: Saving service mon spec with placement count:5 2024-04-25T18:47:00.902 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:00 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:47:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T18:47:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.202+0000 7fe384800700 1 Processor -- start 2024-04-25T18:47:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.202+0000 7fe384800700 1 -- start start 2024-04-25T18:47:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.203+0000 7fe384800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.203+0000 7fe384800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c109050 con 0x7fe37c108660 2024-04-25T18:47:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.203+0000 7fe381e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.203+0000 7fe381e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56512/0 (socket says 172.21.15.45:56512) 2024-04-25T18:47:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.203+0000 7fe381e00700 1 -- 172.21.15.45:0/3510356118 learned_addr learned my addr 172.21.15.45:0/3510356118 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.204+0000 7fe381e00700 1 -- 172.21.15.45:0/3510356118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c109810 con 0x7fe37c108660 2024-04-25T18:47:02.453 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.204+0000 7fe381e00700 1 --2- 172.21.15.45:0/3510356118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c108a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe374009e80 tx=0x7fe37400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322ce1177828ce0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.204+0000 7fe380a00700 1 -- 172.21.15.45:0/3510356118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe374017070 con 0x7fe37c108660 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.205+0000 7fe380a00700 1 -- 172.21.15.45:0/3510356118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe37400b8c0 con 0x7fe37c108660 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.205+0000 7fe380a00700 1 -- 172.21.15.45:0/3510356118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe374004490 con 0x7fe37c108660 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.205+0000 7fe384800700 1 -- 172.21.15.45:0/3510356118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 msgr2=0x7fe37c108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.205+0000 7fe384800700 1 --2- 172.21.15.45:0/3510356118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c108a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe374009e80 tx=0x7fe37400b330 comp rx=0 tx=0).stop 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.206+0000 7fe384800700 1 -- 172.21.15.45:0/3510356118 shutdown_connections 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.206+0000 7fe384800700 1 --2- 172.21.15.45:0/3510356118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c108a80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.206+0000 7fe384800700 1 -- 172.21.15.45:0/3510356118 >> 172.21.15.45:0/3510356118 conn(0x7fe37c103c40 msgr2=0x7fe37c106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.206+0000 7fe384800700 1 -- 172.21.15.45:0/3510356118 shutdown_connections 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.206+0000 7fe384800700 1 -- 172.21.15.45:0/3510356118 wait complete. 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.207+0000 7fe384800700 1 Processor -- start 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.207+0000 7fe384800700 1 -- start start 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.207+0000 7fe384800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:02.454 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.207+0000 7fe384800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe37c10ca40 con 0x7fe37c108660 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.208+0000 7fe381e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.208+0000 7fe381e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c19e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56516/0 (socket says 172.21.15.45:56516) 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.208+0000 7fe381e00700 1 -- 172.21.15.45:0/631102747 learned_addr learned my addr 172.21.15.45:0/631102747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.208+0000 7fe381e00700 1 -- 172.21.15.45:0/631102747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37c19ec20 con 0x7fe37c108660 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.208+0000 7fe381e00700 1 --2- 172.21.15.45:0/631102747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c19e6e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe374004a60 tx=0x7fe37400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.209+0000 7fe372a00700 1 -- 172.21.15.45:0/631102747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe374020020 con 0x7fe37c108660 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.209+0000 7fe372a00700 1 -- 172.21.15.45:0/631102747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe374012430 con 0x7fe37c108660 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.209+0000 7fe384800700 1 -- 172.21.15.45:0/631102747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37c19eeb0 con 0x7fe37c108660 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.209+0000 7fe372a00700 1 -- 172.21.15.45:0/631102747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe374017040 con 0x7fe37c108660 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.209+0000 7fe384800700 1 -- 172.21.15.45:0/631102747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37c1a1ba0 con 0x7fe37c108660 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.210+0000 7fe372a00700 1 -- 172.21.15.45:0/631102747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe374023020 con 0x7fe37c108660 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.210+0000 7fe384800700 1 -- 172.21.15.45:0/631102747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe37c10cbc0 con 0x7fe37c108660 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.211+0000 7fe372a00700 1 --2- 172.21.15.45:0/631102747 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fe368039600 0x7fe36803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.211+0000 7fe372a00700 1 -- 172.21.15.45:0/631102747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe37400f050 con 0x7fe37c108660 2024-04-25T18:47:02.455 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.211+0000 7fe381400700 1 --2- 172.21.15.45:0/631102747 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fe368039600 0x7fe36803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.212+0000 7fe381400700 1 --2- 172.21.15.45:0/631102747 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fe368039600 0x7fe36803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe36c006fd0 tx=0x7fe36c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.216+0000 7fe372a00700 1 -- 172.21.15.45:0/631102747 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe37402a020 con 0x7fe37c108660 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.462+0000 7fe384800700 1 -- 172.21.15.45:0/631102747 --> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe37c104e10 con 0x7fe368039600 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.471+0000 7fe372a00700 1 -- 172.21.15.45:0/631102747 <== mgr.14120 v2:172.21.15.45:6800/3065212608 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe37c104e10 con 0x7fe368039600 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.474+0000 7fe384800700 1 -- 172.21.15.45:0/631102747 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fe368039600 msgr2=0x7fe36803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.474+0000 7fe384800700 1 --2- 172.21.15.45:0/631102747 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fe368039600 0x7fe36803bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe36c006fd0 tx=0x7fe36c006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.474+0000 7fe384800700 1 -- 172.21.15.45:0/631102747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 msgr2=0x7fe37c19e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.474+0000 7fe384800700 1 --2- 172.21.15.45:0/631102747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c19e6e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe374004a60 tx=0x7fe37400bd50 comp rx=0 tx=0).stop 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.474+0000 7fe384800700 1 -- 172.21.15.45:0/631102747 shutdown_connections 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.474+0000 7fe384800700 1 --2- 172.21.15.45:0/631102747 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fe368039600 0x7fe36803bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.474+0000 7fe384800700 1 --2- 172.21.15.45:0/631102747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe37c108660 0x7fe37c19e6e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.474+0000 7fe384800700 1 -- 172.21.15.45:0/631102747 >> 172.21.15.45:0/631102747 conn(0x7fe37c103c40 msgr2=0x7fe37c104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.475+0000 7fe384800700 1 -- 172.21.15.45:0/631102747 shutdown_connections 2024-04-25T18:47:02.456 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:01.475+0000 7fe384800700 1 -- 172.21.15.45:0/631102747 wait complete. 2024-04-25T18:47:02.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:02 smithi045 ceph-mon[27776]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:02.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:02 smithi045 ceph-mon[27776]: Saving service mgr spec with placement count:2 2024-04-25T18:47:02.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:02 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:47:02.705 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:02 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:47:03.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.021+0000 7fad71800700 1 Processor -- start 2024-04-25T18:47:03.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.021+0000 7fad71800700 1 -- start start 2024-04-25T18:47:03.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.022+0000 7fad71800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c10b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:03.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.022+0000 7fad71800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c07ceb0 con 0x7fad6c109040 2024-04-25T18:47:03.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.022+0000 7fad6aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c10b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:03.902 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.022+0000 7fad6aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c10b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56530/0 (socket says 172.21.15.45:56530) 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.022+0000 7fad6aa00700 1 -- 172.21.15.45:0/2219518623 learned_addr learned my addr 172.21.15.45:0/2219518623 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.022+0000 7fad6aa00700 1 -- 172.21.15.45:0/2219518623 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c10b970 con 0x7fad6c109040 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.023+0000 7fad6aa00700 1 --2- 172.21.15.45:0/2219518623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c10b430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0098a0 tx=0x7fad5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15076c00e06d680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.023+0000 7fad69600700 1 -- 172.21.15.45:0/2219518623 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad5c004160 con 0x7fad6c109040 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.023+0000 7fad69600700 1 -- 172.21.15.45:0/2219518623 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c004300 con 0x7fad6c109040 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.023+0000 7fad69600700 1 -- 172.21.15.45:0/2219518623 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad5c011a20 con 0x7fad6c109040 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.024+0000 7fad71800700 1 -- 172.21.15.45:0/2219518623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 msgr2=0x7fad6c10b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.024+0000 7fad71800700 1 --2- 172.21.15.45:0/2219518623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c10b430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fad5c0098a0 tx=0x7fad5c009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.024+0000 7fad71800700 1 -- 172.21.15.45:0/2219518623 shutdown_connections 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.024+0000 7fad71800700 1 --2- 172.21.15.45:0/2219518623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c10b430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.024+0000 7fad71800700 1 -- 172.21.15.45:0/2219518623 >> 172.21.15.45:0/2219518623 conn(0x7fad6c102a70 msgr2=0x7fad6c104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.025+0000 7fad71800700 1 -- 172.21.15.45:0/2219518623 shutdown_connections 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.025+0000 7fad71800700 1 -- 172.21.15.45:0/2219518623 wait complete. 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.026+0000 7fad71800700 1 Processor -- start 2024-04-25T18:47:03.903 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.026+0000 7fad71800700 1 -- start start 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.026+0000 7fad71800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c19e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.026+0000 7fad71800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c10c980 con 0x7fad6c109040 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.027+0000 7fad6aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c19e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.027+0000 7fad6aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c19e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56534/0 (socket says 172.21.15.45:56534) 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.027+0000 7fad6aa00700 1 -- 172.21.15.45:0/2510544439 learned_addr learned my addr 172.21.15.45:0/2510544439 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.027+0000 7fad6aa00700 1 -- 172.21.15.45:0/2510544439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c19ec40 con 0x7fad6c109040 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.027+0000 7fad6aa00700 1 --2- 172.21.15.45:0/2510544439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c19e700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fad5c004660 tx=0x7fad5c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.027+0000 7fad63400700 1 -- 172.21.15.45:0/2510544439 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad5c003aa0 con 0x7fad6c109040 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.028+0000 7fad63400700 1 -- 172.21.15.45:0/2510544439 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad5c01b8f0 con 0x7fad6c109040 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.028+0000 7fad71800700 1 -- 172.21.15.45:0/2510544439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c19eed0 con 0x7fad6c109040 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.028+0000 7fad63400700 1 -- 172.21.15.45:0/2510544439 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad5c023900 con 0x7fad6c109040 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.028+0000 7fad71800700 1 -- 172.21.15.45:0/2510544439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c1a1bc0 con 0x7fad6c109040 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.029+0000 7fad71800700 1 -- 172.21.15.45:0/2510544439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad6c0640a0 con 0x7fad6c109040 2024-04-25T18:47:03.904 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.029+0000 7fad63400700 1 -- 172.21.15.45:0/2510544439 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fad5c01baa0 con 0x7fad6c109040 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.030+0000 7fad63400700 1 --2- 172.21.15.45:0/2510544439 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fad50039600 0x7fad5003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.030+0000 7fad63400700 1 -- 172.21.15.45:0/2510544439 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fad5c04df80 con 0x7fad6c109040 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.030+0000 7fad6a000700 1 --2- 172.21.15.45:0/2510544439 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fad50039600 0x7fad5003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.031+0000 7fad6a000700 1 --2- 172.21.15.45:0/2510544439 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fad50039600 0x7fad5003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fad58006fd0 tx=0x7fad58006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.035+0000 7fad63400700 1 -- 172.21.15.45:0/2510544439 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad5c01f030 con 0x7fad6c109040 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.264+0000 7fad71800700 1 -- 172.21.15.45:0/2510544439 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fad6c1a21c0 con 0x7fad6c109040 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.267+0000 7fad63400700 1 -- 172.21.15.45:0/2510544439 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fad5c04d8d0 con 0x7fad6c109040 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.270+0000 7fad71800700 1 -- 172.21.15.45:0/2510544439 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fad50039600 msgr2=0x7fad5003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.270+0000 7fad71800700 1 --2- 172.21.15.45:0/2510544439 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fad50039600 0x7fad5003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fad58006fd0 tx=0x7fad58006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.270+0000 7fad71800700 1 -- 172.21.15.45:0/2510544439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 msgr2=0x7fad6c19e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.270+0000 7fad71800700 1 --2- 172.21.15.45:0/2510544439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c19e700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fad5c004660 tx=0x7fad5c011aa0 comp rx=0 tx=0).stop 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.271+0000 7fad71800700 1 -- 172.21.15.45:0/2510544439 shutdown_connections 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.271+0000 7fad71800700 1 --2- 172.21.15.45:0/2510544439 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fad50039600 0x7fad5003bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:03.905 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.271+0000 7fad71800700 1 --2- 172.21.15.45:0/2510544439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad6c109040 0x7fad6c19e700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:03.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.271+0000 7fad71800700 1 -- 172.21.15.45:0/2510544439 >> 172.21.15.45:0/2510544439 conn(0x7fad6c102a70 msgr2=0x7fad6c103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:03.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.271+0000 7fad71800700 1 -- 172.21.15.45:0/2510544439 shutdown_connections 2024-04-25T18:47:03.906 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:03.271+0000 7fad71800700 1 -- 172.21.15.45:0/2510544439 wait complete. 2024-04-25T18:47:04.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:04 smithi045 ceph-mon[27776]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:04.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:04 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:47:04.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:04 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2510544439' entity='client.admin' 2024-04-25T18:47:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.557+0000 7f1504c00700 1 Processor -- start 2024-04-25T18:47:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.557+0000 7f1504c00700 1 -- start start 2024-04-25T18:47:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.558+0000 7f1504c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f1500108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.558+0000 7f1504c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1500109000 con 0x7f1500108610 2024-04-25T18:47:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.558+0000 7f14fde00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f1500108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.558+0000 7f14fde00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f1500108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59222/0 (socket says 172.21.15.45:59222) 2024-04-25T18:47:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.558+0000 7f14fde00700 1 -- 172.21.15.45:0/200606018 learned_addr learned my addr 172.21.15.45:0/200606018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.559+0000 7f14fde00700 1 -- 172.21.15.45:0/200606018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1500109810 con 0x7f1500108610 2024-04-25T18:47:05.867 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.559+0000 7f14fde00700 1 --2- 172.21.15.45:0/200606018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f1500108a30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098a0 tx=0x7f14f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4723005d7ac5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.560+0000 7f14fca00700 1 -- 172.21.15.45:0/200606018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f0004160 con 0x7f1500108610 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.560+0000 7f14fca00700 1 -- 172.21.15.45:0/200606018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f0004300 con 0x7f1500108610 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.560+0000 7f14fca00700 1 -- 172.21.15.45:0/200606018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f0011a20 con 0x7f1500108610 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.560+0000 7f1504c00700 1 -- 172.21.15.45:0/200606018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 msgr2=0x7f1500108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.560+0000 7f1504c00700 1 --2- 172.21.15.45:0/200606018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f1500108a30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f14f00098a0 tx=0x7f14f0009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.560+0000 7f1504c00700 1 -- 172.21.15.45:0/200606018 shutdown_connections 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.560+0000 7f1504c00700 1 --2- 172.21.15.45:0/200606018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f1500108a30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.560+0000 7f1504c00700 1 -- 172.21.15.45:0/200606018 >> 172.21.15.45:0/200606018 conn(0x7f1500103c70 msgr2=0x7f1500106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.561+0000 7f1504c00700 1 -- 172.21.15.45:0/200606018 shutdown_connections 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.561+0000 7f1504c00700 1 -- 172.21.15.45:0/200606018 wait complete. 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.562+0000 7f1504c00700 1 Processor -- start 2024-04-25T18:47:05.868 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.562+0000 7f1504c00700 1 -- start start 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.563+0000 7f1504c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f150019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.563+0000 7f1504c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f150010cbc0 con 0x7f1500108610 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.563+0000 7f14fde00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f150019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.563+0000 7f14fde00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f150019e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59234/0 (socket says 172.21.15.45:59234) 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.563+0000 7f14fde00700 1 -- 172.21.15.45:0/2045653018 learned_addr learned my addr 172.21.15.45:0/2045653018 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.564+0000 7f14fde00700 1 -- 172.21.15.45:0/2045653018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150019ec10 con 0x7f1500108610 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.564+0000 7f14fde00700 1 --2- 172.21.15.45:0/2045653018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f150019e6d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14f0000c00 tx=0x7f14f0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.564+0000 7f14f6a00700 1 -- 172.21.15.45:0/2045653018 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f0003aa0 con 0x7f1500108610 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.564+0000 7f14f6a00700 1 -- 172.21.15.45:0/2045653018 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f001b8f0 con 0x7f1500108610 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.564+0000 7f14f6a00700 1 -- 172.21.15.45:0/2045653018 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14f0023890 con 0x7f1500108610 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.564+0000 7f1504c00700 1 -- 172.21.15.45:0/2045653018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150019eea0 con 0x7f1500108610 2024-04-25T18:47:05.869 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.565+0000 7f1504c00700 1 -- 172.21.15.45:0/2045653018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15001a1b90 con 0x7f1500108610 2024-04-25T18:47:05.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.566+0000 7f14f6a00700 1 -- 172.21.15.45:0/2045653018 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f14f001b440 con 0x7f1500108610 2024-04-25T18:47:05.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.566+0000 7f14f6a00700 1 --2- 172.21.15.45:0/2045653018 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f14e4039600 0x7f14e403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:05.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.566+0000 7f14f6a00700 1 -- 172.21.15.45:0/2045653018 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f14f004deb0 con 0x7f1500108610 2024-04-25T18:47:05.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.566+0000 7f1504c00700 1 -- 172.21.15.45:0/2045653018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f150010cd40 con 0x7f1500108610 2024-04-25T18:47:05.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.567+0000 7f14fd400700 1 --2- 172.21.15.45:0/2045653018 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f14e4039600 0x7f14e403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:05.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.567+0000 7f14fd400700 1 --2- 172.21.15.45:0/2045653018 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f14e4039600 0x7f14e403bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f14ec006fd0 tx=0x7f14ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:05.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.572+0000 7f14f6a00700 1 -- 172.21.15.45:0/2045653018 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14f001f030 con 0x7f1500108610 2024-04-25T18:47:05.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.823+0000 7f1504c00700 1 -- 172.21.15.45:0/2045653018 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f15001a1d60 con 0x7f1500108610 2024-04-25T18:47:05.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.827+0000 7f14f6a00700 1 -- 172.21.15.45:0/2045653018 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f14f00202b0 con 0x7f1500108610 2024-04-25T18:47:05.870 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.831+0000 7f1504c00700 1 -- 172.21.15.45:0/2045653018 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f14e4039600 msgr2=0x7f14e403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:05.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.831+0000 7f1504c00700 1 --2- 172.21.15.45:0/2045653018 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f14e4039600 0x7f14e403bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f14ec006fd0 tx=0x7f14ec006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:05.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.831+0000 7f1504c00700 1 -- 172.21.15.45:0/2045653018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 msgr2=0x7f150019e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:05.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.831+0000 7f1504c00700 1 --2- 172.21.15.45:0/2045653018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f150019e6d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14f0000c00 tx=0x7f14f0011aa0 comp rx=0 tx=0).stop 2024-04-25T18:47:05.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.831+0000 7f1504c00700 1 -- 172.21.15.45:0/2045653018 shutdown_connections 2024-04-25T18:47:05.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.831+0000 7f1504c00700 1 --2- 172.21.15.45:0/2045653018 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f14e4039600 0x7f14e403bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:05.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.831+0000 7f1504c00700 1 --2- 172.21.15.45:0/2045653018 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1500108610 0x7f150019e6d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:05.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.831+0000 7f1504c00700 1 -- 172.21.15.45:0/2045653018 >> 172.21.15.45:0/2045653018 conn(0x7f1500103c70 msgr2=0x7f15001046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:05.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.831+0000 7f1504c00700 1 -- 172.21.15.45:0/2045653018 shutdown_connections 2024-04-25T18:47:05.871 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:04.831+0000 7f1504c00700 1 -- 172.21.15.45:0/2045653018 wait complete. 2024-04-25T18:47:05.871 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling the dashboard module... 2024-04-25T18:47:06.173 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:05 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2045653018' entity='client.admin' 2024-04-25T18:47:06.174 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:05 smithi045 ceph-mon[27776]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:06.174 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:05 smithi045 ceph-mon[27776]: from='mgr.14120 172.21.15.45:0/2502728808' entity='mgr.x' 2024-04-25T18:47:07.188 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:06 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3727825981' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T18:47:07.188 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:07 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: ignoring --setuser ceph since I am not root 2024-04-25T18:47:07.188 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:07 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: ignoring --setgroup ceph since I am not root 2024-04-25T18:47:07.188 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:07 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:07.185+0000 7f03725d8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:47:07.584 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.479+0000 7fa399a00700 1 Processor -- start 2024-04-25T18:47:07.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.479+0000 7fa399a00700 1 -- start start 2024-04-25T18:47:07.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.480+0000 7fa399a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa394108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:07.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.480+0000 7fa399a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa394109000 con 0x7fa394108610 2024-04-25T18:47:07.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.480+0000 7fa392a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa394108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:07.585 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.480+0000 7fa392a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa394108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59250/0 (socket says 172.21.15.45:59250) 2024-04-25T18:47:07.586 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.480+0000 7fa392a00700 1 -- 172.21.15.45:0/1402751442 learned_addr learned my addr 172.21.15.45:0/1402751442 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:07.586 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.480+0000 7fa392a00700 1 -- 172.21.15.45:0/1402751442 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa394109810 con 0x7fa394108610 2024-04-25T18:47:07.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.481+0000 7fa392a00700 1 --2- 172.21.15.45:0/1402751442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa394108a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa3840098a0 tx=0x7fa384009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4cd8a7ad1764da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:07.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.481+0000 7fa391600700 1 -- 172.21.15.45:0/1402751442 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa384004160 con 0x7fa394108610 2024-04-25T18:47:07.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.481+0000 7fa391600700 1 -- 172.21.15.45:0/1402751442 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa384004300 con 0x7fa394108610 2024-04-25T18:47:07.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.481+0000 7fa391600700 1 -- 172.21.15.45:0/1402751442 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa384011a20 con 0x7fa394108610 2024-04-25T18:47:07.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.482+0000 7fa399a00700 1 -- 172.21.15.45:0/1402751442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 msgr2=0x7fa394108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:07.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.482+0000 7fa399a00700 1 --2- 172.21.15.45:0/1402751442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa394108a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa3840098a0 tx=0x7fa384009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:07.587 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.482+0000 7fa399a00700 1 -- 172.21.15.45:0/1402751442 shutdown_connections 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.482+0000 7fa399a00700 1 --2- 172.21.15.45:0/1402751442 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa394108a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.482+0000 7fa399a00700 1 -- 172.21.15.45:0/1402751442 >> 172.21.15.45:0/1402751442 conn(0x7fa394103c10 msgr2=0x7fa394106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.482+0000 7fa399a00700 1 -- 172.21.15.45:0/1402751442 shutdown_connections 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.482+0000 7fa399a00700 1 -- 172.21.15.45:0/1402751442 wait complete. 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.483+0000 7fa399a00700 1 Processor -- start 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.484+0000 7fa399a00700 1 -- start start 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.484+0000 7fa399a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa39419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.484+0000 7fa399a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa39410cbc0 con 0x7fa394108610 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.484+0000 7fa392a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa39419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.484+0000 7fa392a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa39419c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59252/0 (socket says 172.21.15.45:59252) 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.484+0000 7fa392a00700 1 -- 172.21.15.45:0/3727825981 learned_addr learned my addr 172.21.15.45:0/3727825981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.485+0000 7fa392a00700 1 -- 172.21.15.45:0/3727825981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39419ca70 con 0x7fa394108610 2024-04-25T18:47:07.589 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.485+0000 7fa392a00700 1 --2- 172.21.15.45:0/3727825981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa39419c530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa384000c00 tx=0x7fa384011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:07.590 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.485+0000 7fa38b400700 1 -- 172.21.15.45:0/3727825981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa384003aa0 con 0x7fa394108610 2024-04-25T18:47:07.592 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.485+0000 7fa38b400700 1 -- 172.21.15.45:0/3727825981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38401b8f0 con 0x7fa394108610 2024-04-25T18:47:07.592 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.485+0000 7fa38b400700 1 -- 172.21.15.45:0/3727825981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa384023890 con 0x7fa394108610 2024-04-25T18:47:07.592 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.485+0000 7fa399a00700 1 -- 172.21.15.45:0/3727825981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39419cd00 con 0x7fa394108610 2024-04-25T18:47:07.592 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.485+0000 7fa399a00700 1 -- 172.21.15.45:0/3727825981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39419f9f0 con 0x7fa394108610 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.487+0000 7fa38b400700 1 -- 172.21.15.45:0/3727825981 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa38402c3f0 con 0x7fa394108610 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.487+0000 7fa399a00700 1 -- 172.21.15.45:0/3727825981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39410cd40 con 0x7fa394108610 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.487+0000 7fa38b400700 1 --2- 172.21.15.45:0/3727825981 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fa3780395b0 0x7fa37803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.487+0000 7fa38b400700 1 -- 172.21.15.45:0/3727825981 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa38404e100 con 0x7fa394108610 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.488+0000 7fa392000700 1 --2- 172.21.15.45:0/3727825981 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fa3780395b0 0x7fa37803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.488+0000 7fa392000700 1 --2- 172.21.15.45:0/3727825981 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fa3780395b0 0x7fa37803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa380006fd0 tx=0x7fa380006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.494+0000 7fa38b400700 1 -- 172.21.15.45:0/3727825981 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa38401f030 con 0x7fa394108610 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.790+0000 7fa399a00700 1 -- 172.21.15.45:0/3727825981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa3940640a0 con 0x7fa394108610 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.836+0000 7fa38b400700 1 -- 172.21.15.45:0/3727825981 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa38401bda0 con 0x7fa394108610 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.839+0000 7fa38b400700 1 -- 172.21.15.45:0/3727825981 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa3840193d0 con 0x7fa394108610 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.840+0000 7fa399a00700 1 -- 172.21.15.45:0/3727825981 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fa3780395b0 msgr2=0x7fa37803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.840+0000 7fa399a00700 1 --2- 172.21.15.45:0/3727825981 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fa3780395b0 0x7fa37803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa380006fd0 tx=0x7fa380006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.841+0000 7fa399a00700 1 -- 172.21.15.45:0/3727825981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 msgr2=0x7fa39419c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.841+0000 7fa399a00700 1 --2- 172.21.15.45:0/3727825981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa39419c530 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa384000c00 tx=0x7fa384011aa0 comp rx=0 tx=0).stop 2024-04-25T18:47:07.593 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.842+0000 7fa399a00700 1 -- 172.21.15.45:0/3727825981 shutdown_connections 2024-04-25T18:47:07.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.842+0000 7fa399a00700 1 --2- 172.21.15.45:0/3727825981 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7fa3780395b0 0x7fa37803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:07.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.842+0000 7fa399a00700 1 --2- 172.21.15.45:0/3727825981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa394108610 0x7fa39419c530 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:07.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.842+0000 7fa399a00700 1 -- 172.21.15.45:0/3727825981 >> 172.21.15.45:0/3727825981 conn(0x7fa394103c10 msgr2=0x7fa394104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:07.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.842+0000 7fa399a00700 1 -- 172.21.15.45:0/3727825981 shutdown_connections 2024-04-25T18:47:07.594 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:06.842+0000 7fa399a00700 1 -- 172.21.15.45:0/3727825981 wait complete. 2024-04-25T18:47:07.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:07 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:07.438+0000 7f03725d8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:47:07.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:07 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:07.573+0000 7f03725d8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:47:08.006 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:07 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:07.990+0000 7f03725d8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:47:08.006 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:07 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3727825981' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T18:47:08.006 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:07 smithi045 ceph-mon[27776]: mgrmap e9: x(active, since 24s) 2024-04-25T18:47:08.359 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:08 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:08.122+0000 7f03725d8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:47:09.115 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:08 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1040532335' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T18:47:09.120 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:47:09.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T18:47:09.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T18:47:09.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T18:47:09.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T18:47:09.124 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:47:09.125 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.149+0000 7f1a56800700 1 Processor -- start 2024-04-25T18:47:09.125 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.149+0000 7f1a56800700 1 -- start start 2024-04-25T18:47:09.125 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.149+0000 7f1a56800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5010ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.125 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.149+0000 7f1a56800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5010b260 con 0x7f1a5010a870 2024-04-25T18:47:09.125 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.150+0000 7f1a4f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5010ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.125 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.150+0000 7f1a4f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5010ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59278/0 (socket says 172.21.15.45:59278) 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.150+0000 7f1a4f400700 1 -- 172.21.15.45:0/219949576 learned_addr learned my addr 172.21.15.45:0/219949576 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.150+0000 7f1a4f400700 1 -- 172.21.15.45:0/219949576 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5010ba20 con 0x7f1a5010a870 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.151+0000 7f1a4f400700 1 --2- 172.21.15.45:0/219949576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5010ac90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1a400098a0 tx=0x7f1a40009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e058f5ee5fb6baf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.151+0000 7f1a4e000700 1 -- 172.21.15.45:0/219949576 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a40004160 con 0x7f1a5010a870 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.151+0000 7f1a4e000700 1 -- 172.21.15.45:0/219949576 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a40004300 con 0x7f1a5010a870 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.151+0000 7f1a4e000700 1 -- 172.21.15.45:0/219949576 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a40011a20 con 0x7f1a5010a870 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.152+0000 7f1a56800700 1 -- 172.21.15.45:0/219949576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 msgr2=0x7f1a5010ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.152+0000 7f1a56800700 1 --2- 172.21.15.45:0/219949576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5010ac90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1a400098a0 tx=0x7f1a40009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.152+0000 7f1a56800700 1 -- 172.21.15.45:0/219949576 shutdown_connections 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.152+0000 7f1a56800700 1 --2- 172.21.15.45:0/219949576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5010ac90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.152+0000 7f1a56800700 1 -- 172.21.15.45:0/219949576 >> 172.21.15.45:0/219949576 conn(0x7f1a5007dca0 msgr2=0x7f1a501083c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.152+0000 7f1a56800700 1 -- 172.21.15.45:0/219949576 shutdown_connections 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.152+0000 7f1a56800700 1 -- 172.21.15.45:0/219949576 wait complete. 2024-04-25T18:47:09.126 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.153+0000 7f1a56800700 1 Processor -- start 2024-04-25T18:47:09.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.153+0000 7f1a56800700 1 -- start start 2024-04-25T18:47:09.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.153+0000 7f1a56800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.153+0000 7f1a56800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5010edd0 con 0x7f1a5010a870 2024-04-25T18:47:09.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.154+0000 7f1a4f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:09.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.154+0000 7f1a4f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59286/0 (socket says 172.21.15.45:59286) 2024-04-25T18:47:09.127 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.154+0000 7f1a4f400700 1 -- 172.21.15.45:0/1040532335 learned_addr learned my addr 172.21.15.45:0/1040532335 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:09.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.154+0000 7f1a4f400700 1 -- 172.21.15.45:0/1040532335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5019ec00 con 0x7f1a5010a870 2024-04-25T18:47:09.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.154+0000 7f1a4f400700 1 --2- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5019e6c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a40000c00 tx=0x7f1a40011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:09.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.155+0000 7f1a47e00700 1 -- 172.21.15.45:0/1040532335 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a40003aa0 con 0x7f1a5010a870 2024-04-25T18:47:09.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.155+0000 7f1a47e00700 1 -- 172.21.15.45:0/1040532335 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a4001b8f0 con 0x7f1a5010a870 2024-04-25T18:47:09.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.155+0000 7f1a47e00700 1 -- 172.21.15.45:0/1040532335 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a40023890 con 0x7f1a5010a870 2024-04-25T18:47:09.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.155+0000 7f1a56800700 1 -- 172.21.15.45:0/1040532335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5019ee90 con 0x7f1a5010a870 2024-04-25T18:47:09.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.155+0000 7f1a56800700 1 -- 172.21.15.45:0/1040532335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a501a1b80 con 0x7f1a5010a870 2024-04-25T18:47:09.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.156+0000 7f1a47e00700 1 -- 172.21.15.45:0/1040532335 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f1a4002c3f0 con 0x7f1a5010a870 2024-04-25T18:47:09.130 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.156+0000 7f1a56800700 1 -- 172.21.15.45:0/1040532335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a5010ef50 con 0x7f1a5010a870 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.157+0000 7f1a47e00700 1 --2- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f1a34039660 0x7f1a3403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.157+0000 7f1a47e00700 1 -- 172.21.15.45:0/1040532335 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1a4004e090 con 0x7f1a5010a870 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.157+0000 7f1a4ea00700 1 -- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f1a34039660 msgr2=0x7f1a3403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3065212608 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.157+0000 7f1a4ea00700 1 --2- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f1a34039660 0x7f1a3403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.162+0000 7f1a47e00700 1 -- 172.21.15.45:0/1040532335 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a4001f030 con 0x7f1a5010a870 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.358+0000 7f1a4ea00700 1 -- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f1a34039660 msgr2=0x7f1a3403bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3065212608 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.358+0000 7f1a4ea00700 1 --2- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f1a34039660 0x7f1a3403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.443+0000 7f1a56800700 1 -- 172.21.15.45:0/1040532335 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f1a501a2160 con 0x7f1a5010a870 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.444+0000 7f1a47e00700 1 -- 172.21.15.45:0/1040532335 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f1a4002b3a0 con 0x7f1a5010a870 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.447+0000 7f1a56800700 1 -- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f1a34039660 msgr2=0x7f1a3403bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.447+0000 7f1a56800700 1 --2- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f1a34039660 0x7f1a3403bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.447+0000 7f1a56800700 1 -- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 msgr2=0x7f1a5019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.447+0000 7f1a56800700 1 --2- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5019e6c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1a40000c00 tx=0x7f1a40011aa0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.447+0000 7f1a56800700 1 -- 172.21.15.45:0/1040532335 shutdown_connections 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.447+0000 7f1a56800700 1 --2- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f1a34039660 0x7f1a3403bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.131 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.447+0000 7f1a56800700 1 --2- 172.21.15.45:0/1040532335 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1a5010a870 0x7f1a5019e6c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:09.132 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.447+0000 7f1a56800700 1 -- 172.21.15.45:0/1040532335 >> 172.21.15.45:0/1040532335 conn(0x7f1a5007dca0 msgr2=0x7f1a50107b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:09.132 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.447+0000 7f1a56800700 1 -- 172.21.15.45:0/1040532335 shutdown_connections 2024-04-25T18:47:09.132 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:08.447+0000 7f1a56800700 1 -- 172.21.15.45:0/1040532335 wait complete. 2024-04-25T18:47:09.132 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for the mgr to restart... 2024-04-25T18:47:09.132 INFO:teuthology.orchestra.run.smithi045.stdout:Waiting for mgr epoch 9... 2024-04-25T18:47:09.997 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:09 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:09.760+0000 7f03725d8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:47:09.997 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:09 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:09.995+0000 7f03725d8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:47:10.338 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:10 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:10.213+0000 7f03725d8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:47:10.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:10 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:10.607+0000 7f03725d8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:47:10.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:10 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:10.731+0000 7f03725d8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:47:11.252 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:10 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:10.989+0000 7f03725d8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:47:11.252 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:11 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:11.250+0000 7f03725d8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:47:12.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:12 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:12.056+0000 7f03725d8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:47:12.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:12 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:12.626+0000 7f03725d8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:47:12.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:12 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:12.757+0000 7f03725d8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:47:13.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:13 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:13.029+0000 7f03725d8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:47:13.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:13 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:13.696+0000 7f03725d8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:47:13.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:13 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:13.825+0000 7f03725d8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:47:14.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:14 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:14.158+0000 7f03725d8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:47:14.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:14 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:14.342+0000 7f03725d8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:47:14.964 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:14 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:14.473+0000 7f03725d8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:47:14.964 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:14 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:14.630+0000 7f03725d8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:47:15.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:15 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:15.006+0000 7f03725d8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:47:16.092 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:15 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:15.794+0000 7f03725d8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:47:16.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:47:16.091+0000 7f03725d8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:47:16.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-mon[27776]: Active manager daemon x restarted 2024-04-25T18:47:16.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-mon[27776]: Activating manager daemon x 2024-04-25T18:47:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-mon[27776]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T18:47:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-mon[27776]: mgrmap e10: x(active, starting, since 0.00512835s) 2024-04-25T18:47:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:47:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:47:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:47:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:47:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:47:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:16 smithi045 ceph-mon[27776]: Manager daemon x is now available 2024-04-25T18:47:17.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:17.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:47:17.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:47:17.425 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:17 smithi045 ceph-mon[27776]: mgrmap e11: x(active, since 1.01003s) 2024-04-25T18:47:17.909 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout { 2024-04-25T18:47:17.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T18:47:17.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T18:47:17.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout } 2024-04-25T18:47:17.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.726+0000 7f4101000700 1 Processor -- start 2024-04-25T18:47:17.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.726+0000 7f4101000700 1 -- start start 2024-04-25T18:47:17.910 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.727+0000 7f4101000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc108ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.727+0000 7f4101000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc1090a0 con 0x7f40fc1086b0 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.727+0000 7f40f9e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc108ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.727+0000 7f40f9e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc108ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59302/0 (socket says 172.21.15.45:59302) 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.727+0000 7f40f9e00700 1 -- 172.21.15.45:0/1812284311 learned_addr learned my addr 172.21.15.45:0/1812284311 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.728+0000 7f40f9e00700 1 -- 172.21.15.45:0/1812284311 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc1098b0 con 0x7f40fc1086b0 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.728+0000 7f40f9e00700 1 --2- 172.21.15.45:0/1812284311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc108ad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0098a0 tx=0x7f40ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb5c0b8842c55b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.728+0000 7f40f8a00700 1 -- 172.21.15.45:0/1812284311 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40ec004160 con 0x7f40fc1086b0 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.728+0000 7f40f8a00700 1 -- 172.21.15.45:0/1812284311 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40ec004300 con 0x7f40fc1086b0 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.729+0000 7f40f8a00700 1 -- 172.21.15.45:0/1812284311 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40ec011a20 con 0x7f40fc1086b0 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.729+0000 7f4101000700 1 -- 172.21.15.45:0/1812284311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 msgr2=0x7f40fc108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:17.913 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.729+0000 7f4101000700 1 --2- 172.21.15.45:0/1812284311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc108ad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0098a0 tx=0x7f40ec009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:17.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.729+0000 7f4101000700 1 -- 172.21.15.45:0/1812284311 shutdown_connections 2024-04-25T18:47:17.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.729+0000 7f4101000700 1 --2- 172.21.15.45:0/1812284311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc108ad0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:17.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.729+0000 7f4101000700 1 -- 172.21.15.45:0/1812284311 >> 172.21.15.45:0/1812284311 conn(0x7f40fc103cb0 msgr2=0x7f40fc1060d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:17.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.730+0000 7f4101000700 1 -- 172.21.15.45:0/1812284311 shutdown_connections 2024-04-25T18:47:17.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.730+0000 7f4101000700 1 -- 172.21.15.45:0/1812284311 wait complete. 2024-04-25T18:47:17.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.731+0000 7f4101000700 1 Processor -- start 2024-04-25T18:47:17.914 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.731+0000 7f4101000700 1 -- start start 2024-04-25T18:47:17.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.731+0000 7f4101000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc07da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:17.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.731+0000 7f4101000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f40fc10cc60 con 0x7f40fc1086b0 2024-04-25T18:47:17.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.731+0000 7f40f9e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc07da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:17.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.731+0000 7f40f9e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc07da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:59310/0 (socket says 172.21.15.45:59310) 2024-04-25T18:47:17.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.731+0000 7f40f9e00700 1 -- 172.21.15.45:0/308559662 learned_addr learned my addr 172.21.15.45:0/308559662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:17.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.732+0000 7f40f9e00700 1 -- 172.21.15.45:0/308559662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40fc07dfa0 con 0x7f40fc1086b0 2024-04-25T18:47:17.918 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.732+0000 7f40f9e00700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc07da60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0119f0 tx=0x7f40ec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.732+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40ec0039e0 con 0x7f40fc1086b0 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.732+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40ec01b8f0 con 0x7f40fc1086b0 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.732+0000 7f4101000700 1 -- 172.21.15.45:0/308559662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40fc07c190 con 0x7f40fc1086b0 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.732+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40ec0237a0 con 0x7f40fc1086b0 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.733+0000 7f4101000700 1 -- 172.21.15.45:0/308559662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40fc07c6d0 con 0x7f40fc1086b0 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.734+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f40ec01b440 con 0x7f40fc1086b0 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.734+0000 7f40f2a00700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 0x7f40e003bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.734+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 --> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f40e003c1e0 con 0x7f40e0039610 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.735+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f40ec04df30 con 0x7f40fc1086b0 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.735+0000 7f40f9400700 1 -- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 msgr2=0x7f40e003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3065212608 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.735+0000 7f40f9400700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 0x7f40e003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.936+0000 7f40f9400700 1 -- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 msgr2=0x7f40e003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3065212608 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:09.936+0000 7f40f9400700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 0x7f40e003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:47:17.919 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:10.337+0000 7f40f9400700 1 -- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 msgr2=0x7f40e003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3065212608 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:10.337+0000 7f40f9400700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 0x7f40e003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:11.138+0000 7f40f9400700 1 -- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 msgr2=0x7f40e003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3065212608 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:11.138+0000 7f40f9400700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 0x7f40e003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:12.740+0000 7f40f9400700 1 -- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 msgr2=0x7f40e003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3065212608 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:12.740+0000 7f40f9400700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 0x7f40e003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:15.943+0000 7f40f9400700 1 -- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 msgr2=0x7f40e003bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/3065212608 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:15.944+0000 7f40f9400700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 0x7f40e003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:16.099+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f40ec016ca0 con 0x7f40fc1086b0 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:16.099+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 msgr2=0x7f40e003bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:16.099+0000 7f40f2a00700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 0x7f40e003bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.102+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f40ec0501a0 con 0x7f40fc1086b0 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.102+0000 7f40f2a00700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f40e003cff0 0x7f40e003f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.103+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f40e003c1e0 con 0x7f40e003cff0 2024-04-25T18:47:17.920 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.104+0000 7f40f9400700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f40e003cff0 0x7f40e003f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:17.922 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.107+0000 7f40f9400700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f40e003cff0 0x7f40e003f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f40e8003a10 tx=0x7f40e8009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.108+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f40e003c1e0 con 0x7f40e003cff0 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.117+0000 7f4101000700 1 -- 172.21.15.45:0/308559662 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f40fc0640a0 con 0x7f40e003cff0 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.117+0000 7f40f2a00700 1 -- 172.21.15.45:0/308559662 <== mgr.14152 v2:172.21.15.45:6800/943541413 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f40fc0640a0 con 0x7f40e003cff0 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.118+0000 7f4101000700 1 -- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f40e003cff0 msgr2=0x7f40e003f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.118+0000 7f4101000700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f40e003cff0 0x7f40e003f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f40e8003a10 tx=0x7f40e8009270 comp rx=0 tx=0).stop 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.118+0000 7f4101000700 1 -- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 msgr2=0x7f40fc07da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.118+0000 7f4101000700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc07da60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0119f0 tx=0x7f40ec011ad0 comp rx=0 tx=0).stop 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.118+0000 7f4101000700 1 -- 172.21.15.45:0/308559662 shutdown_connections 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.118+0000 7f4101000700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f40e003cff0 0x7f40e003f3e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.118+0000 7f4101000700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40fc1086b0 0x7f40fc07da60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.118+0000 7f4101000700 1 --2- 172.21.15.45:0/308559662 >> [v2:172.21.15.45:6800/3065212608,v1:172.21.15.45:6801/3065212608] conn(0x7f40e0039610 0x7f40e003bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.118+0000 7f4101000700 1 -- 172.21.15.45:0/308559662 >> 172.21.15.45:0/308559662 conn(0x7f40fc103cb0 msgr2=0x7f40fc104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.118+0000 7f4101000700 1 -- 172.21.15.45:0/308559662 shutdown_connections 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:17.119+0000 7f4101000700 1 -- 172.21.15.45:0/308559662 wait complete. 2024-04-25T18:47:17.923 INFO:teuthology.orchestra.run.smithi045.stdout:mgr epoch 9 is available 2024-04-25T18:47:17.924 INFO:teuthology.orchestra.run.smithi045.stdout:Generating a dashboard self-signed certificate... 2024-04-25T18:47:18.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:18 smithi045 ceph-mon[27776]: [25/Apr/2024:18:47:17] ENGINE Bus STARTING 2024-04-25T18:47:18.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:18 smithi045 ceph-mon[27776]: [25/Apr/2024:18:47:17] ENGINE Serving on https://172.21.15.45:7150 2024-04-25T18:47:18.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:18 smithi045 ceph-mon[27776]: [25/Apr/2024:18:47:17] ENGINE Serving on http://172.21.15.45:8765 2024-04-25T18:47:18.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:18 smithi045 ceph-mon[27776]: [25/Apr/2024:18:47:17] ENGINE Bus STARTED 2024-04-25T18:47:19.386 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:19 smithi045 ceph-mon[27776]: mgrmap e12: x(active, since 2s) 2024-04-25T18:47:19.386 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:20.113 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.128+0000 7ff2d5400700 1 Processor -- start 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.128+0000 7ff2d5400700 1 -- start start 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.128+0000 7ff2d5400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d010b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.129+0000 7ff2d5400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0063ce0 con 0x7ff2d0109130 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.129+0000 7ff2ce800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d010b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.129+0000 7ff2ce800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d010b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57480/0 (socket says 172.21.15.45:57480) 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.129+0000 7ff2ce800700 1 -- 172.21.15.45:0/1953009406 learned_addr learned my addr 172.21.15.45:0/1953009406 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.130+0000 7ff2ce800700 1 -- 172.21.15.45:0/1953009406 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d010ba60 con 0x7ff2d0109130 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.130+0000 7ff2ce800700 1 --2- 172.21.15.45:0/1953009406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d010b520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00098a0 tx=0x7ff2c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db61f2fc89bfa99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.130+0000 7ff2cde00700 1 -- 172.21.15.45:0/1953009406 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c0004160 con 0x7ff2d0109130 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.130+0000 7ff2cde00700 1 -- 172.21.15.45:0/1953009406 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c0004300 con 0x7ff2d0109130 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.130+0000 7ff2cde00700 1 -- 172.21.15.45:0/1953009406 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c0011a20 con 0x7ff2d0109130 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.131+0000 7ff2d5400700 1 -- 172.21.15.45:0/1953009406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 msgr2=0x7ff2d010b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:20.114 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.131+0000 7ff2d5400700 1 --2- 172.21.15.45:0/1953009406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d010b520 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00098a0 tx=0x7ff2c0009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.132+0000 7ff2d5400700 1 -- 172.21.15.45:0/1953009406 shutdown_connections 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.132+0000 7ff2d5400700 1 --2- 172.21.15.45:0/1953009406 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d010b520 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.132+0000 7ff2d5400700 1 -- 172.21.15.45:0/1953009406 >> 172.21.15.45:0/1953009406 conn(0x7ff2d0102b80 msgr2=0x7ff2d0104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.132+0000 7ff2d5400700 1 -- 172.21.15.45:0/1953009406 shutdown_connections 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.132+0000 7ff2d5400700 1 -- 172.21.15.45:0/1953009406 wait complete. 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.133+0000 7ff2d5400700 1 Processor -- start 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.133+0000 7ff2d5400700 1 -- start start 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.133+0000 7ff2d5400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.133+0000 7ff2d5400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d010ca00 con 0x7ff2d0109130 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.134+0000 7ff2ce800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.134+0000 7ff2ce800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57496/0 (socket says 172.21.15.45:57496) 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.134+0000 7ff2ce800700 1 -- 172.21.15.45:0/2488722921 learned_addr learned my addr 172.21.15.45:0/2488722921 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.134+0000 7ff2ce800700 1 -- 172.21.15.45:0/2488722921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d019ec20 con 0x7ff2d0109130 2024-04-25T18:47:20.115 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.135+0000 7ff2ce800700 1 --2- 172.21.15.45:0/2488722921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d019e6e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0004660 tx=0x7ff2c0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.135+0000 7ff2c7400700 1 -- 172.21.15.45:0/2488722921 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c0003aa0 con 0x7ff2d0109130 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.135+0000 7ff2c7400700 1 -- 172.21.15.45:0/2488722921 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c001b8f0 con 0x7ff2d0109130 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.135+0000 7ff2c7400700 1 -- 172.21.15.45:0/2488722921 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2c0023900 con 0x7ff2d0109130 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.135+0000 7ff2d5400700 1 -- 172.21.15.45:0/2488722921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d019eeb0 con 0x7ff2d0109130 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.135+0000 7ff2d5400700 1 -- 172.21.15.45:0/2488722921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d01a1ba0 con 0x7ff2d0109130 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.136+0000 7ff2d5400700 1 -- 172.21.15.45:0/2488722921 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d00640a0 con 0x7ff2d0109130 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.137+0000 7ff2c7400700 1 -- 172.21.15.45:0/2488722921 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7ff2c001baa0 con 0x7ff2d0109130 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.137+0000 7ff2c7400700 1 --2- 172.21.15.45:0/2488722921 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff2b4039660 0x7ff2b403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.137+0000 7ff2c7400700 1 -- 172.21.15.45:0/2488722921 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff2c004df20 con 0x7ff2d0109130 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.137+0000 7ff2c7e00700 1 --2- 172.21.15.45:0/2488722921 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff2b4039660 0x7ff2b403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.138+0000 7ff2c7e00700 1 --2- 172.21.15.45:0/2488722921 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff2b4039660 0x7ff2b403bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8006fd0 tx=0x7ff2b8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.143+0000 7ff2c7400700 1 -- 172.21.15.45:0/2488722921 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff2c001f030 con 0x7ff2d0109130 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.385+0000 7ff2d5400700 1 -- 172.21.15.45:0/2488722921 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ff2d0103d30 con 0x7ff2b4039660 2024-04-25T18:47:20.116 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.468+0000 7ff2c7400700 1 -- 172.21.15.45:0/2488722921 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ff2d0103d30 con 0x7ff2b4039660 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.471+0000 7ff2d5400700 1 -- 172.21.15.45:0/2488722921 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff2b4039660 msgr2=0x7ff2b403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.471+0000 7ff2d5400700 1 --2- 172.21.15.45:0/2488722921 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff2b4039660 0x7ff2b403bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8006fd0 tx=0x7ff2b8006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.471+0000 7ff2d5400700 1 -- 172.21.15.45:0/2488722921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 msgr2=0x7ff2d019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.471+0000 7ff2d5400700 1 --2- 172.21.15.45:0/2488722921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d019e6e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0004660 tx=0x7ff2c0011aa0 comp rx=0 tx=0).stop 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.472+0000 7ff2d5400700 1 -- 172.21.15.45:0/2488722921 shutdown_connections 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.472+0000 7ff2d5400700 1 --2- 172.21.15.45:0/2488722921 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff2b4039660 0x7ff2b403bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.472+0000 7ff2d5400700 1 --2- 172.21.15.45:0/2488722921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff2d0109130 0x7ff2d019e6e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.472+0000 7ff2d5400700 1 -- 172.21.15.45:0/2488722921 >> 172.21.15.45:0/2488722921 conn(0x7ff2d0102b80 msgr2=0x7ff2d0103620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.472+0000 7ff2d5400700 1 -- 172.21.15.45:0/2488722921 shutdown_connections 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:19.472+0000 7ff2d5400700 1 -- 172.21.15.45:0/2488722921 wait complete. 2024-04-25T18:47:20.117 INFO:teuthology.orchestra.run.smithi045.stdout:Creating initial admin user... 2024-04-25T18:47:20.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:20 smithi045 ceph-mon[27776]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:20.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:20.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:21.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$v9yMhBhazyZo2X9roOAqkel3vOclQ4xPiuGXe65ukiyoY1KON0H3G", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714070841, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T18:47:21.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.737+0000 7f0a6c800700 1 Processor -- start 2024-04-25T18:47:21.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.737+0000 7f0a6c800700 1 -- start start 2024-04-25T18:47:21.939 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.738+0000 7f0a6c800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a64108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.738+0000 7f0a6c800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a64109000 con 0x7f0a64108610 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.738+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a64108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.738+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a64108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57510/0 (socket says 172.21.15.45:57510) 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.738+0000 7f0a69e00700 1 -- 172.21.15.45:0/2287537490 learned_addr learned my addr 172.21.15.45:0/2287537490 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.739+0000 7f0a69e00700 1 -- 172.21.15.45:0/2287537490 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a64109810 con 0x7f0a64108610 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.739+0000 7f0a69e00700 1 --2- 172.21.15.45:0/2287537490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a64108a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0a580098a0 tx=0x7f0a58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f8b74909b31528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.739+0000 7f0a68a00700 1 -- 172.21.15.45:0/2287537490 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a58004160 con 0x7f0a64108610 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.740+0000 7f0a68a00700 1 -- 172.21.15.45:0/2287537490 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a58004300 con 0x7f0a64108610 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.740+0000 7f0a68a00700 1 -- 172.21.15.45:0/2287537490 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a58011a20 con 0x7f0a64108610 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.740+0000 7f0a6c800700 1 -- 172.21.15.45:0/2287537490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 msgr2=0x7f0a64108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.740+0000 7f0a6c800700 1 --2- 172.21.15.45:0/2287537490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a64108a30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0a580098a0 tx=0x7f0a58009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.741+0000 7f0a6c800700 1 -- 172.21.15.45:0/2287537490 shutdown_connections 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.741+0000 7f0a6c800700 1 --2- 172.21.15.45:0/2287537490 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a64108a30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:21.940 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.741+0000 7f0a6c800700 1 -- 172.21.15.45:0/2287537490 >> 172.21.15.45:0/2287537490 conn(0x7f0a64103c70 msgr2=0x7f0a64106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.741+0000 7f0a6c800700 1 -- 172.21.15.45:0/2287537490 shutdown_connections 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.741+0000 7f0a6c800700 1 -- 172.21.15.45:0/2287537490 wait complete. 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.742+0000 7f0a6c800700 1 Processor -- start 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.742+0000 7f0a6c800700 1 -- start start 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.742+0000 7f0a6c800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a6419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.742+0000 7f0a6c800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6410cbc0 con 0x7f0a64108610 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.743+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a6419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.743+0000 7f0a69e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a6419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57520/0 (socket says 172.21.15.45:57520) 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.743+0000 7f0a69e00700 1 -- 172.21.15.45:0/2498872250 learned_addr learned my addr 172.21.15.45:0/2498872250 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.743+0000 7f0a69e00700 1 -- 172.21.15.45:0/2498872250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6419ec70 con 0x7f0a64108610 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.743+0000 7f0a69e00700 1 --2- 172.21.15.45:0/2498872250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a6419e730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a580119f0 tx=0x7f0a58011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.744+0000 7f0a5ea00700 1 -- 172.21.15.45:0/2498872250 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a580039e0 con 0x7f0a64108610 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.744+0000 7f0a5ea00700 1 -- 172.21.15.45:0/2498872250 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a5801b8f0 con 0x7f0a64108610 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.744+0000 7f0a5ea00700 1 -- 172.21.15.45:0/2498872250 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0a580237a0 con 0x7f0a64108610 2024-04-25T18:47:21.941 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.744+0000 7f0a6c800700 1 -- 172.21.15.45:0/2498872250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6419ef00 con 0x7f0a64108610 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.744+0000 7f0a6c800700 1 -- 172.21.15.45:0/2498872250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a641a1bf0 con 0x7f0a64108610 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.745+0000 7f0a6c800700 1 -- 172.21.15.45:0/2498872250 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6410cd40 con 0x7f0a64108610 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.746+0000 7f0a5ea00700 1 -- 172.21.15.45:0/2498872250 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f0a5801b440 con 0x7f0a64108610 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.746+0000 7f0a5ea00700 1 --2- 172.21.15.45:0/2498872250 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0a4c0395b0 0x7f0a4c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.746+0000 7f0a5ea00700 1 -- 172.21.15.45:0/2498872250 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0a5804dfe0 con 0x7f0a64108610 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.746+0000 7f0a69400700 1 --2- 172.21.15.45:0/2498872250 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0a4c0395b0 0x7f0a4c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.747+0000 7f0a69400700 1 --2- 172.21.15.45:0/2498872250 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0a4c0395b0 0x7f0a4c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a54006fd0 tx=0x7f0a54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:20.752+0000 7f0a5ea00700 1 -- 172.21.15.45:0/2498872250 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0a5801f030 con 0x7f0a64108610 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.004+0000 7f0a6c800700 1 -- 172.21.15.45:0/2498872250 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f0a64104df0 con 0x7f0a4c0395b0 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.279+0000 7f0a5ea00700 1 -- 172.21.15.45:0/2498872250 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f0a64104df0 con 0x7f0a4c0395b0 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.282+0000 7f0a6c800700 1 -- 172.21.15.45:0/2498872250 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0a4c0395b0 msgr2=0x7f0a4c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.282+0000 7f0a6c800700 1 --2- 172.21.15.45:0/2498872250 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0a4c0395b0 0x7f0a4c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a54006fd0 tx=0x7f0a54006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.282+0000 7f0a6c800700 1 -- 172.21.15.45:0/2498872250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 msgr2=0x7f0a6419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:21.942 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.282+0000 7f0a6c800700 1 --2- 172.21.15.45:0/2498872250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a6419e730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a580119f0 tx=0x7f0a58011ad0 comp rx=0 tx=0).stop 2024-04-25T18:47:21.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.282+0000 7f0a6c800700 1 -- 172.21.15.45:0/2498872250 shutdown_connections 2024-04-25T18:47:21.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.282+0000 7f0a6c800700 1 --2- 172.21.15.45:0/2498872250 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0a4c0395b0 0x7f0a4c03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:21.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.282+0000 7f0a6c800700 1 --2- 172.21.15.45:0/2498872250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0a64108610 0x7f0a6419e730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:21.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.282+0000 7f0a6c800700 1 -- 172.21.15.45:0/2498872250 >> 172.21.15.45:0/2498872250 conn(0x7f0a64103c70 msgr2=0x7f0a641046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:21.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.282+0000 7f0a6c800700 1 -- 172.21.15.45:0/2498872250 shutdown_connections 2024-04-25T18:47:21.943 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:21.283+0000 7f0a6c800700 1 -- 172.21.15.45:0/2498872250 wait complete. 2024-04-25T18:47:21.943 INFO:teuthology.orchestra.run.smithi045.stdout:Fetching dashboard port number... 2024-04-25T18:47:22.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:21 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:22.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:21 smithi045 ceph-mon[27776]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:22.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:21 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:23.044 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:22 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1131755416' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T18:47:23.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T18:47:23.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.570+0000 7fa688e00700 1 Processor -- start 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.571+0000 7fa688e00700 1 -- start start 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.571+0000 7fa688e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68407c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.571+0000 7fa688e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa68407c780 con 0x7fa68407dd50 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.572+0000 7fa681e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68407c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.572+0000 7fa681e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68407c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57524/0 (socket says 172.21.15.45:57524) 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.572+0000 7fa681e00700 1 -- 172.21.15.45:0/3786810174 learned_addr learned my addr 172.21.15.45:0/3786810174 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.572+0000 7fa681e00700 1 -- 172.21.15.45:0/3786810174 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68407c900 con 0x7fa68407dd50 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.573+0000 7fa681e00700 1 --2- 172.21.15.45:0/3786810174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68407c1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6740098a0 tx=0x7fa674009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc366c42414be08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.573+0000 7fa680a00700 1 -- 172.21.15.45:0/3786810174 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa674004160 con 0x7fa68407dd50 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.573+0000 7fa680a00700 1 -- 172.21.15.45:0/3786810174 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa674004300 con 0x7fa68407dd50 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.573+0000 7fa680a00700 1 -- 172.21.15.45:0/3786810174 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa674011a20 con 0x7fa68407dd50 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.574+0000 7fa688e00700 1 -- 172.21.15.45:0/3786810174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 msgr2=0x7fa68407c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.574+0000 7fa688e00700 1 --2- 172.21.15.45:0/3786810174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68407c1b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa6740098a0 tx=0x7fa674009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.574+0000 7fa688e00700 1 -- 172.21.15.45:0/3786810174 shutdown_connections 2024-04-25T18:47:23.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.574+0000 7fa688e00700 1 --2- 172.21.15.45:0/3786810174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68407c1b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:23.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.574+0000 7fa688e00700 1 -- 172.21.15.45:0/3786810174 >> 172.21.15.45:0/3786810174 conn(0x7fa684103c70 msgr2=0x7fa684106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:23.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.574+0000 7fa688e00700 1 -- 172.21.15.45:0/3786810174 shutdown_connections 2024-04-25T18:47:23.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.574+0000 7fa688e00700 1 -- 172.21.15.45:0/3786810174 wait complete. 2024-04-25T18:47:23.849 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.575+0000 7fa688e00700 1 Processor -- start 2024-04-25T18:47:23.849 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.576+0000 7fa688e00700 1 -- start start 2024-04-25T18:47:23.849 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.576+0000 7fa688e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:23.849 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.576+0000 7fa688e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa68410ebf0 con 0x7fa68407dd50 2024-04-25T18:47:23.849 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.576+0000 7fa681e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:23.849 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.576+0000 7fa681e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57528/0 (socket says 172.21.15.45:57528) 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.576+0000 7fa681e00700 1 -- 172.21.15.45:0/1131755416 learned_addr learned my addr 172.21.15.45:0/1131755416 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.577+0000 7fa681e00700 1 -- 172.21.15.45:0/1131755416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68419ebe0 con 0x7fa68407dd50 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.577+0000 7fa681e00700 1 --2- 172.21.15.45:0/1131755416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68419e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa674004660 tx=0x7fa674011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.577+0000 7fa67aa00700 1 -- 172.21.15.45:0/1131755416 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa674003aa0 con 0x7fa68407dd50 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.577+0000 7fa67aa00700 1 -- 172.21.15.45:0/1131755416 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa67401b8f0 con 0x7fa68407dd50 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.577+0000 7fa67aa00700 1 -- 172.21.15.45:0/1131755416 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa674023900 con 0x7fa68407dd50 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.577+0000 7fa688e00700 1 -- 172.21.15.45:0/1131755416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68419ee70 con 0x7fa68407dd50 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.577+0000 7fa688e00700 1 -- 172.21.15.45:0/1131755416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6841a1b60 con 0x7fa68407dd50 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.579+0000 7fa67aa00700 1 -- 172.21.15.45:0/1131755416 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fa674023bc0 con 0x7fa68407dd50 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.579+0000 7fa688e00700 1 -- 172.21.15.45:0/1131755416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6840640a0 con 0x7fa68407dd50 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.579+0000 7fa67aa00700 1 --2- 172.21.15.45:0/1131755416 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa668039600 0x7fa66803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.579+0000 7fa67aa00700 1 -- 172.21.15.45:0/1131755416 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa67404e030 con 0x7fa68407dd50 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.580+0000 7fa681400700 1 --2- 172.21.15.45:0/1131755416 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa668039600 0x7fa66803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:23.850 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.580+0000 7fa681400700 1 --2- 172.21.15.45:0/1131755416 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa668039600 0x7fa66803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa670006fd0 tx=0x7fa670006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.586+0000 7fa67aa00700 1 -- 172.21.15.45:0/1131755416 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa67401f030 con 0x7fa68407dd50 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.824+0000 7fa688e00700 1 -- 172.21.15.45:0/1131755416 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fa6841a2150 con 0x7fa68407dd50 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.824+0000 7fa67aa00700 1 -- 172.21.15.45:0/1131755416 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fa6740292a0 con 0x7fa68407dd50 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.827+0000 7fa688e00700 1 -- 172.21.15.45:0/1131755416 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa668039600 msgr2=0x7fa66803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.827+0000 7fa688e00700 1 --2- 172.21.15.45:0/1131755416 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa668039600 0x7fa66803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa670006fd0 tx=0x7fa670006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.827+0000 7fa688e00700 1 -- 172.21.15.45:0/1131755416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 msgr2=0x7fa68419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.827+0000 7fa688e00700 1 --2- 172.21.15.45:0/1131755416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68419e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa674004660 tx=0x7fa674011aa0 comp rx=0 tx=0).stop 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.827+0000 7fa688e00700 1 -- 172.21.15.45:0/1131755416 shutdown_connections 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.827+0000 7fa688e00700 1 --2- 172.21.15.45:0/1131755416 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa668039600 0x7fa66803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.827+0000 7fa688e00700 1 --2- 172.21.15.45:0/1131755416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa68407dd50 0x7fa68419e6a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.827+0000 7fa688e00700 1 -- 172.21.15.45:0/1131755416 >> 172.21.15.45:0/1131755416 conn(0x7fa684103c70 msgr2=0x7fa6841046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.828+0000 7fa688e00700 1 -- 172.21.15.45:0/1131755416 shutdown_connections 2024-04-25T18:47:23.851 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:22.828+0000 7fa688e00700 1 -- 172.21.15.45:0/1131755416 wait complete. 2024-04-25T18:47:23.876 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T18:47:23.877 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout disabled 2024-04-25T18:47:23.901 INFO:teuthology.orchestra.run.smithi045.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T18:47:23.901 INFO:teuthology.orchestra.run.smithi045.stdout:systemctl: stdout inactive 2024-04-25T18:47:23.901 INFO:teuthology.orchestra.run.smithi045.stdout:firewalld.service is not enabled 2024-04-25T18:47:23.901 INFO:teuthology.orchestra.run.smithi045.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T18:47:23.908 INFO:teuthology.orchestra.run.smithi045.stdout:Ceph Dashboard is now available at: 2024-04-25T18:47:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: URL: https://smithi045.front.sepia.ceph.com:8443/ 2024-04-25T18:47:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: User: admin 2024-04-25T18:47:23.908 INFO:teuthology.orchestra.run.smithi045.stdout: Password: giwv5i5eu1 2024-04-25T18:47:23.909 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:23.909 INFO:teuthology.orchestra.run.smithi045.stdout:Saving cluster configuration to /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config directory 2024-04-25T18:47:23.912 INFO:teuthology.orchestra.run.smithi045.stdout:Enabling autotune for osd_memory_target 2024-04-25T18:47:24.108 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:23 smithi045 ceph-mon[27776]: mgrmap e13: x(active, since 6s) 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.004+0000 7f9209600700 1 Processor -- start 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.005+0000 7f9209600700 1 -- start start 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.005+0000 7f9209600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920410ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.005+0000 7f9209600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f920410b220 con 0x7f920410a830 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.005+0000 7f9202a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920410ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.005+0000 7f9202a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920410ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32912/0 (socket says 172.21.15.45:32912) 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.005+0000 7f9202a00700 1 -- 172.21.15.45:0/2318287371 learned_addr learned my addr 172.21.15.45:0/2318287371 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.006+0000 7f9202a00700 1 -- 172.21.15.45:0/2318287371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920410ba00 con 0x7f920410a830 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.006+0000 7f9202a00700 1 --2- 172.21.15.45:0/2318287371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920410ac50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f91f40098a0 tx=0x7f91f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c693de59118f548 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.007+0000 7f9201600700 1 -- 172.21.15.45:0/2318287371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91f4004160 con 0x7f920410a830 2024-04-25T18:47:25.843 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.007+0000 7f9201600700 1 -- 172.21.15.45:0/2318287371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f4004300 con 0x7f920410a830 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.007+0000 7f9201600700 1 -- 172.21.15.45:0/2318287371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91f4011a20 con 0x7f920410a830 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.007+0000 7f9209600700 1 -- 172.21.15.45:0/2318287371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 msgr2=0x7f920410ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.007+0000 7f9209600700 1 --2- 172.21.15.45:0/2318287371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920410ac50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f91f40098a0 tx=0x7f91f4009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.007+0000 7f9209600700 1 -- 172.21.15.45:0/2318287371 shutdown_connections 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.007+0000 7f9209600700 1 --2- 172.21.15.45:0/2318287371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920410ac50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.008+0000 7f9209600700 1 -- 172.21.15.45:0/2318287371 >> 172.21.15.45:0/2318287371 conn(0x7f920407dbc0 msgr2=0x7f920407dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.008+0000 7f9209600700 1 -- 172.21.15.45:0/2318287371 shutdown_connections 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.008+0000 7f9209600700 1 -- 172.21.15.45:0/2318287371 wait complete. 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.009+0000 7f9209600700 1 Processor -- start 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.009+0000 7f9209600700 1 -- start start 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.009+0000 7f9209600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.009+0000 7f9209600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f920410edb0 con 0x7f920410a830 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.010+0000 7f9202a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.010+0000 7f9202a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920419e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32920/0 (socket says 172.21.15.45:32920) 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.010+0000 7f9202a00700 1 -- 172.21.15.45:0/749639972 learned_addr learned my addr 172.21.15.45:0/749639972 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:25.844 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.010+0000 7f9202a00700 1 -- 172.21.15.45:0/749639972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920419ec40 con 0x7f920410a830 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.010+0000 7f9202a00700 1 --2- 172.21.15.45:0/749639972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920419e700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91f4000c00 tx=0x7f91f4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.011+0000 7f91fb400700 1 -- 172.21.15.45:0/749639972 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91f4003aa0 con 0x7f920410a830 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.011+0000 7f91fb400700 1 -- 172.21.15.45:0/749639972 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f401b8f0 con 0x7f920410a830 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.011+0000 7f91fb400700 1 -- 172.21.15.45:0/749639972 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f91f4023890 con 0x7f920410a830 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.011+0000 7f9209600700 1 -- 172.21.15.45:0/749639972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920419eed0 con 0x7f920410a830 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.011+0000 7f9209600700 1 -- 172.21.15.45:0/749639972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92041a1bc0 con 0x7f920410a830 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.012+0000 7f9209600700 1 -- 172.21.15.45:0/749639972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920410ef30 con 0x7f920410a830 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.012+0000 7f91fb400700 1 -- 172.21.15.45:0/749639972 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f91f401b440 con 0x7f920410a830 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.012+0000 7f91fb400700 1 --2- 172.21.15.45:0/749639972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91e8039700 0x7f91e803bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.013+0000 7f91fb400700 1 -- 172.21.15.45:0/749639972 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f91f404e0e0 con 0x7f920410a830 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.013+0000 7f9202000700 1 --2- 172.21.15.45:0/749639972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91e8039700 0x7f91e803bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.013+0000 7f9202000700 1 --2- 172.21.15.45:0/749639972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91e8039700 0x7f91e803bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91f0006fd0 tx=0x7f91f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.018+0000 7f91fb400700 1 -- 172.21.15.45:0/749639972 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f91f401f030 con 0x7f920410a830 2024-04-25T18:47:25.845 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.247+0000 7f9209600700 1 -- 172.21.15.45:0/749639972 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f92041a2210 con 0x7f920410a830 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.248+0000 7f91fb400700 1 -- 172.21.15.45:0/749639972 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f91f40185b0 con 0x7f920410a830 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.250+0000 7f9209600700 1 -- 172.21.15.45:0/749639972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91e8039700 msgr2=0x7f91e803bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.250+0000 7f9209600700 1 --2- 172.21.15.45:0/749639972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91e8039700 0x7f91e803bbc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f91f0006fd0 tx=0x7f91f0006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.251+0000 7f9209600700 1 -- 172.21.15.45:0/749639972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 msgr2=0x7f920419e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.251+0000 7f9209600700 1 --2- 172.21.15.45:0/749639972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920419e700 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f91f4000c00 tx=0x7f91f4011aa0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.251+0000 7f9209600700 1 -- 172.21.15.45:0/749639972 shutdown_connections 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.251+0000 7f9209600700 1 --2- 172.21.15.45:0/749639972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91e8039700 0x7f91e803bbc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.251+0000 7f9209600700 1 --2- 172.21.15.45:0/749639972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f920410a830 0x7f920419e700 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.251+0000 7f9209600700 1 -- 172.21.15.45:0/749639972 >> 172.21.15.45:0/749639972 conn(0x7f920407dbc0 msgr2=0x7f9204108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.251+0000 7f9209600700 1 -- 172.21.15.45:0/749639972 shutdown_connections 2024-04-25T18:47:25.846 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:25.251+0000 7f9209600700 1 -- 172.21.15.45:0/749639972 wait complete. 2024-04-25T18:47:27.422 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.443+0000 7f8c14c00700 1 Processor -- start 2024-04-25T18:47:27.423 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.443+0000 7f8c14c00700 1 -- start start 2024-04-25T18:47:27.423 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.443+0000 7f8c14c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c1010b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.423 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.443+0000 7f8c14c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8c10063d20 con 0x7f8c10108ef0 2024-04-25T18:47:27.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.444+0000 7f8c0f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c1010b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.444+0000 7f8c0f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c1010b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32930/0 (socket says 172.21.15.45:32930) 2024-04-25T18:47:27.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.444+0000 7f8c0f400700 1 -- 172.21.15.45:0/206656153 learned_addr learned my addr 172.21.15.45:0/206656153 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:27.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.444+0000 7f8c0f400700 1 -- 172.21.15.45:0/206656153 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c1010b820 con 0x7f8c10108ef0 2024-04-25T18:47:27.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.445+0000 7f8c0f400700 1 --2- 172.21.15.45:0/206656153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c1010b2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8c000098a0 tx=0x7f8c00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3932f009198ebb92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:27.424 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.445+0000 7f8c0ea00700 1 -- 172.21.15.45:0/206656153 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c00004160 con 0x7f8c10108ef0 2024-04-25T18:47:27.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.445+0000 7f8c0ea00700 1 -- 172.21.15.45:0/206656153 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c00004300 con 0x7f8c10108ef0 2024-04-25T18:47:27.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.445+0000 7f8c0ea00700 1 -- 172.21.15.45:0/206656153 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c00011a20 con 0x7f8c10108ef0 2024-04-25T18:47:27.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.446+0000 7f8c14c00700 1 -- 172.21.15.45:0/206656153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 msgr2=0x7f8c1010b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.425 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.446+0000 7f8c14c00700 1 --2- 172.21.15.45:0/206656153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c1010b2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8c000098a0 tx=0x7f8c00009bb0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.426 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.446+0000 7f8c14c00700 1 -- 172.21.15.45:0/206656153 shutdown_connections 2024-04-25T18:47:27.426 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.446+0000 7f8c14c00700 1 --2- 172.21.15.45:0/206656153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c1010b2e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.426 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.446+0000 7f8c14c00700 1 -- 172.21.15.45:0/206656153 >> 172.21.15.45:0/206656153 conn(0x7f8c10102960 msgr2=0x7f8c10104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.446+0000 7f8c14c00700 1 -- 172.21.15.45:0/206656153 shutdown_connections 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.446+0000 7f8c14c00700 1 -- 172.21.15.45:0/206656153 wait complete. 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.447+0000 7f8c14c00700 1 Processor -- start 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.448+0000 7f8c14c00700 1 -- start start 2024-04-25T18:47:27.427 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.448+0000 7f8c14c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c10108770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.428 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.448+0000 7f8c14c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1010c820 con 0x7f8c10108ef0 2024-04-25T18:47:27.428 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.448+0000 7f8c0f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c10108770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.428 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.448+0000 7f8c0f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c10108770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32932/0 (socket says 172.21.15.45:32932) 2024-04-25T18:47:27.428 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.448+0000 7f8c0f400700 1 -- 172.21.15.45:0/629480014 learned_addr learned my addr 172.21.15.45:0/629480014 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:27.428 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.449+0000 7f8c0f400700 1 -- 172.21.15.45:0/629480014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c10108cb0 con 0x7f8c10108ef0 2024-04-25T18:47:27.428 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.449+0000 7f8c0f400700 1 --2- 172.21.15.45:0/629480014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c10108770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c00004660 tx=0x7f8c00011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.449+0000 7f8c0cc00700 1 -- 172.21.15.45:0/629480014 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c00003aa0 con 0x7f8c10108ef0 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.449+0000 7f8c0cc00700 1 -- 172.21.15.45:0/629480014 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c0001b8f0 con 0x7f8c10108ef0 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.449+0000 7f8c14c00700 1 -- 172.21.15.45:0/629480014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c10106ea0 con 0x7f8c10108ef0 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.449+0000 7f8c0cc00700 1 -- 172.21.15.45:0/629480014 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c00023900 con 0x7f8c10108ef0 2024-04-25T18:47:27.429 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.449+0000 7f8c14c00700 1 -- 172.21.15.45:0/629480014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c101073e0 con 0x7f8c10108ef0 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.451+0000 7f8c0cc00700 1 -- 172.21.15.45:0/629480014 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8c0001baa0 con 0x7f8c10108ef0 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.451+0000 7f8c14c00700 1 -- 172.21.15.45:0/629480014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c100640a0 con 0x7f8c10108ef0 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.451+0000 7f8c0cc00700 1 --2- 172.21.15.45:0/629480014 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8bf4039700 0x7f8bf403bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.451+0000 7f8c0cc00700 1 -- 172.21.15.45:0/629480014 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8c0004e010 con 0x7f8c10108ef0 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.451+0000 7f8c07e00700 1 --2- 172.21.15.45:0/629480014 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8bf4039700 0x7f8bf403bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.452+0000 7f8c07e00700 1 --2- 172.21.15.45:0/629480014 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8bf4039700 0x7f8bf403bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8006fd0 tx=0x7f8bf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.457+0000 7f8c0cc00700 1 -- 172.21.15.45:0/629480014 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c0001f030 con 0x7f8c10108ef0 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.810+0000 7f8c14c00700 1 -- 172.21.15.45:0/629480014 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f8c10107660 con 0x7f8c10108ef0 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.814+0000 7f8c0cc00700 1 -- 172.21.15.45:0/629480014 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f8c00018350 con 0x7f8c10108ef0 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.817+0000 7f8c14c00700 1 -- 172.21.15.45:0/629480014 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8bf4039700 msgr2=0x7f8bf403bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.817+0000 7f8c14c00700 1 --2- 172.21.15.45:0/629480014 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8bf4039700 0x7f8bf403bbc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8006fd0 tx=0x7f8bf8006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:27.432 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.817+0000 7f8c14c00700 1 -- 172.21.15.45:0/629480014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 msgr2=0x7f8c10108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:27.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.817+0000 7f8c14c00700 1 --2- 172.21.15.45:0/629480014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c10108770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8c00004660 tx=0x7f8c00011aa0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.818+0000 7f8c14c00700 1 -- 172.21.15.45:0/629480014 shutdown_connections 2024-04-25T18:47:27.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.818+0000 7f8c14c00700 1 --2- 172.21.15.45:0/629480014 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8bf4039700 0x7f8bf403bbc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.818+0000 7f8c14c00700 1 --2- 172.21.15.45:0/629480014 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8c10108ef0 0x7f8c10108770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:27.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.818+0000 7f8c14c00700 1 -- 172.21.15.45:0/629480014 >> 172.21.15.45:0/629480014 conn(0x7f8c10102960 msgr2=0x7f8c10103410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:27.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.818+0000 7f8c14c00700 1 -- 172.21.15.45:0/629480014 shutdown_connections 2024-04-25T18:47:27.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr 2024-04-25T18:47:26.818+0000 7f8c14c00700 1 -- 172.21.15.45:0/629480014 wait complete. 2024-04-25T18:47:27.433 INFO:teuthology.orchestra.run.smithi045.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: ceph telemetry on 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:27.434 INFO:teuthology.orchestra.run.smithi045.stdout:For more information see: 2024-04-25T18:47:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T18:47:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:27.435 INFO:teuthology.orchestra.run.smithi045.stdout:Bootstrap complete. 2024-04-25T18:47:27.541 INFO:tasks.cephadm:Fetching config... 2024-04-25T18:47:27.541 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:47:27.541 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T18:47:27.593 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T18:47:27.593 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:47:27.593 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T18:47:27.639 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T18:47:27.639 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:47:27.639 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T18:47:27.746 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T18:47:27.746 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:47:27.746 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T18:47:27.840 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T18:47:27.841 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDExa0soJz1qBmgK4LdK81Z5ovpCPaZP9w0Inkioej3e6VrKUDAgw3+bC4M9zgf9u8n0+Fa4zV9SNAl9qzkF0tropb4sa/J3uUHculs5lNGNd2AAFgMns/ZDaoXHGHKH7mGyKAg+Db8LxfIubbAvqwPleQ9CjSpmMCrBbVWDFjmZPUpBd5tP9Z2sXzi218CyCj4P7KhADYH3jiMZlhOzuvEFUVyAVee/qJT3AaHGyqbHXl0ZhJqrjJxF0elBOtAhDUiU+0XHwFyv0L2vY8//2WrL17pmdfuVKwVT4yAjZXB5MdkbpjWl8QcLs3PKvAwxk3kQ8iObiErcIFsdZkzferLlHn245mx1fth5yHKn6qswARHNZTWUZq5WID37f4yav/RXu6v1o2DdwlJDNLDTwSU27vfsC6Ke4t+7cCUSuAvUu+8g+Hc4mI5olrG5UknHiEhVH3fN2W2IGNq/7LQjWwhJ+pFqo6oLEQm5BTD6EVf2n00Tmf204Dv7R7I142bHSM= ceph-ea2833ba-0333-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T18:47:27.962 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDExa0soJz1qBmgK4LdK81Z5ovpCPaZP9w0Inkioej3e6VrKUDAgw3+bC4M9zgf9u8n0+Fa4zV9SNAl9qzkF0tropb4sa/J3uUHculs5lNGNd2AAFgMns/ZDaoXHGHKH7mGyKAg+Db8LxfIubbAvqwPleQ9CjSpmMCrBbVWDFjmZPUpBd5tP9Z2sXzi218CyCj4P7KhADYH3jiMZlhOzuvEFUVyAVee/qJT3AaHGyqbHXl0ZhJqrjJxF0elBOtAhDUiU+0XHwFyv0L2vY8//2WrL17pmdfuVKwVT4yAjZXB5MdkbpjWl8QcLs3PKvAwxk3kQ8iObiErcIFsdZkzferLlHn245mx1fth5yHKn6qswARHNZTWUZq5WID37f4yav/RXu6v1o2DdwlJDNLDTwSU27vfsC6Ke4t+7cCUSuAvUu+8g+Hc4mI5olrG5UknHiEhVH3fN2W2IGNq/7LQjWwhJ+pFqo6oLEQm5BTD6EVf2n00Tmf204Dv7R7I142bHSM= ceph-ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:47:27.992 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:27 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/629480014' entity='client.admin' 2024-04-25T18:47:28.007 DEBUG:teuthology.orchestra.run.smithi062:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDExa0soJz1qBmgK4LdK81Z5ovpCPaZP9w0Inkioej3e6VrKUDAgw3+bC4M9zgf9u8n0+Fa4zV9SNAl9qzkF0tropb4sa/J3uUHculs5lNGNd2AAFgMns/ZDaoXHGHKH7mGyKAg+Db8LxfIubbAvqwPleQ9CjSpmMCrBbVWDFjmZPUpBd5tP9Z2sXzi218CyCj4P7KhADYH3jiMZlhOzuvEFUVyAVee/qJT3AaHGyqbHXl0ZhJqrjJxF0elBOtAhDUiU+0XHwFyv0L2vY8//2WrL17pmdfuVKwVT4yAjZXB5MdkbpjWl8QcLs3PKvAwxk3kQ8iObiErcIFsdZkzferLlHn245mx1fth5yHKn6qswARHNZTWUZq5WID37f4yav/RXu6v1o2DdwlJDNLDTwSU27vfsC6Ke4t+7cCUSuAvUu+8g+Hc4mI5olrG5UknHiEhVH3fN2W2IGNq/7LQjWwhJ+pFqo6oLEQm5BTD6EVf2n00Tmf204Dv7R7I142bHSM= ceph-ea2833ba-0333-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T18:47:28.048 INFO:teuthology.orchestra.run.smithi062.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDExa0soJz1qBmgK4LdK81Z5ovpCPaZP9w0Inkioej3e6VrKUDAgw3+bC4M9zgf9u8n0+Fa4zV9SNAl9qzkF0tropb4sa/J3uUHculs5lNGNd2AAFgMns/ZDaoXHGHKH7mGyKAg+Db8LxfIubbAvqwPleQ9CjSpmMCrBbVWDFjmZPUpBd5tP9Z2sXzi218CyCj4P7KhADYH3jiMZlhOzuvEFUVyAVee/qJT3AaHGyqbHXl0ZhJqrjJxF0elBOtAhDUiU+0XHwFyv0L2vY8//2WrL17pmdfuVKwVT4yAjZXB5MdkbpjWl8QcLs3PKvAwxk3kQ8iObiErcIFsdZkzferLlHn245mx1fth5yHKn6qswARHNZTWUZq5WID37f4yav/RXu6v1o2DdwlJDNLDTwSU27vfsC6Ke4t+7cCUSuAvUu+8g+Hc4mI5olrG5UknHiEhVH3fN2W2IGNq/7LQjWwhJ+pFqo6oLEQm5BTD6EVf2n00Tmf204Dv7R7I142bHSM= ceph-ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:47:28.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDExa0soJz1qBmgK4LdK81Z5ovpCPaZP9w0Inkioej3e6VrKUDAgw3+bC4M9zgf9u8n0+Fa4zV9SNAl9qzkF0tropb4sa/J3uUHculs5lNGNd2AAFgMns/ZDaoXHGHKH7mGyKAg+Db8LxfIubbAvqwPleQ9CjSpmMCrBbVWDFjmZPUpBd5tP9Z2sXzi218CyCj4P7KhADYH3jiMZlhOzuvEFUVyAVee/qJT3AaHGyqbHXl0ZhJqrjJxF0elBOtAhDUiU+0XHwFyv0L2vY8//2WrL17pmdfuVKwVT4yAjZXB5MdkbpjWl8QcLs3PKvAwxk3kQ8iObiErcIFsdZkzferLlHn245mx1fth5yHKn6qswARHNZTWUZq5WID37f4yav/RXu6v1o2DdwlJDNLDTwSU27vfsC6Ke4t+7cCUSuAvUu+8g+Hc4mI5olrG5UknHiEhVH3fN2W2IGNq/7LQjWwhJ+pFqo6oLEQm5BTD6EVf2n00Tmf204Dv7R7I142bHSM= ceph-ea2833ba-0333-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T18:47:28.099 INFO:teuthology.orchestra.run.smithi079.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDExa0soJz1qBmgK4LdK81Z5ovpCPaZP9w0Inkioej3e6VrKUDAgw3+bC4M9zgf9u8n0+Fa4zV9SNAl9qzkF0tropb4sa/J3uUHculs5lNGNd2AAFgMns/ZDaoXHGHKH7mGyKAg+Db8LxfIubbAvqwPleQ9CjSpmMCrBbVWDFjmZPUpBd5tP9Z2sXzi218CyCj4P7KhADYH3jiMZlhOzuvEFUVyAVee/qJT3AaHGyqbHXl0ZhJqrjJxF0elBOtAhDUiU+0XHwFyv0L2vY8//2WrL17pmdfuVKwVT4yAjZXB5MdkbpjWl8QcLs3PKvAwxk3kQ8iObiErcIFsdZkzferLlHn245mx1fth5yHKn6qswARHNZTWUZq5WID37f4yav/RXu6v1o2DdwlJDNLDTwSU27vfsC6Ke4t+7cCUSuAvUu+8g+Hc4mI5olrG5UknHiEhVH3fN2W2IGNq/7LQjWwhJ+pFqo6oLEQm5BTD6EVf2n00Tmf204Dv7R7I142bHSM= ceph-ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:47:28.110 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T18:47:28.558 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.090+0000 7f02a2600700 1 -- 172.21.15.45:0/1607095927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f029c103bb0 msgr2=0x7f029c103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.090+0000 7f02a2600700 1 --2- 172.21.15.45:0/1607095927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f029c103bb0 0x7f029c103fd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f028c009910 tx=0x7f028c009c20 comp rx=0 tx=0).stop 2024-04-25T18:47:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.090+0000 7f02a2600700 1 -- 172.21.15.45:0/1607095927 shutdown_connections 2024-04-25T18:47:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.090+0000 7f02a2600700 1 --2- 172.21.15.45:0/1607095927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f029c103bb0 0x7f029c103fd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.090+0000 7f02a2600700 1 -- 172.21.15.45:0/1607095927 >> 172.21.15.45:0/1607095927 conn(0x7f029c0fda90 msgr2=0x7f029c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.090+0000 7f02a2600700 1 -- 172.21.15.45:0/1607095927 shutdown_connections 2024-04-25T18:47:31.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.090+0000 7f02a2600700 1 -- 172.21.15.45:0/1607095927 wait complete. 2024-04-25T18:47:31.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.091+0000 7f02a2600700 1 Processor -- start 2024-04-25T18:47:31.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.092+0000 7f02a2600700 1 -- start start 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.092+0000 7f02a2600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f029c103bb0 0x7f029c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.092+0000 7f02a2600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f029c108520 con 0x7f029c103bb0 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.092+0000 7f029b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f029c103bb0 0x7f029c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.092+0000 7f029b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f029c103bb0 0x7f029c076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32950/0 (socket says 172.21.15.45:32950) 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.092+0000 7f029b400700 1 -- 172.21.15.45:0/398419786 learned_addr learned my addr 172.21.15.45:0/398419786 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.093+0000 7f029b400700 1 -- 172.21.15.45:0/398419786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f029c076c40 con 0x7f029c103bb0 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.093+0000 7f029b400700 1 --2- 172.21.15.45:0/398419786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f029c103bb0 0x7f029c076700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f028c005950 tx=0x7f028c004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.093+0000 7f0298c00700 1 -- 172.21.15.45:0/398419786 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028c026070 con 0x7f029c103bb0 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.093+0000 7f02a2600700 1 -- 172.21.15.45:0/398419786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029c074e30 con 0x7f029c103bb0 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.093+0000 7f0298c00700 1 -- 172.21.15.45:0/398419786 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028c00bc00 con 0x7f029c103bb0 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.093+0000 7f0298c00700 1 -- 172.21.15.45:0/398419786 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f028c003a10 con 0x7f029c103bb0 2024-04-25T18:47:31.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.094+0000 7f02a2600700 1 -- 172.21.15.45:0/398419786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f029c075370 con 0x7f029c103bb0 2024-04-25T18:47:31.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.095+0000 7f0298c00700 1 -- 172.21.15.45:0/398419786 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f028c029070 con 0x7f029c103bb0 2024-04-25T18:47:31.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.095+0000 7f02a2600700 1 -- 172.21.15.45:0/398419786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f029c108ca0 con 0x7f029c103bb0 2024-04-25T18:47:31.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.095+0000 7f0298c00700 1 --2- 172.21.15.45:0/398419786 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f027c0396b0 0x7f027c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.095+0000 7f0298c00700 1 -- 172.21.15.45:0/398419786 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f028c01e070 con 0x7f029c103bb0 2024-04-25T18:47:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.095+0000 7f0292a00700 1 --2- 172.21.15.45:0/398419786 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f027c0396b0 0x7f027c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:31.097 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.096+0000 7f0292a00700 1 --2- 172.21.15.45:0/398419786 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f027c0396b0 0x7f027c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0284006fd0 tx=0x7f0284006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:31.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.102+0000 7f0298c00700 1 -- 172.21.15.45:0/398419786 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f028c013070 con 0x7f029c103bb0 2024-04-25T18:47:31.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.335+0000 7f02a2600700 1 -- 172.21.15.45:0/398419786 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f029c190e90 con 0x7f029c103bb0 2024-04-25T18:47:31.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.339+0000 7f0298c00700 1 -- 172.21.15.45:0/398419786 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f028c0300c0 con 0x7f029c103bb0 2024-04-25T18:47:31.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.342+0000 7f02a2600700 1 -- 172.21.15.45:0/398419786 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f027c0396b0 msgr2=0x7f027c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:31.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.342+0000 7f02a2600700 1 --2- 172.21.15.45:0/398419786 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f027c0396b0 0x7f027c03bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0284006fd0 tx=0x7f0284006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:31.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.342+0000 7f02a2600700 1 -- 172.21.15.45:0/398419786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f029c103bb0 msgr2=0x7f029c076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:31.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.342+0000 7f02a2600700 1 --2- 172.21.15.45:0/398419786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f029c103bb0 0x7f029c076700 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f028c005950 tx=0x7f028c004fa0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.342+0000 7f02a2600700 1 -- 172.21.15.45:0/398419786 shutdown_connections 2024-04-25T18:47:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.342+0000 7f02a2600700 1 --2- 172.21.15.45:0/398419786 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f027c0396b0 0x7f027c03bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.342+0000 7f02a2600700 1 --2- 172.21.15.45:0/398419786 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f029c103bb0 0x7f029c076700 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.342+0000 7f02a2600700 1 -- 172.21.15.45:0/398419786 >> 172.21.15.45:0/398419786 conn(0x7f029c0fda90 msgr2=0x7f029c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.343+0000 7f02a2600700 1 -- 172.21.15.45:0/398419786 shutdown_connections 2024-04-25T18:47:31.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:31.343+0000 7f02a2600700 1 -- 172.21.15.45:0/398419786 wait complete. 2024-04-25T18:47:31.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:31.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:31.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:31.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:31.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:31.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:31.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:31 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/398419786' entity='client.admin' 2024-04-25T18:47:31.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:31.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:32.079 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T18:47:32.079 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T18:47:32.518 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.431+0000 7efdbf400700 1 -- 172.21.15.45:0/4256366576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb8108c50 msgr2=0x7efdb8109070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.431+0000 7efdbf400700 1 --2- 172.21.15.45:0/4256366576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb8108c50 0x7efdb8109070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efdac009910 tx=0x7efdac009c20 comp rx=0 tx=0).stop 2024-04-25T18:47:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.431+0000 7efdbf400700 1 -- 172.21.15.45:0/4256366576 shutdown_connections 2024-04-25T18:47:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.431+0000 7efdbf400700 1 --2- 172.21.15.45:0/4256366576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb8108c50 0x7efdb8109070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.431+0000 7efdbf400700 1 -- 172.21.15.45:0/4256366576 >> 172.21.15.45:0/4256366576 conn(0x7efdb8077b30 msgr2=0x7efdb8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.431+0000 7efdbf400700 1 -- 172.21.15.45:0/4256366576 shutdown_connections 2024-04-25T18:47:34.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.431+0000 7efdbf400700 1 -- 172.21.15.45:0/4256366576 wait complete. 2024-04-25T18:47:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.433+0000 7efdbf400700 1 Processor -- start 2024-04-25T18:47:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.433+0000 7efdbf400700 1 -- start start 2024-04-25T18:47:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.434+0000 7efdbf400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb8108c50 0x7efdb819e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:34.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.434+0000 7efdbf400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efdb810ea20 con 0x7efdb8108c50 2024-04-25T18:47:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.434+0000 7efdbca00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb8108c50 0x7efdb819e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.434+0000 7efdbca00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb8108c50 0x7efdb819e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32972/0 (socket says 172.21.15.45:32972) 2024-04-25T18:47:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.434+0000 7efdbca00700 1 -- 172.21.15.45:0/34851716 learned_addr learned my addr 172.21.15.45:0/34851716 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:34.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.435+0000 7efdbca00700 1 -- 172.21.15.45:0/34851716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb819e650 con 0x7efdb8108c50 2024-04-25T18:47:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.435+0000 7efdbca00700 1 --2- 172.21.15.45:0/34851716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb8108c50 0x7efdb819e110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efdac005950 tx=0x7efdac004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.435+0000 7efdb5600700 1 -- 172.21.15.45:0/34851716 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdac026070 con 0x7efdb8108c50 2024-04-25T18:47:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.435+0000 7efdb5600700 1 -- 172.21.15.45:0/34851716 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdac00bc00 con 0x7efdb8108c50 2024-04-25T18:47:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.435+0000 7efdb5600700 1 -- 172.21.15.45:0/34851716 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efdac003a10 con 0x7efdb8108c50 2024-04-25T18:47:34.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.435+0000 7efdbf400700 1 -- 172.21.15.45:0/34851716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb819e8e0 con 0x7efdb8108c50 2024-04-25T18:47:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.435+0000 7efdbf400700 1 -- 172.21.15.45:0/34851716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb81a15d0 con 0x7efdb8108c50 2024-04-25T18:47:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.436+0000 7efdb5600700 1 -- 172.21.15.45:0/34851716 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efdac029070 con 0x7efdb8108c50 2024-04-25T18:47:34.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.437+0000 7efdbf400700 1 -- 172.21.15.45:0/34851716 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb810f1a0 con 0x7efdb8108c50 2024-04-25T18:47:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.437+0000 7efdb5600700 1 --2- 172.21.15.45:0/34851716 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efda00396b0 0x7efda003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.437+0000 7efdb5600700 1 -- 172.21.15.45:0/34851716 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efdac01e070 con 0x7efdb8108c50 2024-04-25T18:47:34.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.437+0000 7efdb7e00700 1 --2- 172.21.15.45:0/34851716 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efda00396b0 0x7efda003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:34.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.438+0000 7efdb7e00700 1 --2- 172.21.15.45:0/34851716 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efda00396b0 0x7efda003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efda4006fd0 tx=0x7efda4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:34.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.443+0000 7efdb5600700 1 -- 172.21.15.45:0/34851716 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efdac013070 con 0x7efdb8108c50 2024-04-25T18:47:34.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.676+0000 7efdbf400700 1 -- 172.21.15.45:0/34851716 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7efdb802e090 con 0x7efda00396b0 2024-04-25T18:47:34.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.685+0000 7efdb5600700 1 -- 172.21.15.45:0/34851716 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efdb802e090 con 0x7efda00396b0 2024-04-25T18:47:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.687+0000 7efdbf400700 1 -- 172.21.15.45:0/34851716 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efda00396b0 msgr2=0x7efda003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.687+0000 7efdbf400700 1 --2- 172.21.15.45:0/34851716 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efda00396b0 0x7efda003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efda4006fd0 tx=0x7efda4006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.687+0000 7efdbf400700 1 -- 172.21.15.45:0/34851716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb8108c50 msgr2=0x7efdb819e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.688+0000 7efdbf400700 1 --2- 172.21.15.45:0/34851716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb8108c50 0x7efdb819e110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efdac005950 tx=0x7efdac004fa0 comp rx=0 tx=0).stop 2024-04-25T18:47:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.688+0000 7efdbf400700 1 -- 172.21.15.45:0/34851716 shutdown_connections 2024-04-25T18:47:34.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.688+0000 7efdbf400700 1 --2- 172.21.15.45:0/34851716 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efda00396b0 0x7efda003bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.688+0000 7efdbf400700 1 --2- 172.21.15.45:0/34851716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efdb8108c50 0x7efdb819e110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.688+0000 7efdbf400700 1 -- 172.21.15.45:0/34851716 >> 172.21.15.45:0/34851716 conn(0x7efdb8077b30 msgr2=0x7efdb8078640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.688+0000 7efdbf400700 1 -- 172.21.15.45:0/34851716 shutdown_connections 2024-04-25T18:47:34.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:34.688+0000 7efdbf400700 1 -- 172.21.15.45:0/34851716 wait complete. 2024-04-25T18:47:35.442 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi062 2024-04-25T18:47:35.442 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:47:35.443 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/etc/ceph/ceph.conf 2024-04-25T18:47:35.462 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:47:35.462 DEBUG:teuthology.orchestra.run.smithi062:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:35.520 INFO:tasks.cephadm:Adding host smithi062 to orchestrator... 2024-04-25T18:47:35.520 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch host add smithi062 2024-04-25T18:47:35.939 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:35.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:35 smithi045 ceph-mon[27776]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:35.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:35.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:35.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:47:35.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:47:35.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:35 smithi045 ceph-mon[27776]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T18:47:36.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:36 smithi045 ceph-mon[27776]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:36.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:36 smithi045 ceph-mon[27776]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:37.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:37 smithi045 ceph-mon[27776]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:37.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:37 smithi045 ceph-mon[27776]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:37.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:37.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:37.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.965+0000 7f2441c00700 1 -- 172.21.15.45:0/2589841977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c1084e0 msgr2=0x7f243c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.965+0000 7f2441c00700 1 --2- 172.21.15.45:0/2589841977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c1084e0 0x7f243c1088c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f242c00b0a0 tx=0x7f242c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:47:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.965+0000 7f2441c00700 1 -- 172.21.15.45:0/2589841977 shutdown_connections 2024-04-25T18:47:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.965+0000 7f2441c00700 1 --2- 172.21.15.45:0/2589841977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c1084e0 0x7f243c1088c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.965+0000 7f2441c00700 1 -- 172.21.15.45:0/2589841977 >> 172.21.15.45:0/2589841977 conn(0x7f243c076ea0 msgr2=0x7f243c0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.965+0000 7f2441c00700 1 -- 172.21.15.45:0/2589841977 shutdown_connections 2024-04-25T18:47:37.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.965+0000 7f2441c00700 1 -- 172.21.15.45:0/2589841977 wait complete. 2024-04-25T18:47:37.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.966+0000 7f2441c00700 1 Processor -- start 2024-04-25T18:47:37.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.967+0000 7f2441c00700 1 -- start start 2024-04-25T18:47:37.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.967+0000 7f2441c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c19a2b0 0x7f243c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:37.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.967+0000 7f2441c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f243c10c920 con 0x7f243c19a2b0 2024-04-25T18:47:37.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.967+0000 7f243aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c19a2b0 0x7f243c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:37.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.967+0000 7f243aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c19a2b0 0x7f243c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57002/0 (socket says 172.21.15.45:57002) 2024-04-25T18:47:37.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.967+0000 7f243aa00700 1 -- 172.21.15.45:0/3002274415 learned_addr learned my addr 172.21.15.45:0/3002274415 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:37.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.968+0000 7f243aa00700 1 -- 172.21.15.45:0/3002274415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f243c19abd0 con 0x7f243c19a2b0 2024-04-25T18:47:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.968+0000 7f243aa00700 1 --2- 172.21.15.45:0/3002274415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c19a2b0 0x7f243c19a690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f242c009040 tx=0x7f242c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.968+0000 7f2433e00700 1 -- 172.21.15.45:0/3002274415 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f242c021070 con 0x7f243c19a2b0 2024-04-25T18:47:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.968+0000 7f2433e00700 1 -- 172.21.15.45:0/3002274415 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f242c01c460 con 0x7f243c19a2b0 2024-04-25T18:47:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.968+0000 7f2433e00700 1 -- 172.21.15.45:0/3002274415 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f242c017040 con 0x7f243c19a2b0 2024-04-25T18:47:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.968+0000 7f2441c00700 1 -- 172.21.15.45:0/3002274415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f243c19ae60 con 0x7f243c19a2b0 2024-04-25T18:47:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.969+0000 7f2441c00700 1 -- 172.21.15.45:0/3002274415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f243c19ed50 con 0x7f243c19a2b0 2024-04-25T18:47:37.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.970+0000 7f2433e00700 1 -- 172.21.15.45:0/3002274415 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f242c024070 con 0x7f243c19a2b0 2024-04-25T18:47:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.970+0000 7f2441c00700 1 -- 172.21.15.45:0/3002274415 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f243c103b90 con 0x7f243c19a2b0 2024-04-25T18:47:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.970+0000 7f2433e00700 1 --2- 172.21.15.45:0/3002274415 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2420039700 0x7f242003bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.970+0000 7f2433e00700 1 -- 172.21.15.45:0/3002274415 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f242c00f050 con 0x7f243c19a2b0 2024-04-25T18:47:37.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.970+0000 7f2432000700 1 --2- 172.21.15.45:0/3002274415 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2420039700 0x7f242003bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:37.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.971+0000 7f2432000700 1 --2- 172.21.15.45:0/3002274415 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2420039700 0x7f242003bbc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2424006fd0 tx=0x7f2424006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:37.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:37.976+0000 7f2433e00700 1 -- 172.21.15.45:0/3002274415 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f242c0232c0 con 0x7f243c19a2b0 2024-04-25T18:47:38.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:38.213+0000 7f2441c00700 1 -- 172.21.15.45:0/3002274415 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}) -- 0x7f243c19efd0 con 0x7f2420039700 2024-04-25T18:47:39.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:39 smithi045 ceph-mon[27776]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:39.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:39 smithi045 ceph-mon[27776]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi062", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:39.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:39 smithi045 ceph-mon[27776]: Deploying cephadm binary to smithi062 2024-04-25T18:47:41.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:41 smithi045 ceph-mon[27776]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:43.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.183+0000 7f2433e00700 1 -- 172.21.15.45:0/3002274415 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f243c19efd0 con 0x7f2420039700 2024-04-25T18:47:43.186 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi062' with addr '172.21.15.62' 2024-04-25T18:47:43.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.187+0000 7f2441c00700 1 -- 172.21.15.45:0/3002274415 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2420039700 msgr2=0x7f242003bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:43.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.187+0000 7f2441c00700 1 --2- 172.21.15.45:0/3002274415 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2420039700 0x7f242003bbc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2424006fd0 tx=0x7f2424006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:43.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.187+0000 7f2441c00700 1 -- 172.21.15.45:0/3002274415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c19a2b0 msgr2=0x7f243c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:43.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.187+0000 7f2441c00700 1 --2- 172.21.15.45:0/3002274415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c19a2b0 0x7f243c19a690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f242c009040 tx=0x7f242c009750 comp rx=0 tx=0).stop 2024-04-25T18:47:43.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.187+0000 7f2441c00700 1 -- 172.21.15.45:0/3002274415 shutdown_connections 2024-04-25T18:47:43.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.187+0000 7f2441c00700 1 --2- 172.21.15.45:0/3002274415 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2420039700 0x7f242003bbc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:43.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.187+0000 7f2441c00700 1 --2- 172.21.15.45:0/3002274415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f243c19a2b0 0x7f243c19a690 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:43.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.187+0000 7f2441c00700 1 -- 172.21.15.45:0/3002274415 >> 172.21.15.45:0/3002274415 conn(0x7f243c076ea0 msgr2=0x7f243c104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:43.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.187+0000 7f2441c00700 1 -- 172.21.15.45:0/3002274415 shutdown_connections 2024-04-25T18:47:43.190 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:43.187+0000 7f2441c00700 1 -- 172.21.15.45:0/3002274415 wait complete. 2024-04-25T18:47:43.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:43 smithi045 ceph-mon[27776]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:43.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:43.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:44.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T18:47:44.530 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:44.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:44 smithi045 ceph-mon[27776]: Added host smithi062 2024-04-25T18:47:44.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:44.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:45.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:45 smithi045 ceph-mon[27776]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.430+0000 7f8afbc00700 1 -- 172.21.15.45:0/3146637465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af4074e10 msgr2=0x7f8af40751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:46.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.430+0000 7f8afbc00700 1 --2- 172.21.15.45:0/3146637465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af4074e10 0x7f8af40751f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8009910 tx=0x7f8ae8009c20 comp rx=0 tx=0).stop 2024-04-25T18:47:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.430+0000 7f8afbc00700 1 -- 172.21.15.45:0/3146637465 shutdown_connections 2024-04-25T18:47:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.430+0000 7f8afbc00700 1 --2- 172.21.15.45:0/3146637465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af4074e10 0x7f8af40751f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.430+0000 7f8afbc00700 1 -- 172.21.15.45:0/3146637465 >> 172.21.15.45:0/3146637465 conn(0x7f8af40fdf80 msgr2=0x7f8af41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.430+0000 7f8afbc00700 1 -- 172.21.15.45:0/3146637465 shutdown_connections 2024-04-25T18:47:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.430+0000 7f8afbc00700 1 -- 172.21.15.45:0/3146637465 wait complete. 2024-04-25T18:47:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.432+0000 7f8afbc00700 1 Processor -- start 2024-04-25T18:47:46.433 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.432+0000 7f8afbc00700 1 -- start start 2024-04-25T18:47:46.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.432+0000 7f8afbc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af419a370 0x7f8af419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:46.434 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.432+0000 7f8afbc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8af410eba0 con 0x7f8af419a370 2024-04-25T18:47:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.432+0000 7f8af9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af419a370 0x7f8af419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.433+0000 7f8af9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af419a370 0x7f8af419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40322/0 (socket says 172.21.15.45:40322) 2024-04-25T18:47:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.433+0000 7f8af9200700 1 -- 172.21.15.45:0/655995328 learned_addr learned my addr 172.21.15.45:0/655995328 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.433+0000 7f8af9200700 1 -- 172.21.15.45:0/655995328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af419ac90 con 0x7f8af419a370 2024-04-25T18:47:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.433+0000 7f8af9200700 1 --2- 172.21.15.45:0/655995328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af419a370 0x7f8af419a750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005950 tx=0x7f8ae8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:46.435 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.433+0000 7f8af1600700 1 -- 172.21.15.45:0/655995328 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae8026070 con 0x7f8af419a370 2024-04-25T18:47:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.434+0000 7f8af1600700 1 -- 172.21.15.45:0/655995328 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae800b750 con 0x7f8af419a370 2024-04-25T18:47:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.434+0000 7f8afbc00700 1 -- 172.21.15.45:0/655995328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af419af20 con 0x7f8af419a370 2024-04-25T18:47:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.434+0000 7f8af1600700 1 -- 172.21.15.45:0/655995328 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae8003810 con 0x7f8af419a370 2024-04-25T18:47:46.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.434+0000 7f8afbc00700 1 -- 172.21.15.45:0/655995328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af419edc0 con 0x7f8af419a370 2024-04-25T18:47:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.435+0000 7f8af1600700 1 -- 172.21.15.45:0/655995328 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8ae8029070 con 0x7f8af419a370 2024-04-25T18:47:46.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.435+0000 7f8afbc00700 1 -- 172.21.15.45:0/655995328 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af40764c0 con 0x7f8af419a370 2024-04-25T18:47:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.435+0000 7f8af1600700 1 --2- 172.21.15.45:0/655995328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8adc0396b0 0x7f8adc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.435+0000 7f8af1600700 1 -- 172.21.15.45:0/655995328 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8ae801e070 con 0x7f8af419a370 2024-04-25T18:47:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.436+0000 7f8af3e00700 1 --2- 172.21.15.45:0/655995328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8adc0396b0 0x7f8adc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:46.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.436+0000 7f8af3e00700 1 --2- 172.21.15.45:0/655995328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8adc0396b0 0x7f8adc03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0006fd0 tx=0x7f8ae0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:46.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.441+0000 7f8af1600700 1 -- 172.21.15.45:0/655995328 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ae8013070 con 0x7f8af419a370 2024-04-25T18:47:46.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.680+0000 7f8afbc00700 1 -- 172.21.15.45:0/655995328 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8af419b720 con 0x7f8adc0396b0 2024-04-25T18:47:46.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.683+0000 7f8af1600700 1 -- 172.21.15.45:0/655995328 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f8af419b720 con 0x7f8adc0396b0 2024-04-25T18:47:46.685 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:46.685 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}] 2024-04-25T18:47:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.686+0000 7f8afbc00700 1 -- 172.21.15.45:0/655995328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8adc0396b0 msgr2=0x7f8adc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.686+0000 7f8afbc00700 1 --2- 172.21.15.45:0/655995328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8adc0396b0 0x7f8adc03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0006fd0 tx=0x7f8ae0006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.686+0000 7f8afbc00700 1 -- 172.21.15.45:0/655995328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af419a370 msgr2=0x7f8af419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:46.688 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.686+0000 7f8afbc00700 1 --2- 172.21.15.45:0/655995328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af419a370 0x7f8af419a750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8005950 tx=0x7f8ae8005780 comp rx=0 tx=0).stop 2024-04-25T18:47:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.687+0000 7f8afbc00700 1 -- 172.21.15.45:0/655995328 shutdown_connections 2024-04-25T18:47:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.687+0000 7f8afbc00700 1 --2- 172.21.15.45:0/655995328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8adc0396b0 0x7f8adc03bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.687+0000 7f8afbc00700 1 --2- 172.21.15.45:0/655995328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af419a370 0x7f8af419a750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.687+0000 7f8afbc00700 1 -- 172.21.15.45:0/655995328 >> 172.21.15.45:0/655995328 conn(0x7f8af40fdf80 msgr2=0x7f8af40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.687+0000 7f8afbc00700 1 -- 172.21.15.45:0/655995328 shutdown_connections 2024-04-25T18:47:46.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:46.687+0000 7f8afbc00700 1 -- 172.21.15.45:0/655995328 wait complete. 2024-04-25T18:47:47.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:47.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:47 smithi045 ceph-mon[27776]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:47.300 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:47 smithi045 ceph-mon[27776]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:47:47.380 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi079 2024-04-25T18:47:47.380 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:47:47.380 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/etc/ceph/ceph.conf 2024-04-25T18:47:47.398 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:47:47.398 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:47.456 INFO:tasks.cephadm:Adding host smithi079 to orchestrator... 2024-04-25T18:47:47.456 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch host add smithi079 2024-04-25T18:47:47.896 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:49.382 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:49 smithi045 ceph-mon[27776]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.887+0000 7f6d63200700 1 -- 172.21.15.45:0/293359004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d641031b0 msgr2=0x7f6d641007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.887+0000 7f6d63200700 1 --2- 172.21.15.45:0/293359004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d641031b0 0x7f6d641007a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009910 tx=0x7f6d54009c20 comp rx=0 tx=0).stop 2024-04-25T18:47:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.888+0000 7f6d63200700 1 -- 172.21.15.45:0/293359004 shutdown_connections 2024-04-25T18:47:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.888+0000 7f6d63200700 1 --2- 172.21.15.45:0/293359004 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d641031b0 0x7f6d641007a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.888+0000 7f6d63200700 1 -- 172.21.15.45:0/293359004 >> 172.21.15.45:0/293359004 conn(0x7f6d640fc400 msgr2=0x7f6d640fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:49.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.888+0000 7f6d63200700 1 -- 172.21.15.45:0/293359004 shutdown_connections 2024-04-25T18:47:49.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.888+0000 7f6d63200700 1 -- 172.21.15.45:0/293359004 wait complete. 2024-04-25T18:47:49.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.889+0000 7f6d63200700 1 Processor -- start 2024-04-25T18:47:49.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.889+0000 7f6d63200700 1 -- start start 2024-04-25T18:47:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.890+0000 7f6d63200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d641031b0 0x7f6d64199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.890+0000 7f6d63200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6410e9d0 con 0x7f6d641031b0 2024-04-25T18:47:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.890+0000 7f6d61e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d641031b0 0x7f6d64199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:49.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.890+0000 7f6d61e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d641031b0 0x7f6d64199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40346/0 (socket says 172.21.15.45:40346) 2024-04-25T18:47:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.890+0000 7f6d61e00700 1 -- 172.21.15.45:0/2630717727 learned_addr learned my addr 172.21.15.45:0/2630717727 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.891+0000 7f6d61e00700 1 -- 172.21.15.45:0/2630717727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6419a480 con 0x7f6d641031b0 2024-04-25T18:47:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.891+0000 7f6d61e00700 1 --2- 172.21.15.45:0/2630717727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d641031b0 0x7f6d64199f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006010 tx=0x7f6d54005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.891+0000 7f6d5aa00700 1 -- 172.21.15.45:0/2630717727 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d54026070 con 0x7f6d641031b0 2024-04-25T18:47:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.891+0000 7f6d5aa00700 1 -- 172.21.15.45:0/2630717727 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d5400bc00 con 0x7f6d641031b0 2024-04-25T18:47:49.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.891+0000 7f6d63200700 1 -- 172.21.15.45:0/2630717727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6419a710 con 0x7f6d641031b0 2024-04-25T18:47:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.891+0000 7f6d5aa00700 1 -- 172.21.15.45:0/2630717727 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d54003910 con 0x7f6d641031b0 2024-04-25T18:47:49.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.892+0000 7f6d63200700 1 -- 172.21.15.45:0/2630717727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6419ab70 con 0x7f6d641031b0 2024-04-25T18:47:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.893+0000 7f6d63200700 1 -- 172.21.15.45:0/2630717727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d641019c0 con 0x7f6d641031b0 2024-04-25T18:47:49.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.893+0000 7f6d5aa00700 1 -- 172.21.15.45:0/2630717727 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6d54029070 con 0x7f6d641031b0 2024-04-25T18:47:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.894+0000 7f6d5aa00700 1 --2- 172.21.15.45:0/2630717727 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6d48039650 0x7f6d4803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.894+0000 7f6d5aa00700 1 -- 172.21.15.45:0/2630717727 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6d5401e070 con 0x7f6d641031b0 2024-04-25T18:47:49.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.894+0000 7f6d61400700 1 --2- 172.21.15.45:0/2630717727 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6d48039650 0x7f6d4803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:49.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.895+0000 7f6d61400700 1 --2- 172.21.15.45:0/2630717727 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6d48039650 0x7f6d4803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6d50006fd0 tx=0x7f6d50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:49.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:49.901+0000 7f6d5aa00700 1 -- 172.21.15.45:0/2630717727 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d54013070 con 0x7f6d641031b0 2024-04-25T18:47:50.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:50.137+0000 7f6d63200700 1 -- 172.21.15.45:0/2630717727 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}) -- 0x7f6d6419b310 con 0x7f6d48039650 2024-04-25T18:47:50.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:50 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:50.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:50 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:50.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:50 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:50.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:50 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:50.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:50 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:47:50.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:50 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:47:50.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:50 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:47:50.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:50 smithi045 ceph-mon[27776]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-25T18:47:50.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:50 smithi045 ceph-mon[27776]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:47:52.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:51 smithi045 ceph-mon[27776]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:52.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:51 smithi045 ceph-mon[27776]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:47:52.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:51 smithi045 ceph-mon[27776]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:47:52.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:51 smithi045 ceph-mon[27776]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:47:52.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:51 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:52.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:51 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:52.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:51 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:52.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:51 smithi045 ceph-mon[27776]: Deploying cephadm binary to smithi079 2024-04-25T18:47:54.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:53 smithi045 ceph-mon[27776]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:55.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.107+0000 7f6d5aa00700 1 -- 172.21.15.45:0/2630717727 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6d6419b310 con 0x7f6d48039650 2024-04-25T18:47:55.110 INFO:teuthology.orchestra.run.smithi045.stdout:Added host 'smithi079' with addr '172.21.15.79' 2024-04-25T18:47:55.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.111+0000 7f6d63200700 1 -- 172.21.15.45:0/2630717727 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6d48039650 msgr2=0x7f6d4803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.111+0000 7f6d63200700 1 --2- 172.21.15.45:0/2630717727 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6d48039650 0x7f6d4803bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6d50006fd0 tx=0x7f6d50006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.111+0000 7f6d63200700 1 -- 172.21.15.45:0/2630717727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d641031b0 msgr2=0x7f6d64199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.111+0000 7f6d63200700 1 --2- 172.21.15.45:0/2630717727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d641031b0 0x7f6d64199f40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006010 tx=0x7f6d54005780 comp rx=0 tx=0).stop 2024-04-25T18:47:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.111+0000 7f6d63200700 1 -- 172.21.15.45:0/2630717727 shutdown_connections 2024-04-25T18:47:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.111+0000 7f6d63200700 1 --2- 172.21.15.45:0/2630717727 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6d48039650 0x7f6d4803bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.111+0000 7f6d63200700 1 --2- 172.21.15.45:0/2630717727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d641031b0 0x7f6d64199f40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.111+0000 7f6d63200700 1 -- 172.21.15.45:0/2630717727 >> 172.21.15.45:0/2630717727 conn(0x7f6d640fc400 msgr2=0x7f6d640fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.111+0000 7f6d63200700 1 -- 172.21.15.45:0/2630717727 shutdown_connections 2024-04-25T18:47:55.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:55.111+0000 7f6d63200700 1 -- 172.21.15.45:0/2630717727 wait complete. 2024-04-25T18:47:55.813 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T18:47:56.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:55 smithi045 ceph-mon[27776]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:56.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:55 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:56.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:55 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:47:56.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:55 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:56.269 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:57.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:56 smithi045 ceph-mon[27776]: Added host smithi079 2024-04-25T18:47:57.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:57.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:57 smithi045 ceph-mon[27776]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:57.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:47:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.192+0000 7f9ab3e00700 1 -- 172.21.15.45:0/1878675851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab4074e10 msgr2=0x7f9ab40751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.192+0000 7f9ab3e00700 1 --2- 172.21.15.45:0/1878675851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab4074e10 0x7f9ab40751f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4009910 tx=0x7f9aa4009c20 comp rx=0 tx=0).stop 2024-04-25T18:47:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.192+0000 7f9ab3e00700 1 -- 172.21.15.45:0/1878675851 shutdown_connections 2024-04-25T18:47:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.192+0000 7f9ab3e00700 1 --2- 172.21.15.45:0/1878675851 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab4074e10 0x7f9ab40751f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.192+0000 7f9ab3e00700 1 -- 172.21.15.45:0/1878675851 >> 172.21.15.45:0/1878675851 conn(0x7f9ab40fdda0 msgr2=0x7f9ab41001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:58.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.192+0000 7f9ab3e00700 1 -- 172.21.15.45:0/1878675851 shutdown_connections 2024-04-25T18:47:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.192+0000 7f9ab3e00700 1 -- 172.21.15.45:0/1878675851 wait complete. 2024-04-25T18:47:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.193+0000 7f9ab3e00700 1 Processor -- start 2024-04-25T18:47:58.195 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.193+0000 7f9ab3e00700 1 -- start start 2024-04-25T18:47:58.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.194+0000 7f9ab3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab4074e10 0x7f9ab4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:58.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.194+0000 7f9ab3e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab410e9e0 con 0x7f9ab4074e10 2024-04-25T18:47:58.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.194+0000 7f9ab2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab4074e10 0x7f9ab4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:58.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.194+0000 7f9ab2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab4074e10 0x7f9ab4199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38778/0 (socket says 172.21.15.45:38778) 2024-04-25T18:47:58.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.194+0000 7f9ab2a00700 1 -- 172.21.15.45:0/822968774 learned_addr learned my addr 172.21.15.45:0/822968774 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:47:58.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.194+0000 7f9ab2a00700 1 -- 172.21.15.45:0/822968774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab419a490 con 0x7f9ab4074e10 2024-04-25T18:47:58.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.195+0000 7f9ab2a00700 1 --2- 172.21.15.45:0/822968774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab4074e10 0x7f9ab4199f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4006010 tx=0x7f9aa4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:58.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.195+0000 7f9ab8a00700 1 -- 172.21.15.45:0/822968774 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aa4026070 con 0x7f9ab4074e10 2024-04-25T18:47:58.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.195+0000 7f9ab8a00700 1 -- 172.21.15.45:0/822968774 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa400bc00 con 0x7f9ab4074e10 2024-04-25T18:47:58.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.195+0000 7f9ab8a00700 1 -- 172.21.15.45:0/822968774 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9aa4003910 con 0x7f9ab4074e10 2024-04-25T18:47:58.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.195+0000 7f9ab3e00700 1 -- 172.21.15.45:0/822968774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab419a720 con 0x7f9ab4074e10 2024-04-25T18:47:58.197 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.195+0000 7f9ab3e00700 1 -- 172.21.15.45:0/822968774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab419ab80 con 0x7f9ab4074e10 2024-04-25T18:47:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.196+0000 7f9ab8a00700 1 -- 172.21.15.45:0/822968774 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9aa4029070 con 0x7f9ab4074e10 2024-04-25T18:47:58.198 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.196+0000 7f9ab3e00700 1 -- 172.21.15.45:0/822968774 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab40764c0 con 0x7f9ab4074e10 2024-04-25T18:47:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.197+0000 7f9ab8a00700 1 --2- 172.21.15.45:0/822968774 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9a940396b0 0x7f9a9403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:47:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.197+0000 7f9ab8a00700 1 -- 172.21.15.45:0/822968774 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9aa401e070 con 0x7f9ab4074e10 2024-04-25T18:47:58.199 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.197+0000 7f9aaa000700 1 --2- 172.21.15.45:0/822968774 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9a940396b0 0x7f9a9403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:47:58.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.198+0000 7f9aaa000700 1 --2- 172.21.15.45:0/822968774 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9a940396b0 0x7f9a9403bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c006fd0 tx=0x7f9a9c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:47:58.205 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.203+0000 7f9ab8a00700 1 -- 172.21.15.45:0/822968774 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9aa4013070 con 0x7f9ab4074e10 2024-04-25T18:47:58.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.441+0000 7f9ab3e00700 1 -- 172.21.15.45:0/822968774 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ab419b2b0 con 0x7f9a940396b0 2024-04-25T18:47:58.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.443+0000 7f9ab8a00700 1 -- 172.21.15.45:0/822968774 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f9ab419b2b0 con 0x7f9a940396b0 2024-04-25T18:47:58.445 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:47:58.445 INFO:teuthology.orchestra.run.smithi045.stdout:[{"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.62", "hostname": "smithi062", "labels": [], "status": ""}, {"addr": "172.21.15.79", "hostname": "smithi079", "labels": [], "status": ""}] 2024-04-25T18:47:58.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.446+0000 7f9ab3e00700 1 -- 172.21.15.45:0/822968774 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9a940396b0 msgr2=0x7f9a9403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:58.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.446+0000 7f9ab3e00700 1 --2- 172.21.15.45:0/822968774 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9a940396b0 0x7f9a9403bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c006fd0 tx=0x7f9a9c006e40 comp rx=0 tx=0).stop 2024-04-25T18:47:58.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.446+0000 7f9ab3e00700 1 -- 172.21.15.45:0/822968774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab4074e10 msgr2=0x7f9ab4199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:47:58.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.446+0000 7f9ab3e00700 1 --2- 172.21.15.45:0/822968774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab4074e10 0x7f9ab4199f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4006010 tx=0x7f9aa4005780 comp rx=0 tx=0).stop 2024-04-25T18:47:58.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.446+0000 7f9ab3e00700 1 -- 172.21.15.45:0/822968774 shutdown_connections 2024-04-25T18:47:58.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.446+0000 7f9ab3e00700 1 --2- 172.21.15.45:0/822968774 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9a940396b0 0x7f9a9403bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:58.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.446+0000 7f9ab3e00700 1 --2- 172.21.15.45:0/822968774 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9ab4074e10 0x7f9ab4199f50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:47:58.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.446+0000 7f9ab3e00700 1 -- 172.21.15.45:0/822968774 >> 172.21.15.45:0/822968774 conn(0x7f9ab40fdda0 msgr2=0x7f9ab41085a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:47:58.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.446+0000 7f9ab3e00700 1 -- 172.21.15.45:0/822968774 shutdown_connections 2024-04-25T18:47:58.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:47:58.447+0000 7f9ab3e00700 1 -- 172.21.15.45:0/822968774 wait complete. 2024-04-25T18:47:59.112 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T18:47:59.113 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T18:47:59.571 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:47:59.985 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:59 smithi045 ceph-mon[27776]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:47:59.985 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:47:59 smithi045 ceph-mon[27776]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:48:01.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.573+0000 7f305d200700 1 -- 172.21.15.45:0/2270549416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581084e0 msgr2=0x7f30581088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:01.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.573+0000 7f305d200700 1 --2- 172.21.15.45:0/2270549416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581084e0 0x7f30581088c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3048009910 tx=0x7f3048009c20 comp rx=0 tx=0).stop 2024-04-25T18:48:01.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.573+0000 7f305d200700 1 -- 172.21.15.45:0/2270549416 shutdown_connections 2024-04-25T18:48:01.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.573+0000 7f305d200700 1 --2- 172.21.15.45:0/2270549416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581084e0 0x7f30581088c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:01.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.573+0000 7f305d200700 1 -- 172.21.15.45:0/2270549416 >> 172.21.15.45:0/2270549416 conn(0x7f3058076ea0 msgr2=0x7f30580772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:01.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.573+0000 7f305d200700 1 -- 172.21.15.45:0/2270549416 shutdown_connections 2024-04-25T18:48:01.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.573+0000 7f305d200700 1 -- 172.21.15.45:0/2270549416 wait complete. 2024-04-25T18:48:01.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.574+0000 7f305d200700 1 Processor -- start 2024-04-25T18:48:01.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.574+0000 7f305d200700 1 -- start start 2024-04-25T18:48:01.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.575+0000 7f305d200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581084e0 0x7f305819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:01.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.575+0000 7f305d200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f305810c920 con 0x7f30581084e0 2024-04-25T18:48:01.577 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.575+0000 7f3055e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581084e0 0x7f305819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:01.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.575+0000 7f3055e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581084e0 0x7f305819a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38788/0 (socket says 172.21.15.45:38788) 2024-04-25T18:48:01.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.575+0000 7f3055e00700 1 -- 172.21.15.45:0/1556423110 learned_addr learned my addr 172.21.15.45:0/1556423110 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:48:01.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.576+0000 7f3055e00700 1 -- 172.21.15.45:0/1556423110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305819a610 con 0x7f30581084e0 2024-04-25T18:48:01.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.576+0000 7f3055e00700 1 --2- 172.21.15.45:0/1556423110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581084e0 0x7f305819a0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f30480056d0 tx=0x7f30480057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:01.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.576+0000 7f304f400700 1 -- 172.21.15.45:0/1556423110 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3048026070 con 0x7f30581084e0 2024-04-25T18:48:01.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.576+0000 7f304f400700 1 -- 172.21.15.45:0/1556423110 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f304800bba0 con 0x7f30581084e0 2024-04-25T18:48:01.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.576+0000 7f304f400700 1 -- 172.21.15.45:0/1556423110 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3048003920 con 0x7f30581084e0 2024-04-25T18:48:01.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.576+0000 7f305d200700 1 -- 172.21.15.45:0/1556423110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305819a900 con 0x7f30581084e0 2024-04-25T18:48:01.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.576+0000 7f305d200700 1 -- 172.21.15.45:0/1556423110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305819e9a0 con 0x7f30581084e0 2024-04-25T18:48:01.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.578+0000 7f304f400700 1 -- 172.21.15.45:0/1556423110 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3048029070 con 0x7f30581084e0 2024-04-25T18:48:01.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.578+0000 7f305d200700 1 -- 172.21.15.45:0/1556423110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3058103b90 con 0x7f30581084e0 2024-04-25T18:48:01.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.578+0000 7f304f400700 1 --2- 172.21.15.45:0/1556423110 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f303c039650 0x7f303c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:01.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.578+0000 7f304f400700 1 -- 172.21.15.45:0/1556423110 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f304801e070 con 0x7f30581084e0 2024-04-25T18:48:01.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.579+0000 7f304d400700 1 --2- 172.21.15.45:0/1556423110 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f303c039650 0x7f303c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:01.581 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.579+0000 7f304d400700 1 --2- 172.21.15.45:0/1556423110 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f303c039650 0x7f303c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3040006fd0 tx=0x7f3040006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:01.588 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.587+0000 7f304f400700 1 -- 172.21.15.45:0/1556423110 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3048013070 con 0x7f30581084e0 2024-04-25T18:48:01.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:01.831+0000 7f305d200700 1 -- 172.21.15.45:0/1556423110 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f305819b430 con 0x7f30581084e0 2024-04-25T18:48:02.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.052+0000 7f304f400700 1 -- 172.21.15.45:0/1556423110 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f30480282c0 con 0x7f30581084e0 2024-04-25T18:48:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.056+0000 7f305d200700 1 -- 172.21.15.45:0/1556423110 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f303c039650 msgr2=0x7f303c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.056+0000 7f305d200700 1 --2- 172.21.15.45:0/1556423110 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f303c039650 0x7f303c03bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3040006fd0 tx=0x7f3040006e40 comp rx=0 tx=0).stop 2024-04-25T18:48:02.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.056+0000 7f305d200700 1 -- 172.21.15.45:0/1556423110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581084e0 msgr2=0x7f305819a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:02.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.056+0000 7f305d200700 1 --2- 172.21.15.45:0/1556423110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581084e0 0x7f305819a0d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f30480056d0 tx=0x7f30480057b0 comp rx=0 tx=0).stop 2024-04-25T18:48:02.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.056+0000 7f305d200700 1 -- 172.21.15.45:0/1556423110 shutdown_connections 2024-04-25T18:48:02.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.056+0000 7f305d200700 1 --2- 172.21.15.45:0/1556423110 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f303c039650 0x7f303c03bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:02.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.056+0000 7f305d200700 1 --2- 172.21.15.45:0/1556423110 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f30581084e0 0x7f305819a0d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:02.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.056+0000 7f305d200700 1 -- 172.21.15.45:0/1556423110 >> 172.21.15.45:0/1556423110 conn(0x7f3058076ea0 msgr2=0x7f30580fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:02.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.057+0000 7f305d200700 1 -- 172.21.15.45:0/1556423110 shutdown_connections 2024-04-25T18:48:02.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:02.057+0000 7f305d200700 1 -- 172.21.15.45:0/1556423110 wait complete. 2024-04-25T18:48:02.060 INFO:teuthology.orchestra.run.smithi045.stderr:adjusted tunables profile to default 2024-04-25T18:48:02.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:01 smithi045 ceph-mon[27776]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:02.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:01 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:02.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:01 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:02.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:01 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:02.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:01 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:02.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:01 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:48:02.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:01 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:02.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:01 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:02.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:01 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1556423110' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T18:48:02.772 INFO:tasks.cephadm:Adding mon.a on smithi045 2024-04-25T18:48:02.772 INFO:tasks.cephadm:Adding mon.b on smithi062 2024-04-25T18:48:02.772 INFO:tasks.cephadm:Adding mon.c on smithi079 2024-04-25T18:48:02.773 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi045:172.21.15.45=a;smithi062:172.21.15.62=b;smithi079:172.21.15.79=c' 2024-04-25T18:48:02.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:02 smithi045 ceph-mon[27776]: Updating smithi079:/etc/ceph/ceph.conf 2024-04-25T18:48:02.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:02 smithi045 ceph-mon[27776]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:02 smithi045 ceph-mon[27776]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:48:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:02 smithi045 ceph-mon[27776]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:48:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:02 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1556423110' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T18:48:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:02 smithi045 ceph-mon[27776]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:02.997 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:03 smithi045 ceph-mon[27776]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:04.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.467+0000 7f6b2d787700 1 -- 172.21.15.79:0/242009669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b28108550 msgr2=0x7f6b28108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.467+0000 7f6b2d787700 1 --2- 172.21.15.79:0/242009669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b28108550 0x7f6b28108930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000b0a0 tx=0x7f6b2000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.467+0000 7f6b2d787700 1 -- 172.21.15.79:0/242009669 shutdown_connections 2024-04-25T18:48:04.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.467+0000 7f6b2d787700 1 --2- 172.21.15.79:0/242009669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b28108550 0x7f6b28108930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.467+0000 7f6b2d787700 1 -- 172.21.15.79:0/242009669 >> 172.21.15.79:0/242009669 conn(0x7f6b28077ac0 msgr2=0x7f6b28077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:04.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.467+0000 7f6b2d787700 1 -- 172.21.15.79:0/242009669 shutdown_connections 2024-04-25T18:48:04.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.467+0000 7f6b2d787700 1 -- 172.21.15.79:0/242009669 wait complete. 2024-04-25T18:48:04.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.468+0000 7f6b2d787700 1 Processor -- start 2024-04-25T18:48:04.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.468+0000 7f6b2d787700 1 -- start start 2024-04-25T18:48:04.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.468+0000 7f6b2d787700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b28108550 0x7f6b28101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.468+0000 7f6b2d787700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6b2810c9b0 con 0x7f6b28108550 2024-04-25T18:48:04.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.469+0000 7f6b26ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b28108550 0x7f6b28101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.469+0000 7f6b26ffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b28108550 0x7f6b28101bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.79:53512/0 (socket says 172.21.15.79:53512) 2024-04-25T18:48:04.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.469+0000 7f6b26ffd700 1 -- 172.21.15.79:0/2498496355 learned_addr learned my addr 172.21.15.79:0/2498496355 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-25T18:48:04.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.469+0000 7f6b26ffd700 1 -- 172.21.15.79:0/2498496355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b28103aa0 con 0x7f6b28108550 2024-04-25T18:48:04.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.470+0000 7f6b26ffd700 1 --2- 172.21.15.79:0/2498496355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b28108550 0x7f6b28101bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b20006130 tx=0x7f6b200097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:04.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.470+0000 7f6b0ffff700 1 -- 172.21.15.79:0/2498496355 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b20021020 con 0x7f6b28108550 2024-04-25T18:48:04.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.470+0000 7f6b0ffff700 1 -- 172.21.15.79:0/2498496355 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b200079b0 con 0x7f6b28108550 2024-04-25T18:48:04.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.470+0000 7f6b2d787700 1 -- 172.21.15.79:0/2498496355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b28102150 con 0x7f6b28108550 2024-04-25T18:48:04.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.470+0000 7f6b0ffff700 1 -- 172.21.15.79:0/2498496355 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b20017040 con 0x7f6b28108550 2024-04-25T18:48:04.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.470+0000 7f6b2d787700 1 -- 172.21.15.79:0/2498496355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b28102610 con 0x7f6b28108550 2024-04-25T18:48:04.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.471+0000 7f6b2d787700 1 -- 172.21.15.79:0/2498496355 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b280640a0 con 0x7f6b28108550 2024-04-25T18:48:04.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.472+0000 7f6b0ffff700 1 -- 172.21.15.79:0/2498496355 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6b20024020 con 0x7f6b28108550 2024-04-25T18:48:04.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.472+0000 7f6b0ffff700 1 --2- 172.21.15.79:0/2498496355 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6b10039650 0x7f6b1003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:04.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.472+0000 7f6b0ffff700 1 -- 172.21.15.79:0/2498496355 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6b2000f050 con 0x7f6b28108550 2024-04-25T18:48:04.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.473+0000 7f6b267fc700 1 --2- 172.21.15.79:0/2498496355 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6b10039650 0x7f6b1003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:04.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.474+0000 7f6b267fc700 1 --2- 172.21.15.79:0/2498496355 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6b10039650 0x7f6b1003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c006fd0 tx=0x7f6b1c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:04.488 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.487+0000 7f6b0ffff700 1 -- 172.21.15.79:0/2498496355 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6b2002b020 con 0x7f6b28108550 2024-04-25T18:48:04.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.693+0000 7f6b2d787700 1 -- 172.21.15.79:0/2498496355 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi062:172.21.15.62=b;smithi079:172.21.15.79=c", "target": ["mon-mgr", ""]}) -- 0x7f6b28103d40 con 0x7f6b10039650 2024-04-25T18:48:04.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.708+0000 7f6b0ffff700 1 -- 172.21.15.79:0/2498496355 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6b1c00a040 con 0x7f6b10039650 2024-04-25T18:48:04.710 INFO:teuthology.orchestra.run.smithi079.stdout:Scheduled mon update... 2024-04-25T18:48:04.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.711+0000 7f6b2d787700 1 -- 172.21.15.79:0/2498496355 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6b10039650 msgr2=0x7f6b1003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.711+0000 7f6b2d787700 1 --2- 172.21.15.79:0/2498496355 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6b10039650 0x7f6b1003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c006fd0 tx=0x7f6b1c006e40 comp rx=0 tx=0).stop 2024-04-25T18:48:04.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.711+0000 7f6b2d787700 1 -- 172.21.15.79:0/2498496355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b28108550 msgr2=0x7f6b28101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:04.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.711+0000 7f6b2d787700 1 --2- 172.21.15.79:0/2498496355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b28108550 0x7f6b28101bb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6b20006130 tx=0x7f6b200097d0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.711+0000 7f6b2d787700 1 -- 172.21.15.79:0/2498496355 shutdown_connections 2024-04-25T18:48:04.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.711+0000 7f6b2d787700 1 --2- 172.21.15.79:0/2498496355 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6b10039650 0x7f6b1003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.711+0000 7f6b2d787700 1 --2- 172.21.15.79:0/2498496355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b28108550 0x7f6b28101bb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:04.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.711+0000 7f6b2d787700 1 -- 172.21.15.79:0/2498496355 >> 172.21.15.79:0/2498496355 conn(0x7f6b28077ac0 msgr2=0x7f6b28103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:04.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.711+0000 7f6b2d787700 1 -- 172.21.15.79:0/2498496355 shutdown_connections 2024-04-25T18:48:04.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:04.711+0000 7f6b2d787700 1 -- 172.21.15.79:0/2498496355 wait complete. 2024-04-25T18:48:05.217 DEBUG:teuthology.orchestra.run.smithi062:mon.b> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T18:48:05.220 DEBUG:teuthology.orchestra.run.smithi079:mon.c> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T18:48:05.222 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T18:48:05.222 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi062:172.21.15.62=b;smithi079:172.21.15.79=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi062:172.21.15.62=b;smithi079:172.21.15.79=c;count:3 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:05 smithi045 ceph-mon[27776]: Deploying daemon mon.c on smithi079 2024-04-25T18:48:06.393 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:07.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:07 smithi045 ceph-mon[27776]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:09.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.024+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2513791033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83b4106ac0 msgr2=0x7f83b4106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:09.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.024+0000 7f83ba4f5700 1 --2- 172.21.15.79:0/2513791033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83b4106ac0 0x7f83b4106ea0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009910 tx=0x7f83a4009c20 comp rx=0 tx=0).stop 2024-04-25T18:48:09.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.024+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2513791033 shutdown_connections 2024-04-25T18:48:09.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.024+0000 7f83ba4f5700 1 --2- 172.21.15.79:0/2513791033 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83b4106ac0 0x7f83b4106ea0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:09.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.024+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2513791033 >> 172.21.15.79:0/2513791033 conn(0x7f83b4077ac0 msgr2=0x7f83b4077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:09.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.024+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2513791033 shutdown_connections 2024-04-25T18:48:09.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.024+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2513791033 wait complete. 2024-04-25T18:48:09.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.024+0000 7f83ba4f5700 1 Processor -- start 2024-04-25T18:48:09.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.024+0000 7f83ba4f5700 1 -- start start 2024-04-25T18:48:09.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.025+0000 7f83ba4f5700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83b4106ac0 0x7f83b4196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:09.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.025+0000 7f83ba4f5700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f83b410afb0 con 0x7f83b4106ac0 2024-04-25T18:48:09.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.025+0000 7f83b3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83b4106ac0 0x7f83b4196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:09.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.025+0000 7f83b3fff700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83b4106ac0 0x7f83b4196dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.79:53526/0 (socket says 172.21.15.79:53526) 2024-04-25T18:48:09.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.025+0000 7f83b3fff700 1 -- 172.21.15.79:0/2382186142 learned_addr learned my addr 172.21.15.79:0/2382186142 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-25T18:48:09.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.026+0000 7f83b3fff700 1 -- 172.21.15.79:0/2382186142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b4197300 con 0x7f83b4106ac0 2024-04-25T18:48:09.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.026+0000 7f83b3fff700 1 --2- 172.21.15.79:0/2382186142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83b4106ac0 0x7f83b4196dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83a40056d0 tx=0x7f83a40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:09.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.026+0000 7f83b17fa700 1 -- 172.21.15.79:0/2382186142 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83a4026070 con 0x7f83b4106ac0 2024-04-25T18:48:09.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.026+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2382186142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b41975f0 con 0x7f83b4106ac0 2024-04-25T18:48:09.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.026+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2382186142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b419b690 con 0x7f83b4106ac0 2024-04-25T18:48:09.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.027+0000 7f83b17fa700 1 -- 172.21.15.79:0/2382186142 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a400b6f0 con 0x7f83b4106ac0 2024-04-25T18:48:09.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.027+0000 7f83b17fa700 1 -- 172.21.15.79:0/2382186142 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83a4030060 con 0x7f83b4106ac0 2024-04-25T18:48:09.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.027+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2382186142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a0005330 con 0x7f83b4106ac0 2024-04-25T18:48:09.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.028+0000 7f83b17fa700 1 -- 172.21.15.79:0/2382186142 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f83a4029070 con 0x7f83b4106ac0 2024-04-25T18:48:09.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.028+0000 7f83b17fa700 1 --2- 172.21.15.79:0/2382186142 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f839c039630 0x7f839c03baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:09.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.028+0000 7f83b17fa700 1 -- 172.21.15.79:0/2382186142 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f83a401e070 con 0x7f83b4106ac0 2024-04-25T18:48:09.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.029+0000 7f83b37fe700 1 --2- 172.21.15.79:0/2382186142 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f839c039630 0x7f839c03baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:09.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.029+0000 7f83b37fe700 1 --2- 172.21.15.79:0/2382186142 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f839c039630 0x7f839c03baf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83a8006fd0 tx=0x7f83a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:09.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.044+0000 7f83b17fa700 1 -- 172.21.15.79:0/2382186142 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f83a4013070 con 0x7f83b4106ac0 2024-04-25T18:48:09.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.349+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2382186142 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f83a0005620 con 0x7f83b4106ac0 2024-04-25T18:48:09.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.350+0000 7f83b17fa700 1 -- 172.21.15.79:0/2382186142 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f83a400d340 con 0x7f83b4106ac0 2024-04-25T18:48:09.352 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:48:09.352 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":1,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","modified":"2024-04-25T18:46:04.854399Z","created":"2024-04-25T18:46:04.854399Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T18:48:09.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.352+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2382186142 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f839c039630 msgr2=0x7f839c03baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.352+0000 7f83ba4f5700 1 --2- 172.21.15.79:0/2382186142 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f839c039630 0x7f839c03baf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f83a8006fd0 tx=0x7f83a8006e40 comp rx=0 tx=0).stop 2024-04-25T18:48:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.352+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2382186142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83b4106ac0 msgr2=0x7f83b4196dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.352+0000 7f83ba4f5700 1 --2- 172.21.15.79:0/2382186142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83b4106ac0 0x7f83b4196dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83a40056d0 tx=0x7f83a40057b0 comp rx=0 tx=0).stop 2024-04-25T18:48:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.352+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2382186142 shutdown_connections 2024-04-25T18:48:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.352+0000 7f83ba4f5700 1 --2- 172.21.15.79:0/2382186142 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f839c039630 0x7f839c03baf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.352+0000 7f83ba4f5700 1 --2- 172.21.15.79:0/2382186142 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f83b4106ac0 0x7f83b4196dc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.352+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2382186142 >> 172.21.15.79:0/2382186142 conn(0x7f83b4077ac0 msgr2=0x7f83b40ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.352+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2382186142 shutdown_connections 2024-04-25T18:48:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:09.352+0000 7f83ba4f5700 1 -- 172.21.15.79:0/2382186142 wait complete. 2024-04-25T18:48:09.354 INFO:teuthology.orchestra.run.smithi079.stderr:dumped monmap epoch 1 2024-04-25T18:48:09.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:09 smithi045 ceph-mon[27776]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:09.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:09 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/2382186142' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:48:10.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:10 smithi079 systemd[1]: Started Ceph mon.c for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:48:11.149 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T18:48:11.150 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T18:48:11.384 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:48:15.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:15.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:48:15.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: mon.a calling monitor election 2024-04-25T18:48:15.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:15.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: mon.c calling monitor election 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: monmap epoch 2 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: last_changed 2024-04-25T18:48:10.305264+0000 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: min_mon_release 19 (squid) 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: election_strategy: 1 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: fsmap 2024-04-25T18:48:15.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:15.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: mgrmap e13: x(active, since 59s) 2024-04-25T18:48:15.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: overall HEALTH_OK 2024-04-25T18:48:15.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:15.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:15.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:15.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:48:15.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: mon.a calling monitor election 2024-04-25T18:48:15.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:15.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: mon.c calling monitor election 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: monmap epoch 2 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: last_changed 2024-04-25T18:48:10.305264+0000 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: min_mon_release 19 (squid) 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: election_strategy: 1 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: fsmap 2024-04-25T18:48:15.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:15.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: mgrmap e13: x(active, since 59s) 2024-04-25T18:48:15.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: overall HEALTH_OK 2024-04-25T18:48:15.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:15.717 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:15.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.808+0000 7fd27c77b700 1 -- 172.21.15.79:0/1698461876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600035c0 msgr2=0x7fd260005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:15.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.808+0000 7fd27c77b700 1 --2- 172.21.15.79:0/1698461876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600035c0 0x7fd260005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd274101380 tx=0x7fd26c00feb0 comp rx=0 tx=0).stop 2024-04-25T18:48:15.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.809+0000 7fd27c77b700 1 -- 172.21.15.79:0/1698461876 shutdown_connections 2024-04-25T18:48:15.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.809+0000 7fd27c77b700 1 --2- 172.21.15.79:0/1698461876 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600035c0 0x7fd260005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:15.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.809+0000 7fd27c77b700 1 -- 172.21.15.79:0/1698461876 >> 172.21.15.79:0/1698461876 conn(0x7fd2740fcf80 msgr2=0x7fd2740ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:15.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.809+0000 7fd27c77b700 1 -- 172.21.15.79:0/1698461876 shutdown_connections 2024-04-25T18:48:15.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.809+0000 7fd27c77b700 1 -- 172.21.15.79:0/1698461876 wait complete. 2024-04-25T18:48:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.809+0000 7fd27c77b700 1 Processor -- start 2024-04-25T18:48:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd27c77b700 1 -- start start 2024-04-25T18:48:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd27c77b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600035c0 0x7fd2741030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd27c77b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2741035f0 0x7fd274103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd27c77b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2741093a0 con 0x7fd2600035c0 2024-04-25T18:48:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd27c77b700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd274109220 con 0x7fd2741035f0 2024-04-25T18:48:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd279d16700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2741035f0 0x7fd274103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd279d16700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2741035f0 0x7fd274103a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41346/0 (socket says 172.21.15.79:41346) 2024-04-25T18:48:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd279d16700 1 -- 172.21.15.79:0/3258438444 learned_addr learned my addr 172.21.15.79:0/3258438444 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-25T18:48:15.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd279d16700 1 -- 172.21.15.79:0/3258438444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2741035f0 msgr2=0x7fd274103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T18:48:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd279d16700 1 -- 172.21.15.79:0/3258438444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2741035f0 msgr2=0x7fd274103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T18:48:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd279d16700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2741035f0 0x7fd274103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T18:48:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd279d16700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2741035f0 0x7fd274103a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:48:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.810+0000 7fd27a517700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600035c0 0x7fd2741030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.811+0000 7fd27a517700 1 -- 172.21.15.79:0/3258438444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2741035f0 msgr2=0x7fd274103a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:48:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.811+0000 7fd27a517700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2741035f0 0x7fd274103a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:15.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.811+0000 7fd27a517700 1 -- 172.21.15.79:0/3258438444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd274104fd0 con 0x7fd2600035c0 2024-04-25T18:48:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.811+0000 7fd27a517700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600035c0 0x7fd2741030b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd26c010010 tx=0x7fd26c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.811+0000 7fd26b7fe700 1 -- 172.21.15.79:0/3258438444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd26c004aa0 con 0x7fd2600035c0 2024-04-25T18:48:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.812+0000 7fd27c77b700 1 -- 172.21.15.79:0/3258438444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2741a4220 con 0x7fd2600035c0 2024-04-25T18:48:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.812+0000 7fd26b7fe700 1 -- 172.21.15.79:0/3258438444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd26c004030 con 0x7fd2600035c0 2024-04-25T18:48:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.812+0000 7fd26b7fe700 1 -- 172.21.15.79:0/3258438444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd26c011b30 con 0x7fd2600035c0 2024-04-25T18:48:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.812+0000 7fd27c77b700 1 -- 172.21.15.79:0/3258438444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2741a4760 con 0x7fd2600035c0 2024-04-25T18:48:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.813+0000 7fd27c77b700 1 -- 172.21.15.79:0/3258438444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd274102b90 con 0x7fd2600035c0 2024-04-25T18:48:15.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.813+0000 7fd26b7fe700 1 -- 172.21.15.79:0/3258438444 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd26c023070 con 0x7fd2600035c0 2024-04-25T18:48:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.813+0000 7fd26b7fe700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd2580393f0 0x7fd25803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.813+0000 7fd26b7fe700 1 -- 172.21.15.79:0/3258438444 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd26c020050 con 0x7fd2600035c0 2024-04-25T18:48:15.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.814+0000 7fd279d16700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd2580393f0 0x7fd25803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:15.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.815+0000 7fd279d16700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd2580393f0 0x7fd25803b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd264009180 tx=0x7fd264008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:15.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:15.828+0000 7fd26b7fe700 1 -- 172.21.15.79:0/3258438444 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd26c028020 con 0x7fd2600035c0 2024-04-25T18:48:16.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.110+0000 7fd27c77b700 1 -- 172.21.15.79:0/3258438444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd2741a4a10 con 0x7fd2600035c0 2024-04-25T18:48:16.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.111+0000 7fd26b7fe700 1 -- 172.21.15.79:0/3258438444 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7fd26c02adc0 con 0x7fd2600035c0 2024-04-25T18:48:16.112 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:48:16.112 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":2,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","modified":"2024-04-25T18:48:10.305264Z","created":"2024-04-25T18:46:04.854399Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T18:48:16.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 -- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd2580393f0 msgr2=0x7fd25803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:16.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd2580393f0 0x7fd25803b8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd264009180 tx=0x7fd264008040 comp rx=0 tx=0).stop 2024-04-25T18:48:16.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 -- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600035c0 msgr2=0x7fd2741030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:16.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600035c0 0x7fd2741030b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd26c010010 tx=0x7fd26c005020 comp rx=0 tx=0).stop 2024-04-25T18:48:16.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 -- 172.21.15.79:0/3258438444 shutdown_connections 2024-04-25T18:48:16.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd2580393f0 0x7fd25803b8b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:16.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2600035c0 0x7fd2741030b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:16.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 --2- 172.21.15.79:0/3258438444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd2741035f0 0x7fd274103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:16.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 -- 172.21.15.79:0/3258438444 >> 172.21.15.79:0/3258438444 conn(0x7fd2740fcf80 msgr2=0x7fd27406b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:16.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 -- 172.21.15.79:0/3258438444 shutdown_connections 2024-04-25T18:48:16.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:16.113+0000 7fd27c77b700 1 -- 172.21.15.79:0/3258438444 wait complete. 2024-04-25T18:48:16.115 INFO:teuthology.orchestra.run.smithi079.stderr:dumped monmap epoch 2 2024-04-25T18:48:16.674 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:16.674 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:16.674 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:16.675 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:16 smithi079 ceph-mon[33002]: Deploying daemon mon.b on smithi062 2024-04-25T18:48:16.675 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:16 smithi079 ceph-mon[33002]: from='client.? 172.21.15.79:0/3258438444' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:48:16.675 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:16.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:16.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:16.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:16.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:16 smithi045 ceph-mon[27776]: Deploying daemon mon.b on smithi062 2024-04-25T18:48:16.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:16 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/3258438444' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:48:16.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:17.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:17 smithi079 ceph-mon[33002]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:17.701 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T18:48:17.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T18:48:17.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:17 smithi045 ceph-mon[27776]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:17.949 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:48:19.298 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 podman[32825]: 2024-04-25 18:48:19.10616022 +0000 UTC m=+0.020231108 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:48:19.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.334+0000 7f90c0892700 1 -- 172.21.15.79:0/385557749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90bc108630 msgr2=0x7f90bc108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:19.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.334+0000 7f90c0892700 1 --2- 172.21.15.79:0/385557749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90bc108630 0x7f90bc108a10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009910 tx=0x7f90ac009c20 comp rx=0 tx=0).stop 2024-04-25T18:48:19.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.335+0000 7f90c0892700 1 -- 172.21.15.79:0/385557749 shutdown_connections 2024-04-25T18:48:19.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.335+0000 7f90c0892700 1 --2- 172.21.15.79:0/385557749 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90bc108630 0x7f90bc108a10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:19.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.335+0000 7f90c0892700 1 -- 172.21.15.79:0/385557749 >> 172.21.15.79:0/385557749 conn(0x7f90bc0fe050 msgr2=0x7f90bc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:19.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.335+0000 7f90c0892700 1 -- 172.21.15.79:0/385557749 shutdown_connections 2024-04-25T18:48:19.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.335+0000 7f90c0892700 1 -- 172.21.15.79:0/385557749 wait complete. 2024-04-25T18:48:19.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.335+0000 7f90c0892700 1 Processor -- start 2024-04-25T18:48:19.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.335+0000 7f90c0892700 1 -- start start 2024-04-25T18:48:19.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90c0892700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90bc108630 0x7f90bc198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:19.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90c0892700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90bc198640 0x7f90bc19c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:19.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90c0892700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc10caa0 con 0x7f90bc108630 2024-04-25T18:48:19.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90c0892700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc10c920 con 0x7f90bc198640 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90b1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90bc198640 0x7f90bc19c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90b1d9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90bc198640 0x7f90bc19c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:41354/0 (socket says 172.21.15.79:41354) 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90b1d9b700 1 -- 172.21.15.79:0/719552115 learned_addr learned my addr 172.21.15.79:0/719552115 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90b1d9b700 1 -- 172.21.15.79:0/719552115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90bc198640 msgr2=0x7f90bc19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90b1d9b700 1 -- 172.21.15.79:0/719552115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90bc198640 msgr2=0x7f90bc19c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90b1d9b700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90bc198640 0x7f90bc19c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90b1d9b700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90bc198640 0x7f90bc19c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.336+0000 7f90ba59c700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90bc108630 0x7f90bc198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.337+0000 7f90ba59c700 1 -- 172.21.15.79:0/719552115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90bc198640 msgr2=0x7f90bc19c9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.337+0000 7f90ba59c700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90bc198640 0x7f90bc19c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.337+0000 7f90ba59c700 1 -- 172.21.15.79:0/719552115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc198c80 con 0x7f90bc108630 2024-04-25T18:48:19.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.337+0000 7f90ba59c700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90bc108630 0x7f90bc198100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90ac014000 tx=0x7f90ac005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:19.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.337+0000 7f90b3fff700 1 -- 172.21.15.79:0/719552115 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f90ac005dc0 con 0x7f90bc108630 2024-04-25T18:48:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.338+0000 7f90b3fff700 1 -- 172.21.15.79:0/719552115 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90ac00b750 con 0x7f90bc108630 2024-04-25T18:48:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.338+0000 7f90c0892700 1 -- 172.21.15.79:0/719552115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc19cf10 con 0x7f90bc108630 2024-04-25T18:48:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.338+0000 7f90b3fff700 1 -- 172.21.15.79:0/719552115 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f90ac0038f0 con 0x7f90bc108630 2024-04-25T18:48:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.338+0000 7f90c0892700 1 -- 172.21.15.79:0/719552115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc19d490 con 0x7f90bc108630 2024-04-25T18:48:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.339+0000 7f90b3fff700 1 -- 172.21.15.79:0/719552115 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f90ac029070 con 0x7f90bc108630 2024-04-25T18:48:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.339+0000 7f90c0892700 1 -- 172.21.15.79:0/719552115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90bc103ca0 con 0x7f90bc108630 2024-04-25T18:48:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.339+0000 7f90b3fff700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f90a80393f0 0x7f90a803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.339+0000 7f90b3fff700 1 -- 172.21.15.79:0/719552115 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f90ac01e070 con 0x7f90bc108630 2024-04-25T18:48:19.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.340+0000 7f90b1d9b700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f90a80393f0 0x7f90a803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:19.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.341+0000 7f90b1d9b700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f90a80393f0 0x7f90a803b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009180 tx=0x7f90a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:19.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.356+0000 7f90b3fff700 1 -- 172.21.15.79:0/719552115 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90ac013070 con 0x7f90bc108630 2024-04-25T18:48:19.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.439+0000 7f90b3fff700 1 -- 172.21.15.79:0/719552115 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00ec70 con 0x7f90bc108630 2024-04-25T18:48:19.591 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 podman[32825]: 2024-04-25 18:48:19.378183079 +0000 UTC m=+0.292253934 container init 24591487aabe27feffa0457c21ec120500e1ab6a92fd2de66e95b9774a9599de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:48:19.591 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 podman[32825]: 2024-04-25 18:48:19.381405271 +0000 UTC m=+0.295476118 container start 24591487aabe27feffa0457c21ec120500e1ab6a92fd2de66e95b9774a9599de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:48:19.591 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pidfile_write: ignore empty --pid-file 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: load: jerasure load: lrc 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: RocksDB version: 7.9.2 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Git sha 0 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: DB SUMMARY 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: DB Session ID: 8DLJD4FSXO2548KF430B 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: CURRENT file: CURRENT 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.error_if_exists: 0 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.create_if_missing: 0 2024-04-25T18:48:19.592 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.paranoid_checks: 1 2024-04-25T18:48:19.593 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T18:48:19.593 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T18:48:19.593 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T18:48:19.593 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.env: 0x55cb350dfcc0 2024-04-25T18:48:19.593 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T18:48:19.593 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.info_log: 0x55cb360f1be0 2024-04-25T18:48:19.593 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T18:48:19.594 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.statistics: (nil) 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.use_fsync: 0 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_log_file_size: 0 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.allow_fallocate: 1 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.use_direct_reads: 0 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.db_log_dir: 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.wal_dir: 2024-04-25T18:48:19.595 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.write_buffer_manager: 0x55cb360850e0 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T18:48:19.596 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.unordered_write: 0 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.row_cache: None 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.wal_filter: None 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.two_write_queues: 0 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.wal_compression: 0 2024-04-25T18:48:19.597 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.atomic_flush: 0 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.log_readahead_size: 0 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_background_jobs: 2 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_background_compactions: -1 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_subcompactions: 1 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T18:48:19.598 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_open_files: -1 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_background_flushes: -1 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Compression algorithms supported: 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: kZSTD supported: 0 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: kXpressCompression supported: 0 2024-04-25T18:48:19.599 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: kZlibCompression supported: 1 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: kSnappyCompression supported: 1 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: kLZ4Compression supported: 1 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: kBZip2Compression supported: 0 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.merge_operator: 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_filter: None 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T18:48:19.600 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cb360f1dc0) 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: cache_index_and_filter_blocks: 1 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: pin_top_level_index_and_filter: 1 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: index_type: 0 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: data_block_index_type: 0 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: index_shortening: 1 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: checksum: 4 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: no_block_cache: 0 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache: 0x55cb360b71f0 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache_name: BinnedLRUCache 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache_options: 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: capacity : 536870912 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: num_shard_bits : 4 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: strict_capacity_limit : 0 2024-04-25T18:48:19.601 INFO:journalctl@ceph.mon.b.smithi062.stdout: high_pri_pool_ratio: 0.000 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_cache_compressed: (nil) 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: persistent_cache: (nil) 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_size: 4096 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_size_deviation: 10 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_restart_interval: 16 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: index_block_restart_interval: 1 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: metadata_block_size: 4096 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: partition_filters: 0 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: use_delta_encoding: 1 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: filter_policy: bloomfilter 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: whole_key_filtering: 1 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: verify_compression: 0 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: read_amp_bytes_per_bit: 0 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: format_version: 5 2024-04-25T18:48:19.602 INFO:journalctl@ceph.mon.b.smithi062.stdout: enable_index_compression: 1 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout: block_align: 0 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout: max_auto_readahead_size: 262144 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout: prepopulate_block_cache: 0 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout: initial_auto_readahead_size: 8192 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compression: NoCompression 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.num_levels: 7 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T18:48:19.603 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T18:48:19.604 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T18:48:19.605 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T18:48:19.606 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.inplace_update_support: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.bloom_locality: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.max_successive_merges: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.ttl: 2592000 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T18:48:19.607 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.enable_blob_files: false 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.min_blob_size: 0 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7979dbeb-9517-4f66-8626-446aa89c07f2 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070899412064, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070899412482, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714070899, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7979dbeb-9517-4f66-8626-446aa89c07f2", "db_session_id": "8DLJD4FSXO2548KF430B", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714070899412563, "job": 1, "event": "recovery_finished"} 2024-04-25T18:48:19.608 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cb361cc000 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: DB pointer 0x55cb361b6000 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** DB Stats ** 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** Compaction Stats [default] ** 2024-04-25T18:48:19.609 INFO:journalctl@ceph.mon.b.smithi062.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** Compaction Stats [default] ** 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T18:48:19.610 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout: Block cache BinnedLRUCache@0x55cb360b71f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: using public_addr v2:172.21.15.62:0/0 -> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: starting mon.b rank -1 at public addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] at bind addrs [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(???) e0 preinit fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).mds e1 new map 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).mds e1 print_map 2024-04-25T18:48:19.611 INFO:journalctl@ceph.mon.b.smithi062.stdout: e1 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout: legacy client fscid: -1 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout: 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout: No filesystems configured 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.612 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi062", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:48:19.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi079", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Deploying cephadm binary to smithi079 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Added host smithi079 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi079", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1556423110' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Updating smithi079:/etc/ceph/ceph.conf 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:48:19.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:48:19.616 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1556423110' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi045:172.21.15.45=a;smithi062:172.21.15.62=b;smithi079:172.21.15.79=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Saving service mon spec with placement smithi045:172.21.15.45=a;smithi062:172.21.15.62=b;smithi079:172.21.15.79=c;count:3 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.617 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:19.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:19.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Deploying daemon mon.c on smithi079 2024-04-25T18:48:19.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/2382186142' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:48:19.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:19.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:48:19.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.a calling monitor election 2024-04-25T18:48:19.618 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.c calling monitor election 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 bash[32825]: 24591487aabe27feffa0457c21ec120500e1ab6a92fd2de66e95b9774a9599de 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: monmap epoch 2 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: last_changed 2024-04-25T18:48:10.305264+0000 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: min_mon_release 19 (squid) 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: election_strategy: 1 2024-04-25T18:48:19.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: fsmap 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mgrmap e13: x(active, since 59s) 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: overall HEALTH_OK 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: Deploying daemon mon.b on smithi062 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/3258438444' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 ceph-mon[32844]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T18:48:19.620 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:19 smithi062 systemd[1]: Started Ceph mon.b for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:48:19.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:19.643+0000 7f90c0892700 1 -- 172.21.15.79:0/719552115 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f90bc19d740 con 0x7f90bc108630 2024-04-25T18:48:19.646 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:19 smithi079 ceph-mon[33002]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:19.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:19 smithi045 ceph-mon[27776]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:24.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.461+0000 7f90b3fff700 1 -- 172.21.15.79:0/719552115 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7f90ac00b9b0 con 0x7f90bc108630 2024-04-25T18:48:24.462 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-25T18:48:24.462 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":3,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","modified":"2024-04-25T18:48:19.438591Z","created":"2024-04-25T18:46:04.854399Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:3300","nonce":0},{"type":"v1","addr":"172.21.15.79:6789","nonce":0}]},"addr":"172.21.15.79:6789/0","public_addr":"172.21.15.79:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:3300","nonce":0},{"type":"v1","addr":"172.21.15.62:6789","nonce":0}]},"addr":"172.21.15.62:6789/0","public_addr":"172.21.15.62:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T18:48:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 -- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f90a80393f0 msgr2=0x7f90a803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f90a80393f0 0x7f90a803b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f90a4009180 tx=0x7f90a4008040 comp rx=0 tx=0).stop 2024-04-25T18:48:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 -- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90bc108630 msgr2=0x7f90bc198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90bc108630 0x7f90bc198100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f90ac014000 tx=0x7f90ac005030 comp rx=0 tx=0).stop 2024-04-25T18:48:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 -- 172.21.15.79:0/719552115 shutdown_connections 2024-04-25T18:48:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f90a80393f0 0x7f90a803b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:24.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f90bc108630 0x7f90bc198100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 --2- 172.21.15.79:0/719552115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90bc198640 0x7f90bc19c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 -- 172.21.15.79:0/719552115 >> 172.21.15.79:0/719552115 conn(0x7f90bc0fe050 msgr2=0x7f90bc0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 -- 172.21.15.79:0/719552115 shutdown_connections 2024-04-25T18:48:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:24.464+0000 7f90c0892700 1 -- 172.21.15.79:0/719552115 wait complete. 2024-04-25T18:48:24.466 INFO:teuthology.orchestra.run.smithi079.stderr:dumped monmap epoch 3 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: mon.c calling monitor election 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: mon.a calling monitor election 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='client.? 172.21.15.79:0/719552115' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.779 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: monmap epoch 3 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: last_changed 2024-04-25T18:48:19.438591+0000 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: min_mon_release 19 (squid) 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: election_strategy: 1 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: 2: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: fsmap 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: mgrmap e13: x(active, since 68s) 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: overall HEALTH_OK 2024-04-25T18:48:24.780 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:24.781 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:24.781 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:24.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: mon.c calling monitor election 2024-04-25T18:48:24.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:48:24.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: mon.a calling monitor election 2024-04-25T18:48:24.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:24.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/719552115' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:48:24.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:24.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: monmap epoch 3 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: last_changed 2024-04-25T18:48:19.438591+0000 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: min_mon_release 19 (squid) 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: election_strategy: 1 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: 2: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2024-04-25T18:48:24.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: fsmap 2024-04-25T18:48:24.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:24.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: mgrmap e13: x(active, since 68s) 2024-04-25T18:48:24.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: overall HEALTH_OK 2024-04-25T18:48:24.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:24.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:24.966 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:25.158 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T18:48:25.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T18:48:25.609 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:48:25.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:25 smithi045 ceph-mon[27776]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:25.887 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:25.888 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:25.888 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:25.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:25 smithi079 ceph-mon[33002]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:25.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:25.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:25.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:26.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: mon.c calling monitor election 2024-04-25T18:48:26.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:48:26.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: mon.a calling monitor election 2024-04-25T18:48:26.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:26.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:48:26.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/719552115' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T18:48:26.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:26.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: monmap epoch 3 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: last_changed 2024-04-25T18:48:19.438591+0000 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: min_mon_release 19 (squid) 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: election_strategy: 1 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: 2: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2024-04-25T18:48:26.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: fsmap 2024-04-25T18:48:26.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:26.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: mgrmap e13: x(active, since 68s) 2024-04-25T18:48:26.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: overall HEALTH_OK 2024-04-25T18:48:26.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:26.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:26.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:26.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:26.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:26.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:26.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:26 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:27.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: mon.b calling monitor election 2024-04-25T18:48:27.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:27.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: Updating smithi079:/etc/ceph/ceph.conf 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: mon.b calling monitor election 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: mon.c calling monitor election 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: mon.a calling monitor election 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: monmap epoch 3 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: last_changed 2024-04-25T18:48:19.438591+0000 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: min_mon_release 19 (squid) 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: election_strategy: 1 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: 2: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2024-04-25T18:48:27.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: fsmap 2024-04-25T18:48:27.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:27.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: mgrmap e13: x(active, since 70s) 2024-04-25T18:48:27.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: overall HEALTH_OK 2024-04-25T18:48:27.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:27.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:27.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:27 smithi079 ceph-mon[33002]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: mon.b calling monitor election 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: Updating smithi079:/etc/ceph/ceph.conf 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: mon.b calling monitor election 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: mon.c calling monitor election 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: mon.a calling monitor election 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: monmap epoch 3 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: last_changed 2024-04-25T18:48:19.438591+0000 2024-04-25T18:48:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: min_mon_release 19 (squid) 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: election_strategy: 1 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: 2: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: fsmap 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: mgrmap e13: x(active, since 70s) 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: overall HEALTH_OK 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:27 smithi045 ceph-mon[27776]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: mon.b calling monitor election 2024-04-25T18:48:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T18:48:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-25T18:48:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: Updating smithi079:/etc/ceph/ceph.conf 2024-04-25T18:48:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: mon.b calling monitor election 2024-04-25T18:48:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: mon.c calling monitor election 2024-04-25T18:48:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: mon.a calling monitor election 2024-04-25T18:48:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: monmap epoch 3 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: last_changed 2024-04-25T18:48:19.438591+0000 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: created 2024-04-25T18:46:04.854399+0000 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: min_mon_release 19 (squid) 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: election_strategy: 1 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: 0: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.a 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: 1: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.c 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: 2: [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] mon.b 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: fsmap 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: mgrmap e13: x(active, since 70s) 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: overall HEALTH_OK 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:27.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.616 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.616 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.616 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:27.616 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:27 smithi062 ceph-mon[32844]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:48:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.729+0000 7fdba8a00700 1 -- 172.21.15.45:0/348808655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdba41084e0 msgr2=0x7fdba41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.729+0000 7fdba8a00700 1 --2- 172.21.15.45:0/348808655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdba41084e0 0x7fdba41088c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009910 tx=0x7fdb94009c20 comp rx=0 tx=0).stop 2024-04-25T18:48:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.730+0000 7fdba8a00700 1 -- 172.21.15.45:0/348808655 shutdown_connections 2024-04-25T18:48:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.730+0000 7fdba8a00700 1 --2- 172.21.15.45:0/348808655 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdba41084e0 0x7fdba41088c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.730+0000 7fdba8a00700 1 -- 172.21.15.45:0/348808655 >> 172.21.15.45:0/348808655 conn(0x7fdba40fdf80 msgr2=0x7fdba41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.730+0000 7fdba8a00700 1 -- 172.21.15.45:0/348808655 shutdown_connections 2024-04-25T18:48:27.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.730+0000 7fdba8a00700 1 -- 172.21.15.45:0/348808655 wait complete. 2024-04-25T18:48:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.731+0000 7fdba8a00700 1 Processor -- start 2024-04-25T18:48:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.731+0000 7fdba8a00700 1 -- start start 2024-04-25T18:48:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.731+0000 7fdba8a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdba419a560 0x7fdba419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:27.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.731+0000 7fdba8a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdba419ae80 0x7fdba419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.731+0000 7fdba8a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba419f370 0x7fdba419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.731+0000 7fdba8a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdba410c940 con 0x7fdba419a560 2024-04-25T18:48:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.731+0000 7fdba8a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdba410c7c0 con 0x7fdba419f370 2024-04-25T18:48:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.731+0000 7fdba8a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdba410cac0 con 0x7fdba419ae80 2024-04-25T18:48:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdba1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdba419a560 0x7fdba419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdba1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdba419a560 0x7fdba419a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42064/0 (socket says 172.21.15.45:42064) 2024-04-25T18:48:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdba1e00700 1 -- 172.21.15.45:0/3461870897 learned_addr learned my addr 172.21.15.45:0/3461870897 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:48:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdb9be00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdba419ae80 0x7fdba419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:27.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdba2800700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba419f370 0x7fdba419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdba1e00700 1 -- 172.21.15.45:0/3461870897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdba419ae80 msgr2=0x7fdba419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdba1e00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdba419ae80 0x7fdba419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdba1e00700 1 -- 172.21.15.45:0/3461870897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba419f370 msgr2=0x7fdba419f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:27.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdba1e00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba419f370 0x7fdba419f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdba1e00700 1 -- 172.21.15.45:0/3461870897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdba41a73b0 con 0x7fdba419a560 2024-04-25T18:48:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.732+0000 7fdb9be00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdba419ae80 0x7fdba419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.733+0000 7fdba1e00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdba419a560 0x7fdba419a940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdb94005950 tx=0x7fdb94004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.733+0000 7fdb9aa00700 1 -- 172.21.15.45:0/3461870897 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94009e60 con 0x7fdba419a560 2024-04-25T18:48:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.733+0000 7fdb9aa00700 1 -- 172.21.15.45:0/3461870897 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb9400b6f0 con 0x7fdba419a560 2024-04-25T18:48:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.733+0000 7fdb9aa00700 1 -- 172.21.15.45:0/3461870897 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940038e0 con 0x7fdba419a560 2024-04-25T18:48:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.733+0000 7fdba8a00700 1 -- 172.21.15.45:0/3461870897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdba41a76a0 con 0x7fdba419a560 2024-04-25T18:48:27.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.733+0000 7fdba8a00700 1 -- 172.21.15.45:0/3461870897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdba41a7be0 con 0x7fdba419a560 2024-04-25T18:48:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.735+0000 7fdb9aa00700 1 -- 172.21.15.45:0/3461870897 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdb94003a80 con 0x7fdba419a560 2024-04-25T18:48:27.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.735+0000 7fdba8a00700 1 -- 172.21.15.45:0/3461870897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdba40640a0 con 0x7fdba419a560 2024-04-25T18:48:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.735+0000 7fdb9aa00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdb88039ae0 0x7fdb8803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.735+0000 7fdb9aa00700 1 -- 172.21.15.45:0/3461870897 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdb9404d790 con 0x7fdba419a560 2024-04-25T18:48:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.736+0000 7fdb9be00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdb88039ae0 0x7fdb8803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:27.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.736+0000 7fdb9be00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdb88039ae0 0x7fdb8803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009ec0 tx=0x7fdb8c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:27.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.741+0000 7fdb9aa00700 1 -- 172.21.15.45:0/3461870897 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb94013070 con 0x7fdba419a560 2024-04-25T18:48:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.980+0000 7fdba8a00700 1 -- 172.21.15.45:0/3461870897 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fdba41a7e40 con 0x7fdba419a560 2024-04-25T18:48:27.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.980+0000 7fdb9aa00700 1 -- 172.21.15.45:0/3461870897 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7fdb94021b80 con 0x7fdba419a560 2024-04-25T18:48:27.981 INFO:teuthology.orchestra.run.smithi045.stdout:# minimal ceph.conf for ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:27.981 INFO:teuthology.orchestra.run.smithi045.stdout:[global] 2024-04-25T18:48:27.981 INFO:teuthology.orchestra.run.smithi045.stdout: fsid = ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:48:27.982 INFO:teuthology.orchestra.run.smithi045.stdout: mon_host = [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] 2024-04-25T18:48:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.983+0000 7fdba8a00700 1 -- 172.21.15.45:0/3461870897 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdb88039ae0 msgr2=0x7fdb8803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdb88039ae0 0x7fdb8803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009ec0 tx=0x7fdb8c009210 comp rx=0 tx=0).stop 2024-04-25T18:48:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 -- 172.21.15.45:0/3461870897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdba419a560 msgr2=0x7fdba419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdba419a560 0x7fdba419a940 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fdb94005950 tx=0x7fdb94004f80 comp rx=0 tx=0).stop 2024-04-25T18:48:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 -- 172.21.15.45:0/3461870897 shutdown_connections 2024-04-25T18:48:27.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdb88039ae0 0x7fdb8803bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:27.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdba419f370 0x7fdba419f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:27.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdba419a560 0x7fdba419a940 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:27.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 --2- 172.21.15.45:0/3461870897 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdba419ae80 0x7fdba419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:27.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 -- 172.21.15.45:0/3461870897 >> 172.21.15.45:0/3461870897 conn(0x7fdba40fdf80 msgr2=0x7fdba40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:27.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 -- 172.21.15.45:0/3461870897 shutdown_connections 2024-04-25T18:48:27.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:27.984+0000 7fdba8a00700 1 -- 172.21.15.45:0/3461870897 wait complete. 2024-04-25T18:48:28.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:28.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.395 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T18:48:28.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:28.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:48:28.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:28.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: Reconfiguring daemon mon.a on smithi045 2024-04-25T18:48:28.396 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3461870897' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:48:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:28.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: Reconfiguring daemon mon.a on smithi045 2024-04-25T18:48:28.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:28 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3461870897' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:28.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:48:28.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: Reconfiguring daemon mon.a on smithi045 2024-04-25T18:48:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:28 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3461870897' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:28.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:48:28 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:48:28.440+0000 7f032ea00700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T18:48:28.791 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T18:48:28.791 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:48:28.791 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T18:48:28.863 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:48:28.863 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:48:28.975 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:48:28.975 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T18:48:29.008 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:48:29.008 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:48:29.075 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:48:29.075 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T18:48:29.102 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:48:29.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:48:29.166 INFO:tasks.cephadm:Adding mgr.x on smithi045 2024-04-25T18:48:29.166 INFO:tasks.cephadm:Adding mgr.y on smithi062 2024-04-25T18:48:29.166 INFO:tasks.cephadm:Adding mgr.z on smithi079 2024-04-25T18:48:29.166 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi045=x;smithi062=y;smithi079=z' 2024-04-25T18:48:29.440 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:48:29.819 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:29 smithi079 ceph-mon[33002]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:29.894 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:29 smithi045 ceph-mon[27776]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:30.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:29 smithi062 ceph-mon[32844]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.853+0000 7ff855930700 1 -- 172.21.15.79:0/1495842606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8501008b0 msgr2=0x7ff850100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.853+0000 7ff855930700 1 --2- 172.21.15.79:0/1495842606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8501008b0 0x7ff850100cd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff840009910 tx=0x7ff840009c20 comp rx=0 tx=0).stop 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.853+0000 7ff855930700 1 -- 172.21.15.79:0/1495842606 shutdown_connections 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.853+0000 7ff855930700 1 --2- 172.21.15.79:0/1495842606 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8501008b0 0x7ff850100cd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.853+0000 7ff855930700 1 -- 172.21.15.79:0/1495842606 >> 172.21.15.79:0/1495842606 conn(0x7ff8500fc230 msgr2=0x7ff8500fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.853+0000 7ff855930700 1 -- 172.21.15.79:0/1495842606 shutdown_connections 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff855930700 1 -- 172.21.15.79:0/1495842606 wait complete. 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff855930700 1 Processor -- start 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff855930700 1 -- start start 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff855930700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8501008b0 0x7ff850197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff855930700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff850198330 0x7ff85019d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:30.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff855930700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8501987b0 0x7ff85019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff855930700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff850108530 con 0x7ff8501987b0 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff855930700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501083b0 con 0x7ff850198330 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff855930700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff8501086b0 con 0x7ff8501008b0 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff84effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8501008b0 0x7ff850197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff84effd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8501008b0 0x7ff850197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:60206/0 (socket says 172.21.15.79:60206) 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.854+0000 7ff84effd700 1 -- 172.21.15.79:0/1559193844 learned_addr learned my addr 172.21.15.79:0/1559193844 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff83ffff700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff850198330 0x7ff85019d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff84f7fe700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8501987b0 0x7ff85019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff84effd700 1 -- 172.21.15.79:0/1559193844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff850198330 msgr2=0x7ff85019d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff84effd700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff850198330 0x7ff85019d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff84effd700 1 -- 172.21.15.79:0/1559193844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8501987b0 msgr2=0x7ff85019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff84effd700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8501987b0 0x7ff85019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff84effd700 1 -- 172.21.15.79:0/1559193844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8501a0190 con 0x7ff8501008b0 2024-04-25T18:48:30.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff83ffff700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff850198330 0x7ff85019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:48:30.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff84effd700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8501008b0 0x7ff850197df0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff8400059d0 tx=0x7ff8400050d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:30.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff85492e700 1 -- 172.21.15.79:0/1559193844 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840027070 con 0x7ff8501008b0 2024-04-25T18:48:30.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff85492e700 1 -- 172.21.15.79:0/1559193844 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff84000be10 con 0x7ff8501008b0 2024-04-25T18:48:30.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff855930700 1 -- 172.21.15.79:0/1559193844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8501a0420 con 0x7ff8501008b0 2024-04-25T18:48:30.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff85492e700 1 -- 172.21.15.79:0/1559193844 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840022e20 con 0x7ff8501008b0 2024-04-25T18:48:30.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.855+0000 7ff855930700 1 -- 172.21.15.79:0/1559193844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8501a0980 con 0x7ff8501008b0 2024-04-25T18:48:30.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.856+0000 7ff85492e700 1 -- 172.21.15.79:0/1559193844 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff84002a070 con 0x7ff8501008b0 2024-04-25T18:48:30.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.856+0000 7ff855930700 1 -- 172.21.15.79:0/1559193844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff850191380 con 0x7ff8501008b0 2024-04-25T18:48:30.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.856+0000 7ff85492e700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff838039ae0 0x7ff83803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:30.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.856+0000 7ff85492e700 1 -- 172.21.15.79:0/1559193844 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff84004de10 con 0x7ff8501008b0 2024-04-25T18:48:30.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.857+0000 7ff83ffff700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff838039ae0 0x7ff83803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:30.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.858+0000 7ff83ffff700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff838039ae0 0x7ff83803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff844009910 tx=0x7ff844008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:30.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:30.862+0000 7ff85492e700 1 -- 172.21.15.79:0/1559193844 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff840013070 con 0x7ff8501008b0 2024-04-25T18:48:31.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.074+0000 7ff855930700 1 -- 172.21.15.79:0/1559193844 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi062=y;smithi079=z", "target": ["mon-mgr", ""]}) -- 0x7ff850062e80 con 0x7ff838039ae0 2024-04-25T18:48:31.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.087+0000 7ff85492e700 1 -- 172.21.15.79:0/1559193844 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff850062e80 con 0x7ff838039ae0 2024-04-25T18:48:31.088 INFO:teuthology.orchestra.run.smithi079.stdout:Scheduled mgr update... 2024-04-25T18:48:31.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 -- 172.21.15.79:0/1559193844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff838039ae0 msgr2=0x7ff83803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:31.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff838039ae0 0x7ff83803bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff844009910 tx=0x7ff844008040 comp rx=0 tx=0).stop 2024-04-25T18:48:31.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 -- 172.21.15.79:0/1559193844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8501008b0 msgr2=0x7ff850197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:31.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8501008b0 0x7ff850197df0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff8400059d0 tx=0x7ff8400050d0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 -- 172.21.15.79:0/1559193844 shutdown_connections 2024-04-25T18:48:31.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff838039ae0 0x7ff83803bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff8501987b0 0x7ff85019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff8501008b0 0x7ff850197df0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 --2- 172.21.15.79:0/1559193844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff850198330 0x7ff85019d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:31.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 -- 172.21.15.79:0/1559193844 >> 172.21.15.79:0/1559193844 conn(0x7ff8500fc230 msgr2=0x7ff8500fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:31.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 -- 172.21.15.79:0/1559193844 shutdown_connections 2024-04-25T18:48:31.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:48:31.089+0000 7ff855930700 1 -- 172.21.15.79:0/1559193844 wait complete. 2024-04-25T18:48:31.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:31.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:31.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:31 smithi062 ceph-mon[32844]: Reconfiguring mon.b (monmap changed)... 2024-04-25T18:48:31.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:31.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:48:31.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:31.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:31 smithi062 ceph-mon[32844]: Reconfiguring daemon mon.b on smithi062 2024-04-25T18:48:31.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:31 smithi062 ceph-mon[32844]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:31 smithi079 ceph-mon[33002]: Reconfiguring mon.b (monmap changed)... 2024-04-25T18:48:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:48:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:31 smithi079 ceph-mon[33002]: Reconfiguring daemon mon.b on smithi062 2024-04-25T18:48:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:31 smithi079 ceph-mon[33002]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:31.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:31.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:31.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:31 smithi045 ceph-mon[27776]: Reconfiguring mon.b (monmap changed)... 2024-04-25T18:48:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:48:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:31 smithi045 ceph-mon[27776]: Reconfiguring daemon mon.b on smithi062 2024-04-25T18:48:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:31 smithi045 ceph-mon[27776]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:31.664 DEBUG:teuthology.orchestra.run.smithi062:mgr.y> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T18:48:31.667 DEBUG:teuthology.orchestra.run.smithi079:mgr.z> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T18:48:31.669 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T18:48:31.670 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:48:31.670 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T18:48:31.734 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T18:48:31.734 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-04-25T18:48:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T18:48:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:31.785 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-25T18:48:31.785 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:31.785 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:31.785 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 18:47:29.205098963 +0000 2024-04-25T18:48:31.785 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 18:42:29.390308309 +0000 2024-04-25T18:48:31.785 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 18:42:29.390308309 +0000 2024-04-25T18:48:31.785 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 18:42:29.390308309 +0000 2024-04-25T18:48:31.785 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T18:48:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T18:48:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T18:48:31.857 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000304773 s, 1.7 MB/s 2024-04-25T18:48:31.861 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T18:48:31.916 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-04-25T18:48:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T18:48:31.968 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:31.968 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-04-25T18:48:31.968 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:31.968 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:31.968 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 18:47:29.272097948 +0000 2024-04-25T18:48:31.968 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 18:42:30.005298988 +0000 2024-04-25T18:48:31.968 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 18:42:30.005298988 +0000 2024-04-25T18:48:31.969 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 18:42:30.005298988 +0000 2024-04-25T18:48:31.969 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T18:48:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T18:48:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T18:48:32.054 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000279353 s, 1.8 MB/s 2024-04-25T18:48:32.058 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T18:48:32.119 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-04-25T18:48:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T18:48:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:32.173 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-25T18:48:32.173 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:32.173 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:32.173 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 18:47:29.353096722 +0000 2024-04-25T18:48:32.173 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 18:42:30.610289818 +0000 2024-04-25T18:48:32.173 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 18:42:30.610289818 +0000 2024-04-25T18:48:32.173 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 18:42:30.610289818 +0000 2024-04-25T18:48:32.174 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T18:48:32.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:32 smithi045 ceph-mon[27776]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi062=y;smithi079=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:48:32.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:32 smithi045 ceph-mon[27776]: Saving service mgr spec with placement smithi045=x;smithi062=y;smithi079=z;count:3 2024-04-25T18:48:32.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:32.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:32.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:32.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:32 smithi045 ceph-mon[27776]: Reconfiguring mon.c (monmap changed)... 2024-04-25T18:48:32.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:32.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:48:32.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:32.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:32 smithi045 ceph-mon[27776]: Reconfiguring daemon mon.c on smithi079 2024-04-25T18:48:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T18:48:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T18:48:32.258 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000277655 s, 1.8 MB/s 2024-04-25T18:48:32.262 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T18:48:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:32 smithi062 ceph-mon[32844]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi062=y;smithi079=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:48:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:32 smithi062 ceph-mon[32844]: Saving service mgr spec with placement smithi045=x;smithi062=y;smithi079=z;count:3 2024-04-25T18:48:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:32 smithi062 ceph-mon[32844]: Reconfiguring mon.c (monmap changed)... 2024-04-25T18:48:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:48:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:32 smithi062 ceph-mon[32844]: Reconfiguring daemon mon.c on smithi079 2024-04-25T18:48:32.366 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-04-25T18:48:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T18:48:32.421 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:32.421 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-25T18:48:32.422 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:32.422 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:32.422 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 18:47:29.437095449 +0000 2024-04-25T18:48:32.422 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 18:42:31.222280542 +0000 2024-04-25T18:48:32.422 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 18:42:31.222280542 +0000 2024-04-25T18:48:32.422 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 18:42:31.222280542 +0000 2024-04-25T18:48:32.422 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T18:48:32.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:32 smithi079 ceph-mon[33002]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi045=x;smithi062=y;smithi079=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:48:32.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:32 smithi079 ceph-mon[33002]: Saving service mgr spec with placement smithi045=x;smithi062=y;smithi079=z;count:3 2024-04-25T18:48:32.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:32.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:32.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:32.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:32 smithi079 ceph-mon[33002]: Reconfiguring mon.c (monmap changed)... 2024-04-25T18:48:32.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T18:48:32.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T18:48:32.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:32.423 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:32 smithi079 ceph-mon[33002]: Reconfiguring daemon mon.c on smithi079 2024-04-25T18:48:32.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-25T18:48:32.505 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-25T18:48:32.506 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000295106 s, 1.7 MB/s 2024-04-25T18:48:32.509 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T18:48:32.569 DEBUG:teuthology.orchestra.run.smithi062:> set -ex 2024-04-25T18:48:32.569 DEBUG:teuthology.orchestra.run.smithi062:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T18:48:32.589 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T18:48:32.589 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_1 2024-04-25T18:48:32.647 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T18:48:32.647 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:32.647 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T18:48:32.647 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:32.647 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:32.648 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 18:48:18.887506489 +0000 2024-04-25T18:48:32.648 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 18:41:14.249256415 +0000 2024-04-25T18:48:32.648 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 18:41:14.249256415 +0000 2024-04-25T18:48:32.648 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 18:41:14.249256415 +0000 2024-04-25T18:48:32.648 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T18:48:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T18:48:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T18:48:32.714 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000283638 s, 1.8 MB/s 2024-04-25T18:48:32.716 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T18:48:32.774 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_2 2024-04-25T18:48:32.830 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T18:48:32.831 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:32.831 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T18:48:32.831 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:32.831 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:32.831 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 18:48:18.887506489 +0000 2024-04-25T18:48:32.831 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 18:41:14.596251083 +0000 2024-04-25T18:48:32.831 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 18:41:14.596251083 +0000 2024-04-25T18:48:32.832 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 18:41:14.596251083 +0000 2024-04-25T18:48:32.832 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T18:48:32.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T18:48:32.896 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T18:48:32.896 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.0002036 s, 2.5 MB/s 2024-04-25T18:48:32.897 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T18:48:32.954 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_3 2024-04-25T18:48:33.010 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T18:48:33.010 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:33.010 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T18:48:33.010 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:33.010 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:33.010 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 18:48:18.888506475 +0000 2024-04-25T18:48:33.010 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 18:41:14.976245244 +0000 2024-04-25T18:48:33.010 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 18:41:14.976245244 +0000 2024-04-25T18:48:33.011 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 18:41:14.976245244 +0000 2024-04-25T18:48:33.011 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T18:48:33.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T18:48:33.074 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T18:48:33.074 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000207934 s, 2.5 MB/s 2024-04-25T18:48:33.076 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T18:48:33.134 DEBUG:teuthology.orchestra.run.smithi062:> stat /dev/vg_nvme/lv_4 2024-04-25T18:48:33.171 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:33 smithi079 ceph-mon[33002]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:33.189 INFO:teuthology.orchestra.run.smithi062.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T18:48:33.189 INFO:teuthology.orchestra.run.smithi062.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:33.189 INFO:teuthology.orchestra.run.smithi062.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T18:48:33.189 INFO:teuthology.orchestra.run.smithi062.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:33.190 INFO:teuthology.orchestra.run.smithi062.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:33.190 INFO:teuthology.orchestra.run.smithi062.stdout:Access: 2024-04-25 18:48:18.888506475 +0000 2024-04-25T18:48:33.190 INFO:teuthology.orchestra.run.smithi062.stdout:Modify: 2024-04-25 18:41:15.327239849 +0000 2024-04-25T18:48:33.190 INFO:teuthology.orchestra.run.smithi062.stdout:Change: 2024-04-25 18:41:15.327239849 +0000 2024-04-25T18:48:33.190 INFO:teuthology.orchestra.run.smithi062.stdout: Birth: 2024-04-25 18:41:15.327239849 +0000 2024-04-25T18:48:33.190 DEBUG:teuthology.orchestra.run.smithi062:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T18:48:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records in 2024-04-25T18:48:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:1+0 records out 2024-04-25T18:48:33.253 INFO:teuthology.orchestra.run.smithi062.stderr:512 bytes copied, 0.000227528 s, 2.3 MB/s 2024-04-25T18:48:33.254 DEBUG:teuthology.orchestra.run.smithi062:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T18:48:33.311 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-25T18:48:33.311 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T18:48:33.327 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T18:48:33.327 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-04-25T18:48:33.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:33 smithi062 ceph-mon[32844]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T18:48:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:33.381 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T18:48:33.381 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:33.381 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:33.381 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-04-25 18:48:08.946172284 +0000 2024-04-25T18:48:33.381 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-04-25 18:41:07.592148068 +0000 2024-04-25T18:48:33.381 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-04-25 18:41:07.592148068 +0000 2024-04-25T18:48:33.381 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-04-25 18:41:07.592148068 +0000 2024-04-25T18:48:33.382 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T18:48:33.444 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-04-25T18:48:33.444 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-04-25T18:48:33.444 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000193636 s, 2.6 MB/s 2024-04-25T18:48:33.445 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T18:48:33.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:33 smithi045 ceph-mon[27776]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:33.501 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-04-25T18:48:33.555 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T18:48:33.556 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:33.556 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T18:48:33.556 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:33.556 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:33.556 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-04-25 18:48:08.946172284 +0000 2024-04-25T18:48:33.556 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-04-25 18:41:07.933140574 +0000 2024-04-25T18:48:33.556 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-04-25 18:41:07.933140574 +0000 2024-04-25T18:48:33.556 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-04-25 18:41:07.933140574 +0000 2024-04-25T18:48:33.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T18:48:33.618 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-04-25T18:48:33.618 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-04-25T18:48:33.618 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.00020417 s, 2.5 MB/s 2024-04-25T18:48:33.619 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T18:48:33.675 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-04-25T18:48:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T18:48:33.729 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:33.729 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T18:48:33.729 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:33.730 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:33.730 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-04-25 18:48:08.947172262 +0000 2024-04-25T18:48:33.730 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-04-25 18:41:08.259133409 +0000 2024-04-25T18:48:33.730 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-04-25 18:41:08.259133409 +0000 2024-04-25T18:48:33.730 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-04-25 18:41:08.259133409 +0000 2024-04-25T18:48:33.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T18:48:33.792 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-04-25T18:48:33.792 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-04-25T18:48:33.792 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000229358 s, 2.2 MB/s 2024-04-25T18:48:33.793 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T18:48:33.849 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-04-25T18:48:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T18:48:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T18:48:33.905 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T18:48:33.905 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:48:33.905 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T18:48:33.905 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-04-25 18:48:08.947172262 +0000 2024-04-25T18:48:33.905 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-04-25 18:41:08.588126179 +0000 2024-04-25T18:48:33.905 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-04-25 18:41:08.588126179 +0000 2024-04-25T18:48:33.905 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-04-25 18:41:08.588126179 +0000 2024-04-25T18:48:33.906 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T18:48:33.967 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-04-25T18:48:33.967 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-04-25T18:48:33.967 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000195877 s, 2.6 MB/s 2024-04-25T18:48:33.968 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T18:48:34.023 INFO:tasks.cephadm:Deploying osd.0 on smithi045 with /dev/vg_nvme/lv_4... 2024-04-25T18:48:34.023 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T18:48:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:48:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:48:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:48:34.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:34.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:34 smithi079 ceph-mon[33002]: Deploying daemon mgr.z on smithi079 2024-04-25T18:48:34.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:34.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:34.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:34.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:48:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:48:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:48:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:34 smithi045 ceph-mon[27776]: Deploying daemon mgr.z on smithi079 2024-04-25T18:48:34.493 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:34 smithi062 ceph-mon[32844]: Deploying daemon mgr.z on smithi079 2024-04-25T18:48:35.485 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:35 smithi079 ceph-mon[33002]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:35.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:35 smithi062 ceph-mon[32844]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:35.657 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:35 smithi045 ceph-mon[27776]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:35.747 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:35 smithi079 systemd[1]: Started Ceph mgr.z for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:48:36.034 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:35 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:35.746+0000 7f0d753c5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:48:36.034 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:35 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:35.931+0000 7f0d753c5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:48:36.034 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:36.033+0000 7f0d753c5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:48:36.685 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:36.343+0000 7f0d753c5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:48:36.685 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:36.427+0000 7f0d753c5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:48:37.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:36 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:37.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:36 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:37.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:36 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:37.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:36 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:48:37.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:36 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:48:37.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:36 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:48:37.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:36 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:37.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:36 smithi062 ceph-mon[32844]: Deploying daemon mgr.y on smithi062 2024-04-25T18:48:37.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:37.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:37.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:37.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:48:37.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:48:37.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:48:37.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:37.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:36 smithi079 ceph-mon[33002]: Deploying daemon mgr.y on smithi062 2024-04-25T18:48:37.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:36 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:37.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:36 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:37.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:36 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:37.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:36 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:48:37.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:36 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T18:48:37.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:36 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:48:37.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:36 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:37.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:36 smithi045 ceph-mon[27776]: Deploying daemon mgr.y on smithi062 2024-04-25T18:48:37.219 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T18:48:37.219 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T18:48:37.220 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-25T18:48:37.220 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-25T18:48:37.220 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0347083 s, 302 MB/s 2024-04-25T18:48:37.220 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-25T18:48:37.803 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:37 smithi062 systemd[1]: Starting Ceph mgr.y for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:48:37.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-04-25T18:48:37.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:37 smithi079 ceph-mon[33002]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:37.934 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:37 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:37.628+0000 7f0d753c5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:48:37.934 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:37 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:37.798+0000 7f0d753c5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:48:38.101 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:37 smithi062 ceph-mon[32844]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:38.101 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:37 smithi062 podman[34227]: 2024-04-25 18:48:37.808190516 +0000 UTC m=+0.019348315 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:48:38.101 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:38 smithi062 podman[34227]: 2024-04-25 18:48:38.019527142 +0000 UTC m=+0.230684970 container create 4cd562e72ce52b9a396539c52f784b1121a85e97bbc0cc526d08f107d508eef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:48:38.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:37 smithi045 ceph-mon[27776]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:38.254 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:37 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:37.967+0000 7f0d753c5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:48:38.254 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:38 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:38.252+0000 7f0d753c5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:48:38.302 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:48:38.358 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:38 smithi062 podman[34227]: 2024-04-25 18:48:38.188453779 +0000 UTC m=+0.399611566 container init 4cd562e72ce52b9a396539c52f784b1121a85e97bbc0cc526d08f107d508eef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T18:48:38.358 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:38 smithi062 podman[34227]: 2024-04-25 18:48:38.191171043 +0000 UTC m=+0.402328830 container start 4cd562e72ce52b9a396539c52f784b1121a85e97bbc0cc526d08f107d508eef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T18:48:38.358 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:38 smithi062 ceph-mgr[34246]: -- 172.21.15.62:0/2854326960 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b490e525a0 con 0x55b490e27000 2024-04-25T18:48:38.358 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:38 smithi062 bash[34227]: 4cd562e72ce52b9a396539c52f784b1121a85e97bbc0cc526d08f107d508eef8 2024-04-25T18:48:38.358 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:38 smithi062 systemd[1]: Started Ceph mgr.y for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:48:38.545 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:38 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:38.343+0000 7f0d753c5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:48:38.545 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:38 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:38.544+0000 7f0d753c5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:48:38.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:38 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:38.356+0000 7f2ecbfb5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:48:38.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:38 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:38.552+0000 7f2ecbfb5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:48:38.933 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:38 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:38.746+0000 7f0d753c5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:48:38.996 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:38 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:38.659+0000 7f2ecbfb5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:48:39.333 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:38 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:38.995+0000 7f2ecbfb5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:48:39.333 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:39 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:39.085+0000 7f2ecbfb5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:48:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:39 smithi045 ceph-mon[27776]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:39 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:39 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:39 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:39 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:39 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:39 smithi062 ceph-mon[32844]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:39 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:39 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:39 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:39 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:39 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:39.626 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:39 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:39.388+0000 7f0d753c5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:48:39.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:39 smithi079 ceph-mon[33002]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:39.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:39 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:39 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:39 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:39 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:39.626 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:39 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:39.913 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:39 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:39.817+0000 7f0d753c5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:48:39.913 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:39 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:39.912+0000 7f0d753c5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:48:40.183 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:40 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:40.133+0000 7f0d753c5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:48:40.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.360+0000 7f32f0a00700 1 -- 172.21.15.45:0/794166304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32ec105ef0 msgr2=0x7f32ec106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.360+0000 7f32f0a00700 1 --2- 172.21.15.45:0/794166304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32ec105ef0 0x7f32ec106350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32d8009990 tx=0x7f32d8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:48:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.360+0000 7f32f0a00700 1 -- 172.21.15.45:0/794166304 shutdown_connections 2024-04-25T18:48:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.360+0000 7f32f0a00700 1 --2- 172.21.15.45:0/794166304 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32ec104cf0 0x7f32ec105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.360+0000 7f32f0a00700 1 --2- 172.21.15.45:0/794166304 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec106890 0x7f32ec10d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.360+0000 7f32f0a00700 1 --2- 172.21.15.45:0/794166304 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32ec105ef0 0x7f32ec106350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.360+0000 7f32f0a00700 1 -- 172.21.15.45:0/794166304 >> 172.21.15.45:0/794166304 conn(0x7f32ec100290 msgr2=0x7f32ec1026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:48:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.361+0000 7f32f0a00700 1 -- 172.21.15.45:0/794166304 shutdown_connections 2024-04-25T18:48:40.362 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.361+0000 7f32f0a00700 1 -- 172.21.15.45:0/794166304 wait complete. 2024-04-25T18:48:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32f0a00700 1 Processor -- start 2024-04-25T18:48:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32f0a00700 1 -- start start 2024-04-25T18:48:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32f0a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32ec104cf0 0x7f32ec19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32f0a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32ec105ef0 0x7f32ec19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32f0a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec106890 0x7f32ec1a23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32f0a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1107d0 con 0x7f32ec106890 2024-04-25T18:48:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32f0a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec110650 con 0x7f32ec104cf0 2024-04-25T18:48:40.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32f0a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec110950 con 0x7f32ec105ef0 2024-04-25T18:48:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32e9e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32ec104cf0 0x7f32ec19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32e9e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32ec104cf0 0x7f32ec19ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:57398/0 (socket says 172.21.15.45:57398) 2024-04-25T18:48:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32e9e00700 1 -- 172.21.15.45:0/547078730 learned_addr learned my addr 172.21.15.45:0/547078730 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:48:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.362+0000 7f32e9400700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32ec105ef0 0x7f32ec19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32ea800700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec106890 0x7f32ec1a23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32e9e00700 1 -- 172.21.15.45:0/547078730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32ec105ef0 msgr2=0x7f32ec19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:40.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32e9e00700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32ec105ef0 0x7f32ec19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32e9e00700 1 -- 172.21.15.45:0/547078730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec106890 msgr2=0x7f32ec1a23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:48:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32e9e00700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec106890 0x7f32ec1a23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:48:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32e9e00700 1 -- 172.21.15.45:0/547078730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec1a2a70 con 0x7f32ec104cf0 2024-04-25T18:48:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32e9400700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32ec105ef0 0x7f32ec19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:48:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32e9e00700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32ec104cf0 0x7f32ec19ae10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00efc0 tx=0x7f32dc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32e2a00700 1 -- 172.21.15.45:0/547078730 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0093a0 con 0x7f32ec104cf0 2024-04-25T18:48:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32e2a00700 1 -- 172.21.15.45:0/547078730 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32dc003c00 con 0x7f32ec104cf0 2024-04-25T18:48:40.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32f0a00700 1 -- 172.21.15.45:0/547078730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ec1a2d60 con 0x7f32ec104cf0 2024-04-25T18:48:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.363+0000 7f32e2a00700 1 -- 172.21.15.45:0/547078730 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc004850 con 0x7f32ec104cf0 2024-04-25T18:48:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.364+0000 7f32f0a00700 1 -- 172.21.15.45:0/547078730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ec1a3190 con 0x7f32ec104cf0 2024-04-25T18:48:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.365+0000 7f32e2a00700 1 -- 172.21.15.45:0/547078730 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f32dc012050 con 0x7f32ec104cf0 2024-04-25T18:48:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.365+0000 7f32f0a00700 1 -- 172.21.15.45:0/547078730 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ec069730 con 0x7f32ec104cf0 2024-04-25T18:48:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.365+0000 7f32e2a00700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f32d0039890 0x7f32d003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:48:40.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.365+0000 7f32e2a00700 1 -- 172.21.15.45:0/547078730 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f32dc04cad0 con 0x7f32ec104cf0 2024-04-25T18:48:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.365+0000 7f32e9400700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f32d0039890 0x7f32d003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:48:40.367 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.366+0000 7f32e9400700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f32d0039890 0x7f32d003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f32d800b720 tx=0x7f32d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:48:40.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.371+0000 7f32e2a00700 1 -- 172.21.15.45:0/547078730 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32dc04ee20 con 0x7f32ec104cf0 2024-04-25T18:48:40.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:40.621+0000 7f32f0a00700 1 -- 172.21.15.45:0/547078730 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f32ec062e80 con 0x7f32d0039890 2024-04-25T18:48:40.750 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:40.396+0000 7f2ecbfb5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:48:40.750 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:40.572+0000 7f2ecbfb5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:48:40.750 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:40.748+0000 7f2ecbfb5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:48:40.933 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:40 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:40.654+0000 7f0d753c5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:48:40.934 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:40 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:40.745+0000 7f0d753c5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:48:41.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:41.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:41.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.275 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:48:41.276 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:48:41.276 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:41.276 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:48:41.276 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:48:41.276 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:41.333 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:40 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:40.988+0000 7f0d753c5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:48:41.333 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:41.121+0000 7f0d753c5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:48:41.333 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:41.216+0000 7f0d753c5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:48:41.334 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:41.331+0000 7f0d753c5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:48:41.334 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.334 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.334 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.334 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.334 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:41.334 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:41.334 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.334 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:48:41.334 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:48:41.335 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:41.335 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:48:41.335 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:48:41.335 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:41.048+0000 7f2ecbfb5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:41.144+0000 7f2ecbfb5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:41.354+0000 7f2ecbfb5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T18:48:41.356 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T18:48:41.357 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:41.357 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:48:41.357 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:48:41.357 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:41.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:41 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:41.553+0000 7f2ecbfb5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:48:41.934 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:41.632+0000 7f0d753c5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:48:42.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:42 smithi045 ceph-mon[27776]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:42.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:42 smithi045 ceph-mon[27776]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T18:48:42.206 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:42 smithi045 ceph-mon[27776]: Reconfiguring daemon mgr.x on smithi045 2024-04-25T18:48:42.206 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:42 smithi045 ceph-mon[27776]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:48:42.206 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:42 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:42.363 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:42.182+0000 7f2ecbfb5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:48:42.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:42 smithi062 ceph-mon[32844]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:42.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:42 smithi062 ceph-mon[32844]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T18:48:42.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:42 smithi062 ceph-mon[32844]: Reconfiguring daemon mgr.x on smithi045 2024-04-25T18:48:42.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:42 smithi062 ceph-mon[32844]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:48:42.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:42 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:42.434 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:42.234+0000 7f0d753c5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:48:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:42 smithi079 ceph-mon[33002]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:42 smithi079 ceph-mon[33002]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T18:48:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:42 smithi079 ceph-mon[33002]: Reconfiguring daemon mgr.x on smithi045 2024-04-25T18:48:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:42 smithi079 ceph-mon[33002]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:48:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:42 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:42.928 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:42.618+0000 7f2ecbfb5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:48:42.929 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:42.718+0000 7f2ecbfb5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:48:42.929 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:42.927+0000 7f2ecbfb5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:48:42.934 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:48:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:48:42.449+0000 7f0d753c5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:48:43.411 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: Standby manager daemon z started 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:48:43.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:43 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:48:43.432 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:43.430+0000 7f32e2a00700 1 -- 172.21.15.45:0/547078730 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f32dc014480 con 0x7f32ec104cf0 2024-04-25T18:48:43.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:43.485+0000 7f2ecbfb5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:43.579+0000 7f2ecbfb5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: Standby manager daemon z started 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:48:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:48:43.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: Standby manager daemon z started 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:48:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:43 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.79:0/1471503654' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:48:44.113 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:43.830+0000 7f2ecbfb5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:48:44.113 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:43.968+0000 7f2ecbfb5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:48:44.114 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:44.066+0000 7f2ecbfb5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:48:44.497 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:44 smithi062 ceph-mon[32844]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-25T18:48:44.497 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:48:44.497 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:44.185+0000 7f2ecbfb5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:48:44.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:44 smithi079 ceph-mon[33002]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-25T18:48:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:48:44.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:44 smithi045 ceph-mon[27776]: mgrmap e14: x(active, since 87s), standbys: z 2024-04-25T18:48:44.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:48:44.863 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:44.496+0000 7f2ecbfb5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:48:45.436 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:45.124+0000 7f2ecbfb5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:48:45.436 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:48:45.347+0000 7f2ecbfb5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:48:45.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:45 smithi045 ceph-mon[27776]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:45.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:45 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/754787304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee46df06-8c1b-43ba-a8d7-c8c7a23e8995"}]: dispatch 2024-04-25T18:48:45.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:45 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/754787304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee46df06-8c1b-43ba-a8d7-c8c7a23e8995"}]': finished 2024-04-25T18:48:45.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:45 smithi045 ceph-mon[27776]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T18:48:45.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:48:45.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:45 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:48:45.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:45 smithi045 ceph-mon[27776]: Standby manager daemon y started 2024-04-25T18:48:45.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:45 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:48:45.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:45 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:48:45.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:45 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:48:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-mon[32844]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/754787304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee46df06-8c1b-43ba-a8d7-c8c7a23e8995"}]: dispatch 2024-04-25T18:48:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/754787304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee46df06-8c1b-43ba-a8d7-c8c7a23e8995"}]': finished 2024-04-25T18:48:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-mon[32844]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T18:48:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:48:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:48:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-mon[32844]: Standby manager daemon y started 2024-04-25T18:48:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:48:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:48:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:45 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:48:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:45 smithi079 ceph-mon[33002]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:45 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/754787304' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ee46df06-8c1b-43ba-a8d7-c8c7a23e8995"}]: dispatch 2024-04-25T18:48:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:45 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/754787304' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ee46df06-8c1b-43ba-a8d7-c8c7a23e8995"}]': finished 2024-04-25T18:48:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:45 smithi079 ceph-mon[33002]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T18:48:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:48:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:45 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:48:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:45 smithi079 ceph-mon[33002]: Standby manager daemon y started 2024-04-25T18:48:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:45 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:48:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:45 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:48:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:45 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.62:0/1609665868' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:48:46.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:48:46.266+0000 7f32e2a00700 1 -- 172.21.15.45:0/547078730 <== mon.2 v2:172.21.15.62:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f32dc003e20 con 0x7f32ec104cf0 2024-04-25T18:48:46.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:46 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3774328947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:48:46.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:46 smithi062 ceph-mon[32844]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-25T18:48:46.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:46 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:48:46.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:46 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3774328947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:48:46.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:46 smithi045 ceph-mon[27776]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-25T18:48:46.653 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:46 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:48:46.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:46 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3774328947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:48:46.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:46 smithi079 ceph-mon[33002]: mgrmap e15: x(active, since 90s), standbys: z, y 2024-04-25T18:48:46.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:46 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:48:47.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:47 smithi045 ceph-mon[27776]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:47.790 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:47 smithi079 ceph-mon[33002]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:47.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:47 smithi062 ceph-mon[32844]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:49.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:49 smithi045 ceph-mon[27776]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:49.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:49 smithi062 ceph-mon[32844]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:49.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:49 smithi079 ceph-mon[33002]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:51.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:51 smithi045 ceph-mon[27776]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:51.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:51 smithi062 ceph-mon[32844]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:51.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:51 smithi079 ceph-mon[33002]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:53.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:53 smithi045 ceph-mon[27776]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:53.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:53 smithi062 ceph-mon[32844]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:53.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:53 smithi079 ceph-mon[33002]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:54.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:54 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T18:48:54.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:54 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:54.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:54 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T18:48:54.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:54 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:54.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:54 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T18:48:54.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:54 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:48:55.635 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:55 smithi045 ceph-mon[27776]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:55.635 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:55 smithi045 ceph-mon[27776]: Deploying daemon osd.0 on smithi045 2024-04-25T18:48:55.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:55 smithi062 ceph-mon[32844]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:55.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:55 smithi062 ceph-mon[32844]: Deploying daemon osd.0 on smithi045 2024-04-25T18:48:55.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:55 smithi079 ceph-mon[33002]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:55.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:55 smithi079 ceph-mon[33002]: Deploying daemon osd.0 on smithi045 2024-04-25T18:48:57.794 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:57 smithi045 ceph-mon[27776]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:57.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:57 smithi062 ceph-mon[32844]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:57.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:57 smithi079 ceph-mon[33002]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:59.729 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:48:59 smithi045 ceph-mon[27776]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:59.736 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:48:59 smithi079 ceph-mon[33002]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:48:59.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:48:59 smithi062 ceph-mon[32844]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:01.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:01 smithi062 ceph-mon[32844]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:01.874 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:01 smithi045 ceph-mon[27776]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:01.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:01 smithi079 ceph-mon[33002]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:03.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:03 smithi045 ceph-mon[27776]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:03.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:03.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:03.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:03.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:03 smithi079 ceph-mon[33002]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:03.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:03.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:03.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:03 smithi062 ceph-mon[32844]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:05.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:05 smithi045 ceph-mon[27776]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:05.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:05.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:05.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:05.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:05.612 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:05.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:05 smithi062 ceph-mon[32844]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:05.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:05 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:05.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:05 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:05.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:05 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:05.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:05 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:05.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:05 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:05.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:05 smithi079 ceph-mon[33002]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:05.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:05 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:05.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:05 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:05.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:05 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:05.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:05 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:05.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:05 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:06.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:06 smithi079 ceph-mon[33002]: from='osd.0 [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:49:06.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:06 smithi079 ceph-mon[33002]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:49:06.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:06 smithi045 ceph-mon[27776]: from='osd.0 [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:49:06.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:06 smithi045 ceph-mon[27776]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:49:06.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:06 smithi062 ceph-mon[32844]: from='osd.0 [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:49:06.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:06 smithi062 ceph-mon[32844]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T18:49:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.219+0000 7f32e2a00700 1 -- 172.21.15.45:0/547078730 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f32ec062e80 con 0x7f32d0039890 2024-04-25T18:49:07.223 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 0 on host 'smithi045' 2024-04-25T18:49:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.223+0000 7f32f0a00700 1 -- 172.21.15.45:0/547078730 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f32d0039890 msgr2=0x7f32d003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:07.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.223+0000 7f32f0a00700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f32d0039890 0x7f32d003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f32d800b720 tx=0x7f32d8005fd0 comp rx=0 tx=0).stop 2024-04-25T18:49:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.223+0000 7f32f0a00700 1 -- 172.21.15.45:0/547078730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32ec104cf0 msgr2=0x7f32ec19ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.223+0000 7f32f0a00700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32ec104cf0 0x7f32ec19ae10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f32dc00efc0 tx=0x7f32dc00c3a0 comp rx=0 tx=0).stop 2024-04-25T18:49:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.224+0000 7f32f0a00700 1 -- 172.21.15.45:0/547078730 shutdown_connections 2024-04-25T18:49:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.224+0000 7f32f0a00700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f32d0039890 0x7f32d003bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.224+0000 7f32f0a00700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f32ec104cf0 0x7f32ec19ae10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:07.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.224+0000 7f32f0a00700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f32ec106890 0x7f32ec1a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.224+0000 7f32f0a00700 1 --2- 172.21.15.45:0/547078730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f32ec105ef0 0x7f32ec19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.224+0000 7f32f0a00700 1 -- 172.21.15.45:0/547078730 >> 172.21.15.45:0/547078730 conn(0x7f32ec100290 msgr2=0x7f32ec1091f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.224+0000 7f32f0a00700 1 -- 172.21.15.45:0/547078730 shutdown_connections 2024-04-25T18:49:07.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:49:07.224+0000 7f32f0a00700 1 -- 172.21.15.45:0/547078730 wait complete. 2024-04-25T18:49:07.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:07 smithi045 ceph-mon[27776]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:07.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:07 smithi045 ceph-mon[27776]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T18:49:07.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:07 smithi045 ceph-mon[27776]: from='osd.0 [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:49:07.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:07 smithi045 ceph-mon[27776]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T18:49:07.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:07 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:07.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:07 smithi045 ceph-mon[27776]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:49:07.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:07 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:07.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:07 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:07.655 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:07 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:07.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:07 smithi079 ceph-mon[33002]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:07.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:07 smithi079 ceph-mon[33002]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T18:49:07.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:07 smithi079 ceph-mon[33002]: from='osd.0 [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:49:07.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:07 smithi079 ceph-mon[33002]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T18:49:07.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:07 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:07.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:07 smithi079 ceph-mon[33002]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:49:07.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:07 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:07.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:07 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:07.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:07 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:07.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:07 smithi062 ceph-mon[32844]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:07 smithi062 ceph-mon[32844]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T18:49:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:07 smithi062 ceph-mon[32844]: from='osd.0 [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:49:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:07 smithi062 ceph-mon[32844]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T18:49:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:07 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:07 smithi062 ceph-mon[32844]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:49:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:07 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:07 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:07 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:08.034 DEBUG:teuthology.orchestra.run.smithi045:osd.0> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T18:49:08.039 INFO:tasks.cephadm:Deploying osd.1 on smithi062 with /dev/vg_nvme/lv_4... 2024-04-25T18:49:08.039 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T18:49:08.302 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:49:08.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:08 smithi045 ceph-mon[27776]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:49:08.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:08 smithi045 ceph-mon[27776]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T18:49:08.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:08 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:08.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:08 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:08.605 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 18:49:08 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0[39895]: 2024-04-25T18:49:08.370+0000 7f33c3800700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T18:49:08.606 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 18:49:08 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0[39895]: 2024-04-25T18:49:08.396+0000 7f33b9600700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:49:08.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:08 smithi079 ceph-mon[33002]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:49:08.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:08 smithi079 ceph-mon[33002]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T18:49:08.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:08 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:08.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:08 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:08.713 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:08 smithi062 ceph-mon[32844]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:49:08.714 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:08 smithi062 ceph-mon[32844]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T18:49:08.714 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:08 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:08.714 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:08 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:09.418 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:09 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:49:09.419 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:09 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:49:09.419 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:09 smithi062 ceph-mon[32844]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:09.419 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:09 smithi062 ceph-mon[32844]: from='osd.0 ' entity='osd.0' 2024-04-25T18:49:09.419 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:09 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:09.419 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:09 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:09.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:09 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:49:09.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:09 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:49:09.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:09 smithi045 ceph-mon[27776]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:09.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:09 smithi045 ceph-mon[27776]: from='osd.0 ' entity='osd.0' 2024-04-25T18:49:09.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:09 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:09.605 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:09 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:09.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:09 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:49:09.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:09 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:49:09.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:09 smithi079 ceph-mon[33002]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:09.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:09 smithi079 ceph-mon[33002]: from='osd.0 ' entity='osd.0' 2024-04-25T18:49:09.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:09 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:09.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:09 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:10.000 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T18:49:10.000 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T18:49:10.000 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-25T18:49:10.000 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-25T18:49:10.000 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113773 s, 922 MB/s 2024-04-25T18:49:10.001 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-25T18:49:10.559 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:10 smithi062 ceph-mon[32844]: osd.0 [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] boot 2024-04-25T18:49:10.559 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:10 smithi062 ceph-mon[32844]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T18:49:10.559 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:10.586 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_4 2024-04-25T18:49:10.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:10 smithi079 ceph-mon[33002]: osd.0 [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] boot 2024-04-25T18:49:10.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:10 smithi079 ceph-mon[33002]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T18:49:10.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:10.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:10 smithi045 ceph-mon[27776]: osd.0 [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] boot 2024-04-25T18:49:10.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:10 smithi045 ceph-mon[27776]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T18:49:10.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:49:10.824 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:49:11.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:11 smithi062 ceph-mon[32844]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:11.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:11 smithi062 ceph-mon[32844]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T18:49:11.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:11 smithi079 ceph-mon[33002]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:11.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:11 smithi079 ceph-mon[33002]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T18:49:11.706 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:11 smithi045 ceph-mon[27776]: pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T18:49:11.706 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:11 smithi045 ceph-mon[27776]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T18:49:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.141+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3921023891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad4108830 msgr2=0x7f6ad410ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.141+0000 7f6ad9c11700 1 --2- 172.21.15.62:0/3921023891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad4108830 0x7f6ad410ac20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6acc00b0a0 tx=0x7f6acc00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:49:12.144 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.141+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3921023891 shutdown_connections 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.141+0000 7f6ad9c11700 1 --2- 172.21.15.62:0/3921023891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad4108830 0x7f6ad410ac20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.141+0000 7f6ad9c11700 1 --2- 172.21.15.62:0/3921023891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ad41035d0 0x7f6ad41059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.141+0000 7f6ad9c11700 1 --2- 172.21.15.62:0/3921023891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad4105f00 0x7f6ad41082f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.141+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3921023891 >> 172.21.15.62:0/3921023891 conn(0x7f6ad40fcf40 msgr2=0x7f6ad40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.141+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3921023891 shutdown_connections 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.141+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3921023891 wait complete. 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad9c11700 1 Processor -- start 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad9c11700 1 -- start start 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad9c11700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad41035d0 0x7f6ad419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad9c11700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ad4105f00 0x7f6ad419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad9c11700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad4108830 0x7f6ad41a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad9c11700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad410e5a0 con 0x7f6ad4108830 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad9c11700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad410e420 con 0x7f6ad41035d0 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad9c11700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad410e720 con 0x7f6ad4105f00 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad41035d0 0x7f6ad419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:12.145 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad37fe700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad41035d0 0x7f6ad419ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:51784/0 (socket says 172.21.15.62:51784) 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad37fe700 1 -- 172.21.15.62:0/3062771207 learned_addr learned my addr 172.21.15.62:0/3062771207 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.142+0000 7f6ad2ffd700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ad4105f00 0x7f6ad419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad37fe700 1 -- 172.21.15.62:0/3062771207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ad4105f00 msgr2=0x7f6ad419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad37fe700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ad4105f00 0x7f6ad419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad37fe700 1 -- 172.21.15.62:0/3062771207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad4108830 msgr2=0x7f6ad41a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad37fe700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad4108830 0x7f6ad41a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad37fe700 1 -- 172.21.15.62:0/3062771207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad41a2af0 con 0x7f6ad41035d0 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad2ffd700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ad4105f00 0x7f6ad419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad37fe700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad41035d0 0x7f6ad419ae90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c970 tx=0x7f6abc00cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad0ff9700 1 -- 172.21.15.62:0/3062771207 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc007aa0 con 0x7f6ad41035d0 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad0ff9700 1 -- 172.21.15.62:0/3062771207 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6abc004ce0 con 0x7f6ad41035d0 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3062771207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad41a2de0 con 0x7f6ad41035d0 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad0ff9700 1 -- 172.21.15.62:0/3062771207 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6abc0055c0 con 0x7f6ad41035d0 2024-04-25T18:49:12.146 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.143+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3062771207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad41a32b0 con 0x7f6ad41035d0 2024-04-25T18:49:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.144+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3062771207 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad402df80 con 0x7f6ad41035d0 2024-04-25T18:49:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.145+0000 7f6ad0ff9700 1 -- 172.21.15.62:0/3062771207 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f6abc004750 con 0x7f6ad41035d0 2024-04-25T18:49:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.146+0000 7f6ad0ff9700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6ac00a3c30 0x7f6ac00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.146+0000 7f6ad0ff9700 1 -- 172.21.15.62:0/3062771207 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f6abc0ce770 con 0x7f6ad41035d0 2024-04-25T18:49:12.147 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.146+0000 7f6ad2ffd700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6ac00a3c30 0x7f6ac00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:12.148 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.147+0000 7f6ad2ffd700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6ac00a3c30 0x7f6ac00a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40099d0 tx=0x7f6ac4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:12.152 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.151+0000 7f6ad0ff9700 1 -- 172.21.15.62:0/3062771207 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6abc09cc50 con 0x7f6ad41035d0 2024-04-25T18:49:12.371 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:12.369+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3062771207 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6ad4062e80 con 0x7f6ac00a3c30 2024-04-25T18:49:12.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:12 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:49:12.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:12 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:49:12.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:12 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:12.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:12 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:49:12.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:12 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:49:12.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:12 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:12 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:49:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:12 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:49:12.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:12 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi045 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi045 to 17094M 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi045 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi045 to 17094M 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi045 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi045 to 17094M 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:13.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:15.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:15 smithi062 ceph-mon[32844]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:15.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:15 smithi062 ceph-mon[32844]: from='client.? 172.21.15.62:0/3923642808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550faceb-9f8f-4beb-8dd2-e0c6b4968c5e"}]: dispatch 2024-04-25T18:49:15.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:15 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550faceb-9f8f-4beb-8dd2-e0c6b4968c5e"}]: dispatch 2024-04-25T18:49:15.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:15 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "550faceb-9f8f-4beb-8dd2-e0c6b4968c5e"}]': finished 2024-04-25T18:49:15.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:15 smithi062 ceph-mon[32844]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T18:49:15.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:15.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:15 smithi062 ceph-mon[32844]: from='client.? 172.21.15.62:0/1811539332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:49:15.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:15 smithi079 ceph-mon[33002]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:15.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:15 smithi079 ceph-mon[33002]: from='client.? 172.21.15.62:0/3923642808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550faceb-9f8f-4beb-8dd2-e0c6b4968c5e"}]: dispatch 2024-04-25T18:49:15.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:15 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550faceb-9f8f-4beb-8dd2-e0c6b4968c5e"}]: dispatch 2024-04-25T18:49:15.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:15 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "550faceb-9f8f-4beb-8dd2-e0c6b4968c5e"}]': finished 2024-04-25T18:49:15.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:15 smithi079 ceph-mon[33002]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T18:49:15.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:15.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:15 smithi079 ceph-mon[33002]: from='client.? 172.21.15.62:0/1811539332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:49:15.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:15 smithi045 ceph-mon[27776]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:15.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:15 smithi045 ceph-mon[27776]: from='client.? 172.21.15.62:0/3923642808' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550faceb-9f8f-4beb-8dd2-e0c6b4968c5e"}]: dispatch 2024-04-25T18:49:15.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:15 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "550faceb-9f8f-4beb-8dd2-e0c6b4968c5e"}]: dispatch 2024-04-25T18:49:15.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:15 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "550faceb-9f8f-4beb-8dd2-e0c6b4968c5e"}]': finished 2024-04-25T18:49:15.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:15 smithi045 ceph-mon[27776]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T18:49:15.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:15.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:15 smithi045 ceph-mon[27776]: from='client.? 172.21.15.62:0/1811539332' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:49:17.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:17 smithi045 ceph-mon[27776]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:17 smithi062 ceph-mon[32844]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:17.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:17 smithi079 ceph-mon[33002]: pgmap v60: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:19.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:19 smithi062 ceph-mon[32844]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:19.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:19 smithi079 ceph-mon[33002]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:19.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:19 smithi045 ceph-mon[27776]: pgmap v61: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:21.715 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:21 smithi062 ceph-mon[32844]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:21.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:21 smithi079 ceph-mon[33002]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:21.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:21 smithi045 ceph-mon[27776]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:22.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T18:49:22.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:22.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T18:49:22.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:22.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T18:49:22.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:23.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:23 smithi062 ceph-mon[32844]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:23.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:23 smithi062 ceph-mon[32844]: Deploying daemon osd.1 on smithi062 2024-04-25T18:49:23.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:23 smithi079 ceph-mon[33002]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:23.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:23 smithi079 ceph-mon[33002]: Deploying daemon osd.1 on smithi062 2024-04-25T18:49:23.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:23 smithi045 ceph-mon[27776]: pgmap v63: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:23.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:23 smithi045 ceph-mon[27776]: Deploying daemon osd.1 on smithi062 2024-04-25T18:49:25.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:25 smithi062 ceph-mon[32844]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:25.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:25 smithi079 ceph-mon[33002]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:25.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:25 smithi045 ceph-mon[27776]: pgmap v64: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:27.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:27 smithi062 ceph-mon[32844]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:27.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:27 smithi079 ceph-mon[33002]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:27.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:27 smithi045 ceph-mon[27776]: pgmap v65: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:28.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:28.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:28.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:28.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:28.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:28.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:29.081 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:29.081 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:29.081 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:29.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:29 smithi079 ceph-mon[33002]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:29.954 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:29 smithi062 ceph-mon[32844]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:29.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:29 smithi045 ceph-mon[27776]: pgmap v66: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:31.220 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.219+0000 7f6ad0ff9700 1 -- 172.21.15.62:0/3062771207 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ad4062e80 con 0x7f6ac00a3c30 2024-04-25T18:49:31.221 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 1 on host 'smithi062' 2024-04-25T18:49:31.221 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:31.221 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:31.221 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:31.221 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:31.221 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:31.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.221+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3062771207 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6ac00a3c30 msgr2=0x7f6ac00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:31.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.221+0000 7f6ad9c11700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6ac00a3c30 0x7f6ac00a60f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40099d0 tx=0x7f6ac4008040 comp rx=0 tx=0).stop 2024-04-25T18:49:31.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.221+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3062771207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad41035d0 msgr2=0x7f6ad419ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:31.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.221+0000 7f6ad9c11700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad41035d0 0x7f6ad419ae90 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00c970 tx=0x7f6abc00cd30 comp rx=0 tx=0).stop 2024-04-25T18:49:31.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.222+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3062771207 shutdown_connections 2024-04-25T18:49:31.223 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.222+0000 7f6ad9c11700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6ac00a3c30 0x7f6ac00a60f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:31.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.222+0000 7f6ad9c11700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6ad41035d0 0x7f6ad419ae90 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:31.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.222+0000 7f6ad9c11700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6ad4108830 0x7f6ad41a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:31.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.222+0000 7f6ad9c11700 1 --2- 172.21.15.62:0/3062771207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ad4105f00 0x7f6ad419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:31.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.222+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3062771207 >> 172.21.15.62:0/3062771207 conn(0x7f6ad40fcf40 msgr2=0x7f6ad4102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:31.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.222+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3062771207 shutdown_connections 2024-04-25T18:49:31.224 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:49:31.222+0000 7f6ad9c11700 1 -- 172.21.15.62:0/3062771207 wait complete. 2024-04-25T18:49:31.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:31.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:31.773 DEBUG:teuthology.orchestra.run.smithi062:osd.1> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T18:49:31.775 INFO:tasks.cephadm:Deploying osd.2 on smithi079 with /dev/vg_nvme/lv_4... 2024-04-25T18:49:31.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T18:49:32.015 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:49:32.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:32 smithi062 ceph-mon[32844]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:32.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:32.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:32.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:32.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:32 smithi062 ceph-mon[32844]: from='osd.1 [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:49:32.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:32 smithi062 ceph-mon[32844]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:49:32.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:32 smithi079 ceph-mon[33002]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:32.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:32.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:32.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:32.136 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:32 smithi079 ceph-mon[33002]: from='osd.1 [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:49:32.137 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:32 smithi079 ceph-mon[33002]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:49:32.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:32 smithi045 ceph-mon[27776]: pgmap v67: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:32.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:32.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:32.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:32.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:32 smithi045 ceph-mon[27776]: from='osd.1 [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:49:32.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:32 smithi045 ceph-mon[27776]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T18:49:33.348 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:33 smithi079 ceph-mon[33002]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:33.348 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:33 smithi079 ceph-mon[33002]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T18:49:33.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:33 smithi079 ceph-mon[33002]: from='osd.1 [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:49:33.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:33 smithi079 ceph-mon[33002]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T18:49:33.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:33 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:33.349 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:33 smithi079 ceph-mon[33002]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:49:33.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:33 smithi062 ceph-mon[32844]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:33.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:33 smithi062 ceph-mon[32844]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T18:49:33.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:33 smithi062 ceph-mon[32844]: from='osd.1 [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:49:33.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:33 smithi062 ceph-mon[32844]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T18:49:33.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:33 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:33.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:33 smithi062 ceph-mon[32844]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:49:33.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:33 smithi045 ceph-mon[27776]: pgmap v68: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:33.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:33 smithi045 ceph-mon[27776]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T18:49:33.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:33 smithi045 ceph-mon[27776]: from='osd.1 [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:49:33.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:33 smithi045 ceph-mon[27776]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T18:49:33.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:33 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:33.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:33 smithi045 ceph-mon[27776]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:49:33.931 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T18:49:33.931 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T18:49:33.931 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-04-25T18:49:33.932 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-04-25T18:49:33.932 INFO:teuthology.orchestra.run.smithi079.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0150722 s, 696 MB/s 2024-04-25T18:49:33.932 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 2024-04-25T18:49:33.932 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-04-25T18:49:34.261 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:34 smithi079 ceph-mon[33002]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:49:34.261 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:34 smithi079 ceph-mon[33002]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T18:49:34.261 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:34.261 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:34.261 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:34 smithi079 ceph-mon[33002]: from='osd.1 ' entity='osd.1' 2024-04-25T18:49:34.357 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:34 smithi062 ceph-mon[32844]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:49:34.357 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:34 smithi062 ceph-mon[32844]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T18:49:34.357 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:34.357 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:34.357 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:34 smithi062 ceph-mon[32844]: from='osd.1 ' entity='osd.1' 2024-04-25T18:49:34.358 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 18:49:34 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1[37421]: 2024-04-25T18:49:34.133+0000 7f40aa44f700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T18:49:34.358 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 18:49:34 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1[37421]: 2024-04-25T18:49:34.166+0000 7f40a223a700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:49:34.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:34 smithi045 ceph-mon[27776]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:49:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:34 smithi045 ceph-mon[27776]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T18:49:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:34 smithi045 ceph-mon[27776]: from='osd.1 ' entity='osd.1' 2024-04-25T18:49:34.575 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_4 2024-04-25T18:49:34.817 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:49:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:49:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:49:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi062 2024-04-25T18:49:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:49:35.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi062 to 17088M 2024-04-25T18:49:35.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:35.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:35.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: osd.1 [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] boot 2024-04-25T18:49:35.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T18:49:35.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:35 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi062 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi062 to 17088M 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:35.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:35.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: osd.1 [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] boot 2024-04-25T18:49:35.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T18:49:35.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:35 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:35.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:49:35.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:49:35.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: pgmap v71: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-04-25T18:49:35.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:35.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi062 2024-04-25T18:49:35.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:49:35.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi062 to 17088M 2024-04-25T18:49:35.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:35.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:35.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:35.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: osd.1 [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] boot 2024-04-25T18:49:35.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T18:49:35.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:35 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:49:36.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.273+0000 7f8f45772700 1 -- 172.21.15.79:0/734911169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f40104640 msgr2=0x7f8f4010aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:36.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.273+0000 7f8f45772700 1 --2- 172.21.15.79:0/734911169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f40104640 0x7f8f4010aef0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800b0a0 tx=0x7f8f3800b3b0 comp rx=0 tx=0).stop 2024-04-25T18:49:36.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.273+0000 7f8f45772700 1 -- 172.21.15.79:0/734911169 shutdown_connections 2024-04-25T18:49:36.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.273+0000 7f8f45772700 1 --2- 172.21.15.79:0/734911169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f40102aa0 0x7f8f40102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:36.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.273+0000 7f8f45772700 1 --2- 172.21.15.79:0/734911169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f40104640 0x7f8f4010aef0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.273+0000 7f8f45772700 1 --2- 172.21.15.79:0/734911169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f40103ca0 0x7f8f40104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.273+0000 7f8f45772700 1 -- 172.21.15.79:0/734911169 >> 172.21.15.79:0/734911169 conn(0x7f8f400fe040 msgr2=0x7f8f40100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.273+0000 7f8f45772700 1 -- 172.21.15.79:0/734911169 shutdown_connections 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.273+0000 7f8f45772700 1 -- 172.21.15.79:0/734911169 wait complete. 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f45772700 1 Processor -- start 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f45772700 1 -- start start 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f45772700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f40102aa0 0x7f8f40198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f45772700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f40103ca0 0x7f8f40199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f45772700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f40104640 0x7f8f401a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f45772700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4010e630 con 0x7f8f40102aa0 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f45772700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4010e4b0 con 0x7f8f40103ca0 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f45772700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4010e7b0 con 0x7f8f40104640 2024-04-25T18:49:36.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f3f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f40104640 0x7f8f401a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f3f7fe700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f40104640 0x7f8f401a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:36006/0 (socket says 172.21.15.79:36006) 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.274+0000 7f8f3f7fe700 1 -- 172.21.15.79:0/2321258663 learned_addr learned my addr 172.21.15.79:0/2321258663 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f3e7fc700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f40103ca0 0x7f8f40199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f3f7fe700 1 -- 172.21.15.79:0/2321258663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f40103ca0 msgr2=0x7f8f40199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f3f7fe700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f40103ca0 0x7f8f40199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f3f7fe700 1 -- 172.21.15.79:0/2321258663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f40102aa0 msgr2=0x7f8f40198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f3f7fe700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f40102aa0 0x7f8f40198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f3f7fe700 1 -- 172.21.15.79:0/2321258663 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f401a0950 con 0x7f8f40104640 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f3f7fe700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f40104640 0x7f8f401a0280 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800b070 tx=0x7f8f38004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f37fff700 1 -- 172.21.15.79:0/2321258663 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f38021070 con 0x7f8f40104640 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f37fff700 1 -- 172.21.15.79:0/2321258663 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f38004020 con 0x7f8f40104640 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f45772700 1 -- 172.21.15.79:0/2321258663 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f401a0be0 con 0x7f8f40104640 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f37fff700 1 -- 172.21.15.79:0/2321258663 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3801d690 con 0x7f8f40104640 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.275+0000 7f8f45772700 1 -- 172.21.15.79:0/2321258663 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f401a10e0 con 0x7f8f40104640 2024-04-25T18:49:36.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.276+0000 7f8f45772700 1 -- 172.21.15.79:0/2321258663 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f40069730 con 0x7f8f40104640 2024-04-25T18:49:36.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.277+0000 7f8f37fff700 1 -- 172.21.15.79:0/2321258663 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f8f380044f0 con 0x7f8f40104640 2024-04-25T18:49:36.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.278+0000 7f8f37fff700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8f2c0a3c30 0x7f8f2c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:49:36.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.278+0000 7f8f37fff700 1 -- 172.21.15.79:0/2321258663 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f8f3800f020 con 0x7f8f40104640 2024-04-25T18:49:36.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.279+0000 7f8f3effd700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8f2c0a3c30 0x7f8f2c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:49:36.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.279+0000 7f8f3effd700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8f2c0a3c30 0x7f8f2c0a60f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f4019a1e0 tx=0x7f8f30006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:49:36.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.282+0000 7f8f37fff700 1 -- 172.21.15.79:0/2321258663 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f3802a020 con 0x7f8f40104640 2024-04-25T18:49:36.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:36.490+0000 7f8f45772700 1 -- 172.21.15.79:0/2321258663 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8f40062e80 con 0x7f8f2c0a3c30 2024-04-25T18:49:37.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:37 smithi079 ceph-mon[33002]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:37.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:37 smithi079 ceph-mon[33002]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T18:49:37.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:37 smithi079 ceph-mon[33002]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:49:37.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:49:37.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:49:37.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:37.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:37 smithi045 ceph-mon[27776]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:37.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:37 smithi045 ceph-mon[27776]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T18:49:37.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:37 smithi045 ceph-mon[27776]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:49:37.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:49:37.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:49:37.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:37 smithi062 ceph-mon[32844]: pgmap v73: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:37 smithi062 ceph-mon[32844]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T18:49:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:37 smithi062 ceph-mon[32844]: from='client.24166 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:49:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:49:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:49:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:39 smithi045 ceph-mon[27776]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:39 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/2336796547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30f605cc-9a0c-4410-8d6d-299c2413482b"}]: dispatch 2024-04-25T18:49:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:39 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30f605cc-9a0c-4410-8d6d-299c2413482b"}]: dispatch 2024-04-25T18:49:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:39 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30f605cc-9a0c-4410-8d6d-299c2413482b"}]': finished 2024-04-25T18:49:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:39 smithi045 ceph-mon[27776]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T18:49:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:39 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:39.517 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:39 smithi079 ceph-mon[33002]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:39.517 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:39 smithi079 ceph-mon[33002]: from='client.? 172.21.15.79:0/2336796547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30f605cc-9a0c-4410-8d6d-299c2413482b"}]: dispatch 2024-04-25T18:49:39.517 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:39 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30f605cc-9a0c-4410-8d6d-299c2413482b"}]: dispatch 2024-04-25T18:49:39.517 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:39 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30f605cc-9a0c-4410-8d6d-299c2413482b"}]': finished 2024-04-25T18:49:39.517 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:39 smithi079 ceph-mon[33002]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T18:49:39.517 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:39 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:39.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:39 smithi062 ceph-mon[32844]: pgmap v75: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:39 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/2336796547' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30f605cc-9a0c-4410-8d6d-299c2413482b"}]: dispatch 2024-04-25T18:49:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:39 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "30f605cc-9a0c-4410-8d6d-299c2413482b"}]: dispatch 2024-04-25T18:49:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:39 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "30f605cc-9a0c-4410-8d6d-299c2413482b"}]': finished 2024-04-25T18:49:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:39 smithi062 ceph-mon[32844]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T18:49:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:39 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:40.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:40 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/2660020762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:49:40.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:40 smithi079 ceph-mon[33002]: from='client.? 172.21.15.79:0/2660020762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:49:40.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:40 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/2660020762' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:49:41.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:41 smithi062 ceph-mon[32844]: pgmap v77: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:41.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:41 smithi079 ceph-mon[33002]: pgmap v77: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:41.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:41 smithi045 ceph-mon[27776]: pgmap v77: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:43.395 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:43 smithi079 ceph-mon[33002]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:43.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:43 smithi062 ceph-mon[32844]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:43.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:43 smithi045 ceph-mon[27776]: pgmap v78: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:45.505 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:45 smithi079 ceph-mon[33002]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:45.516 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:45 smithi045 ceph-mon[27776]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:45.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:45 smithi062 ceph-mon[32844]: pgmap v79: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:47.555 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:47 smithi079 ceph-mon[33002]: pgmap v80: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:47.555 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T18:49:47.555 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:47.555 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:47 smithi079 ceph-mon[33002]: Deploying daemon osd.2 on smithi079 2024-04-25T18:49:47.585 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:47 smithi062 ceph-mon[32844]: pgmap v80: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:47.585 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T18:49:47.585 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:47.585 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:47 smithi062 ceph-mon[32844]: Deploying daemon osd.2 on smithi079 2024-04-25T18:49:47.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:47 smithi045 ceph-mon[27776]: pgmap v80: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:47.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T18:49:47.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:47.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:47 smithi045 ceph-mon[27776]: Deploying daemon osd.2 on smithi079 2024-04-25T18:49:49.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:49 smithi062 ceph-mon[32844]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:49.650 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:49 smithi079 ceph-mon[33002]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:49.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:49 smithi045 ceph-mon[27776]: pgmap v81: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:51.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:51 smithi079 ceph-mon[33002]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:51 smithi062 ceph-mon[32844]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:51.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:51 smithi045 ceph-mon[27776]: pgmap v82: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:52.545 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:52 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:52.545 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:52 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:52.545 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:52 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:52.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:52 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:52.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:52 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:52.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:52 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:52.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:52 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:52.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:52 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:52.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:52 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:53.965 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:53 smithi079 ceph-mon[33002]: pgmap v83: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:54.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:53 smithi045 ceph-mon[27776]: pgmap v83: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:54.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:53 smithi062 ceph-mon[32844]: pgmap v83: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:55.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.224+0000 7f8f37fff700 1 -- 172.21.15.79:0/2321258663 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8f40062e80 con 0x7f8f2c0a3c30 2024-04-25T18:49:55.228 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 2 on host 'smithi079' 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 -- 172.21.15.79:0/2321258663 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8f2c0a3c30 msgr2=0x7f8f2c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8f2c0a3c30 0x7f8f2c0a60f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f4019a1e0 tx=0x7f8f30006d20 comp rx=0 tx=0).stop 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 -- 172.21.15.79:0/2321258663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f40104640 msgr2=0x7f8f401a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f40104640 0x7f8f401a0280 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f3800b070 tx=0x7f8f38004ef0 comp rx=0 tx=0).stop 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 -- 172.21.15.79:0/2321258663 shutdown_connections 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f8f2c0a3c30 0x7f8f2c0a60f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8f40104640 0x7f8f401a0280 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8f40102aa0 0x7f8f40198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 --2- 172.21.15.79:0/2321258663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8f40103ca0 0x7f8f40199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 -- 172.21.15.79:0/2321258663 >> 172.21.15.79:0/2321258663 conn(0x7f8f400fe040 msgr2=0x7f8f401070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:49:55.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 -- 172.21.15.79:0/2321258663 shutdown_connections 2024-04-25T18:49:55.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:49:55.227+0000 7f8f45772700 1 -- 172.21.15.79:0/2321258663 wait complete. 2024-04-25T18:49:55.894 DEBUG:teuthology.orchestra.run.smithi079:osd.2> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T18:49:55.896 INFO:tasks.cephadm:Deploying osd.3 on smithi045 with /dev/vg_nvme/lv_3... 2024-04-25T18:49:55.896 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T18:49:56.165 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:55 smithi079 ceph-mon[33002]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:56.165 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:55 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:55 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:55 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:55 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:55 smithi079 ceph-mon[33002]: from='osd.2 [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:49:56.166 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:55 smithi079 ceph-mon[33002]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:49:56.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:55 smithi045 ceph-mon[27776]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:56.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:55 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:55 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:55 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:55 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:55 smithi045 ceph-mon[27776]: from='osd.2 [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:49:56.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:55 smithi045 ceph-mon[27776]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:49:56.343 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:49:56.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:55 smithi062 ceph-mon[32844]: pgmap v84: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:56.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:55 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:55 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:55 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:55 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:56.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:55 smithi062 ceph-mon[32844]: from='osd.2 [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:49:56.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:55 smithi062 ceph-mon[32844]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T18:49:57.524 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:57 smithi045 ceph-mon[27776]: pgmap v85: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:57 smithi045 ceph-mon[27776]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T18:49:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:57 smithi045 ceph-mon[27776]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T18:49:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:57 smithi045 ceph-mon[27776]: from='osd.2 [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:49:57.525 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:57 smithi045 ceph-mon[27776]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:49:57.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:57 smithi062 ceph-mon[32844]: pgmap v85: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:57.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:57 smithi062 ceph-mon[32844]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T18:49:57.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:57 smithi062 ceph-mon[32844]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T18:49:57.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:57.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:57 smithi062 ceph-mon[32844]: from='osd.2 [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:49:57.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:57 smithi062 ceph-mon[32844]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:49:57.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:57 smithi079 ceph-mon[33002]: pgmap v85: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:57.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:57 smithi079 ceph-mon[33002]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T18:49:57.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:57 smithi079 ceph-mon[33002]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T18:49:57.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:57.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:57 smithi079 ceph-mon[33002]: from='osd.2 [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:49:57.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:57 smithi079 ceph-mon[33002]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:49:57.685 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 18:49:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2[38165]: 2024-04-25T18:49:57.313+0000 7f45fb5cf700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T18:49:57.685 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 18:49:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2[38165]: 2024-04-25T18:49:57.316+0000 7f45f73c2700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:49:58.420 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:58 smithi079 ceph-mon[33002]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:49:58.421 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:58 smithi079 ceph-mon[33002]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T18:49:58.421 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:58.421 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:58.421 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:58.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:58 smithi045 ceph-mon[27776]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:49:58.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:58 smithi045 ceph-mon[27776]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T18:49:58.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:58.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:58.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:58.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:58 smithi062 ceph-mon[32844]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:49:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:58 smithi062 ceph-mon[32844]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T18:49:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:59.039 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T18:49:59.039 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T18:49:59.039 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-25T18:49:59.039 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-25T18:49:59.039 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0356039 s, 295 MB/s 2024-04-25T18:49:59.039 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-25T18:49:59.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:49:59.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:49:59.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: pgmap v88: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:59.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: osd.2 [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] boot 2024-04-25T18:49:59.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T18:49:59.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:59.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi079 2024-04-25T18:49:59.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:49:59.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi079 to 17088M 2024-04-25T18:49:59.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:59.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:59.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:59.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:59.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:59.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:49:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:49:59.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:49:59.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: pgmap v88: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:59.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: osd.2 [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] boot 2024-04-25T18:49:59.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T18:49:59.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:59.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi079 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi079 to 17088M 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:59.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:49:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:49:59.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:49:59.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: pgmap v88: 0 pgs: ; 0 B data, 69 MiB used, 179 GiB / 179 GiB avail 2024-04-25T18:49:59.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: osd.2 [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] boot 2024-04-25T18:49:59.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi079 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi079 to 17088M 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:49:59.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:49:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:49:59.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-04-25T18:50:00.259 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:50:00.962 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:00 smithi079 ceph-mon[33002]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T18:50:00.962 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:00 smithi079 ceph-mon[33002]: overall HEALTH_OK 2024-04-25T18:50:00.962 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:00 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:50:00.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:00 smithi045 ceph-mon[27776]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T18:50:00.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:00 smithi045 ceph-mon[27776]: overall HEALTH_OK 2024-04-25T18:50:00.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:00 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:50:01.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:00 smithi062 ceph-mon[32844]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T18:50:01.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:00 smithi062 ceph-mon[32844]: overall HEALTH_OK 2024-04-25T18:50:01.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:00 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:50:02.030 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44113]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:50:02.030 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44113]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.030 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44113]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.030 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44113]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:01 smithi045 ceph-mon[27776]: pgmap v91: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:02.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:01 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T18:50:02.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:01 smithi045 ceph-mon[27776]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T18:50:02.030 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:01 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:50:02.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:01 smithi062 ceph-mon[32844]: pgmap v91: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:02.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:01 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T18:50:02.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:01 smithi062 ceph-mon[32844]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T18:50:02.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:01 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:50:02.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:01 smithi079 ceph-mon[33002]: pgmap v91: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:02.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:01 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T18:50:02.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:01 smithi079 ceph-mon[33002]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T18:50:02.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:01 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T18:50:02.308 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44116]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:50:02.308 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44116]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.308 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44116]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.308 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44116]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.327+0000 7fdcc5600700 1 -- 172.21.15.45:0/2596007516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcc0103530 msgr2=0x7fdcc0105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.327+0000 7fdcc5600700 1 --2- 172.21.15.45:0/2596007516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcc0103530 0x7fdcc0105920 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009990 tx=0x7fdcb0009ca0 comp rx=0 tx=0).stop 2024-04-25T18:50:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.327+0000 7fdcc5600700 1 -- 172.21.15.45:0/2596007516 shutdown_connections 2024-04-25T18:50:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.327+0000 7fdcc5600700 1 --2- 172.21.15.45:0/2596007516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcc0108790 0x7fdcc010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:02.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.327+0000 7fdcc5600700 1 --2- 172.21.15.45:0/2596007516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcc0103530 0x7fdcc0105920 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.327+0000 7fdcc5600700 1 --2- 172.21.15.45:0/2596007516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc0105e60 0x7fdcc0108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.327+0000 7fdcc5600700 1 -- 172.21.15.45:0/2596007516 >> 172.21.15.45:0/2596007516 conn(0x7fdcc00fcf40 msgr2=0x7fdcc00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.327+0000 7fdcc5600700 1 -- 172.21.15.45:0/2596007516 shutdown_connections 2024-04-25T18:50:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.327+0000 7fdcc5600700 1 -- 172.21.15.45:0/2596007516 wait complete. 2024-04-25T18:50:02.330 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.328+0000 7fdcc5600700 1 Processor -- start 2024-04-25T18:50:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.328+0000 7fdcc5600700 1 -- start start 2024-04-25T18:50:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcc5600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcc0103530 0x7fdcc019aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcc5600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc0105e60 0x7fdcc019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcc5600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcc0108790 0x7fdcc01a2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcc5600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc010e590 con 0x7fdcc0103530 2024-04-25T18:50:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcc5600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc010e410 con 0x7fdcc0105e60 2024-04-25T18:50:02.331 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcc5600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc010e710 con 0x7fdcc0108790 2024-04-25T18:50:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcbea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcc0103530 0x7fdcc019aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcbea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcc0103530 0x7fdcc019aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34210/0 (socket says 172.21.15.45:34210) 2024-04-25T18:50:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcbea00700 1 -- 172.21.15.45:0/2751123935 learned_addr learned my addr 172.21.15.45:0/2751123935 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:50:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcbf400700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcc0108790 0x7fdcc01a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:02.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.329+0000 7fdcbe000700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc0105e60 0x7fdcc019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.330+0000 7fdcbe000700 1 -- 172.21.15.45:0/2751123935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcc0108790 msgr2=0x7fdcc01a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.330+0000 7fdcbe000700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcc0108790 0x7fdcc01a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.330+0000 7fdcbe000700 1 -- 172.21.15.45:0/2751123935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcc0103530 msgr2=0x7fdcc019aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.330+0000 7fdcbe000700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcc0103530 0x7fdcc019aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.330+0000 7fdcbe000700 1 -- 172.21.15.45:0/2751123935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc01a2b20 con 0x7fdcc0105e60 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.330+0000 7fdcbe000700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc0105e60 0x7fdcc019b400 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00e9f0 tx=0x7fdcac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.330+0000 7fdcb7400700 1 -- 172.21.15.45:0/2751123935 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac004220 con 0x7fdcc0105e60 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.330+0000 7fdcb7400700 1 -- 172.21.15.45:0/2751123935 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcac0043c0 con 0x7fdcc0105e60 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.330+0000 7fdcc5600700 1 -- 172.21.15.45:0/2751123935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc01a2e10 con 0x7fdcc0105e60 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.330+0000 7fdcb7400700 1 -- 172.21.15.45:0/2751123935 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac005020 con 0x7fdcc0105e60 2024-04-25T18:50:02.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.331+0000 7fdcc5600700 1 -- 172.21.15.45:0/2751123935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc01a32c0 con 0x7fdcc0105e60 2024-04-25T18:50:02.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.332+0000 7fdcc5600700 1 -- 172.21.15.45:0/2751123935 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc0069730 con 0x7fdcc0105e60 2024-04-25T18:50:02.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.333+0000 7fdcb7400700 1 -- 172.21.15.45:0/2751123935 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fdcac004750 con 0x7fdcc0105e60 2024-04-25T18:50:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.335+0000 7fdcb7400700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdca40a3c30 0x7fdca40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.336+0000 7fdcb7400700 1 -- 172.21.15.45:0/2751123935 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fdcac0cee00 con 0x7fdcc0105e60 2024-04-25T18:50:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.336+0000 7fdcbea00700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdca40a3c30 0x7fdca40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.337+0000 7fdcbea00700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdca40a3c30 0x7fdca40a60f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b2e0 tx=0x7fdcb0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:02.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.339+0000 7fdcb7400700 1 -- 172.21.15.45:0/2751123935 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdcac0d3070 con 0x7fdcc0105e60 2024-04-25T18:50:02.419 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41547]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:50:02.420 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41547]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.420 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41547]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.420 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41547]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.420 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41550]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:50:02.420 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41550]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.420 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41550]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.420 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41550]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:02.588+0000 7fdcc5600700 1 -- 172.21.15.45:0/2751123935 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdcc0062e80 con 0x7fdca40a3c30 2024-04-25T18:50:02.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44121]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:50:02.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44121]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44121]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44121]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44144]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:50:02.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44144]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44144]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.591 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 sudo[44144]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.684 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42233]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:50:02.684 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42233]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.684 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42233]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.684 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42233]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.684 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42236]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:50:02.684 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42236]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.684 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42236]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.684 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42236]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42239]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:50:02.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42239]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42239]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42239]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42242]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:50:02.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42242]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42242]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 sudo[42242]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.788 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41553]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T18:50:02.788 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41553]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.788 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41553]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.789 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41553]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.789 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41556]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T18:50:02.789 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41556]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T18:50:02.789 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41556]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T18:50:02.789 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 sudo[41556]: pam_unix(sudo:session): session closed for user root 2024-04-25T18:50:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:50:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T18:50:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:50:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:50:02.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:50:02.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:02 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:03.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:50:03.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:50:03.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:50:03.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:50:03.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:03.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:50:03.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:50:03.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:50:03.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:50:03.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:50:03.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:02 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:03.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:03.120+0000 7fdcb7400700 1 -- 172.21.15.45:0/2751123935 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdcac004a40 con 0x7fdcc0105e60 2024-04-25T18:50:03.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T18:50:03.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T18:50:03.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:03.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:03.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:50:03.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:50:03.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:50:03.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T18:50:03.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:50:03.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:50:03.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:02 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:03.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:03 smithi045 ceph-mon[27776]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:03.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:03 smithi045 ceph-mon[27776]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:03.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:03 smithi045 ceph-mon[27776]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T18:50:03.908 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:03 smithi045 ceph-mon[27776]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T18:50:04.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:03 smithi062 ceph-mon[32844]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:04.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:03 smithi062 ceph-mon[32844]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:04.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:03 smithi062 ceph-mon[32844]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T18:50:04.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:03 smithi062 ceph-mon[32844]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T18:50:04.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:03 smithi079 ceph-mon[33002]: pgmap v94: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:03 smithi079 ceph-mon[33002]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:03 smithi079 ceph-mon[33002]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T18:50:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:03 smithi079 ceph-mon[33002]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T18:50:06.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:05 smithi062 ceph-mon[32844]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:06.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:05 smithi079 ceph-mon[33002]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:06.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:05 smithi045 ceph-mon[27776]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:07.096 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:06 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/863507132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cab3d66-da4b-4960-bf80-8d198d0eac8d"}]: dispatch 2024-04-25T18:50:07.096 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:06 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cab3d66-da4b-4960-bf80-8d198d0eac8d"}]: dispatch 2024-04-25T18:50:07.096 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:06 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cab3d66-da4b-4960-bf80-8d198d0eac8d"}]': finished 2024-04-25T18:50:07.096 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:06 smithi045 ceph-mon[27776]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T18:50:07.096 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:07.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:06 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/863507132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cab3d66-da4b-4960-bf80-8d198d0eac8d"}]: dispatch 2024-04-25T18:50:07.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:06 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cab3d66-da4b-4960-bf80-8d198d0eac8d"}]: dispatch 2024-04-25T18:50:07.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:06 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cab3d66-da4b-4960-bf80-8d198d0eac8d"}]': finished 2024-04-25T18:50:07.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:06 smithi062 ceph-mon[32844]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T18:50:07.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:07.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:06 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/863507132' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cab3d66-da4b-4960-bf80-8d198d0eac8d"}]: dispatch 2024-04-25T18:50:07.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:06 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2cab3d66-da4b-4960-bf80-8d198d0eac8d"}]: dispatch 2024-04-25T18:50:07.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:06 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2cab3d66-da4b-4960-bf80-8d198d0eac8d"}]': finished 2024-04-25T18:50:07.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:06 smithi079 ceph-mon[33002]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T18:50:07.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:08.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:07 smithi062 ceph-mon[32844]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:08.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:07 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2177237706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:50:08.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:07 smithi079 ceph-mon[33002]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:08.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:07 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2177237706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:50:08.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:07 smithi045 ceph-mon[27776]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:08.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:07 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2177237706' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:50:10.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:09 smithi062 ceph-mon[32844]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:10.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:09 smithi079 ceph-mon[33002]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:10.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:09 smithi045 ceph-mon[27776]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:11.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:11 smithi045 ceph-mon[27776]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:12.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:11 smithi062 ceph-mon[32844]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:12.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:11 smithi079 ceph-mon[33002]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 99 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:14.111 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:13 smithi045 ceph-mon[27776]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:14.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:13 smithi062 ceph-mon[32844]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:14.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:13 smithi079 ceph-mon[33002]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:15.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:15 smithi045 ceph-mon[27776]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:15.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T18:50:15.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:16.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:15 smithi062 ceph-mon[32844]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:16.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T18:50:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:15 smithi079 ceph-mon[33002]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T18:50:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:17.094 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:16 smithi045 ceph-mon[27776]: Deploying daemon osd.3 on smithi045 2024-04-25T18:50:17.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:16 smithi062 ceph-mon[32844]: Deploying daemon osd.3 on smithi045 2024-04-25T18:50:17.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:16 smithi079 ceph-mon[33002]: Deploying daemon osd.3 on smithi045 2024-04-25T18:50:18.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:17 smithi062 ceph-mon[32844]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:18.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:17 smithi079 ceph-mon[33002]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:18.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:17 smithi045 ceph-mon[27776]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:20.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:19 smithi062 ceph-mon[32844]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:20.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:19 smithi079 ceph-mon[33002]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:20.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:19 smithi045 ceph-mon[27776]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:21 smithi062 ceph-mon[32844]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:22.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:21 smithi079 ceph-mon[33002]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:22.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:21 smithi045 ceph-mon[27776]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:24.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:23 smithi062 ceph-mon[32844]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:24.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:23 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:24.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:23 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:24.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:23 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:24.150 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:23 smithi045 ceph-mon[27776]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:24.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:23 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:24.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:23 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:24.151 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:23 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:24.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:23 smithi079 ceph-mon[33002]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:24.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:23 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:24.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:23 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:24.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:23 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:25.895 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:25 smithi045 ceph-mon[27776]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:26.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:25 smithi079 ceph-mon[33002]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:26.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:25 smithi062 ceph-mon[32844]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:28.138 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:27 smithi045 ceph-mon[27776]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:28.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:28.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:28.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:28.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:28.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:28.139 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:27 smithi045 ceph-mon[27776]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:50:28.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:27 smithi079 ceph-mon[33002]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:28.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:28.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:28.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:28.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:28.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:28.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:27 smithi079 ceph-mon[33002]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:50:28.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.239+0000 7fdcb7400700 1 -- 172.21.15.45:0/2751123935 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdcc0062e80 con 0x7fdca40a3c30 2024-04-25T18:50:28.241 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 3 on host 'smithi045' 2024-04-25T18:50:28.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 -- 172.21.15.45:0/2751123935 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdca40a3c30 msgr2=0x7fdca40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdca40a3c30 0x7fdca40a60f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fdcb000b2e0 tx=0x7fdcb0005fb0 comp rx=0 tx=0).stop 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 -- 172.21.15.45:0/2751123935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc0105e60 msgr2=0x7fdcc019b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc0105e60 0x7fdcc019b400 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00e9f0 tx=0x7fdcac00ed00 comp rx=0 tx=0).stop 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 -- 172.21.15.45:0/2751123935 shutdown_connections 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdca40a3c30 0x7fdca40a60f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdcc0108790 0x7fdcc01a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdcc0103530 0x7fdcc019aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 --2- 172.21.15.45:0/2751123935 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdcc0105e60 0x7fdcc019b400 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 -- 172.21.15.45:0/2751123935 >> 172.21.15.45:0/2751123935 conn(0x7fdcc00fcf40 msgr2=0x7fdcc0102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.243+0000 7fdcc5600700 1 -- 172.21.15.45:0/2751123935 shutdown_connections 2024-04-25T18:50:28.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:50:28.244+0000 7fdcc5600700 1 -- 172.21.15.45:0/2751123935 wait complete. 2024-04-25T18:50:28.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:27 smithi062 ceph-mon[32844]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:28.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:28.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:28.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:28.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:28.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:28.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:27 smithi062 ceph-mon[32844]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T18:50:29.004 DEBUG:teuthology.orchestra.run.smithi045:osd.3> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T18:50:29.009 INFO:tasks.cephadm:Deploying osd.4 on smithi062 with /dev/vg_nvme/lv_3... 2024-04-25T18:50:29.009 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T18:50:29.265 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:50:29.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:29 smithi062 ceph-mon[32844]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:29.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:29 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:29.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:29 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:29.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:29 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:29.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:29 smithi062 ceph-mon[32844]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T18:50:29.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:29 smithi062 ceph-mon[32844]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T18:50:29.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:29 smithi062 ceph-mon[32844]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:50:29.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:29 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:29.481 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:29 smithi045 ceph-mon[27776]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:29.481 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:29 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:29.481 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:29 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:29.481 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:29 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:29.481 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:29 smithi045 ceph-mon[27776]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T18:50:29.481 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:29 smithi045 ceph-mon[27776]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T18:50:29.482 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:29 smithi045 ceph-mon[27776]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:50:29.482 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:29 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:29.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:29 smithi079 ceph-mon[33002]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:29.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:29 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:29.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:29 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:29.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:29 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:29.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:29 smithi079 ceph-mon[33002]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T18:50:29.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:29 smithi079 ceph-mon[33002]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T18:50:29.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:29 smithi079 ceph-mon[33002]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:50:29.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:29 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:30.550 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:30 smithi062 ceph-mon[32844]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:50:30.550 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:30 smithi062 ceph-mon[32844]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T18:50:30.550 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:30 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:30.550 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:30 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:30.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:30 smithi045 ceph-mon[27776]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:50:30.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:30 smithi045 ceph-mon[27776]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T18:50:30.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:30 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:30.644 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:30 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:30.645 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 18:50:30 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3[46402]: 2024-04-25T18:50:30.528+0000 7f0f90a00700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T18:50:30.645 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 18:50:30 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3[46402]: 2024-04-25T18:50:30.553+0000 7f0f86800700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:50:30.699 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:30 smithi079 ceph-mon[33002]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:50:30.699 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:30 smithi079 ceph-mon[33002]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T18:50:30.699 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:30 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:30.700 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:30 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:31.099 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T18:50:31.100 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T18:50:31.100 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-25T18:50:31.100 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-25T18:50:31.100 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112317 s, 934 MB/s 2024-04-25T18:50:31.100 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-25T18:50:31.684 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:31 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:50:31.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:31 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:50:31.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:31 smithi062 ceph-mon[32844]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:31.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:31 smithi062 ceph-mon[32844]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T18:50:31.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:31 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:31.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:31 smithi062 ceph-mon[32844]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' 2024-04-25T18:50:31.712 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_3 2024-04-25T18:50:31.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:31 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:50:31.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:31 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:50:31.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:31 smithi045 ceph-mon[27776]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:31.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:31 smithi045 ceph-mon[27776]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T18:50:31.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:31 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:31.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:31 smithi045 ceph-mon[27776]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' 2024-04-25T18:50:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:31 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:50:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:31 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:50:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:31 smithi079 ceph-mon[33002]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:31 smithi079 ceph-mon[33002]: osdmap e26: 4 total, 3 up, 4 in 2024-04-25T18:50:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:31 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:31.871 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:31 smithi079 ceph-mon[33002]: from='osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121]' entity='osd.3' 2024-04-25T18:50:31.959 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:50:32.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:32 smithi062 ceph-mon[32844]: osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] boot 2024-04-25T18:50:32.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:32 smithi062 ceph-mon[32844]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T18:50:32.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:32.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:32 smithi062 ceph-mon[32844]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T18:50:32.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:32 smithi045 ceph-mon[27776]: osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] boot 2024-04-25T18:50:32.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:32 smithi045 ceph-mon[27776]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T18:50:32.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:32.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:32 smithi045 ceph-mon[27776]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T18:50:32.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:32 smithi079 ceph-mon[33002]: osd.3 [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] boot 2024-04-25T18:50:32.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:32 smithi079 ceph-mon[33002]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T18:50:32.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:50:32.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:32 smithi079 ceph-mon[33002]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T18:50:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.408+0000 7f08c963a700 1 -- 172.21.15.62:0/4134128950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c4102ac0 msgr2=0x7f08c4102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.408+0000 7f08c963a700 1 --2- 172.21.15.62:0/4134128950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c4102ac0 0x7f08c4102ee0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f08b40099c0 tx=0x7f08b4009cd0 comp rx=0 tx=0).stop 2024-04-25T18:50:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.408+0000 7f08c963a700 1 -- 172.21.15.62:0/4134128950 shutdown_connections 2024-04-25T18:50:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.408+0000 7f08c963a700 1 --2- 172.21.15.62:0/4134128950 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c4102ac0 0x7f08c4102ee0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.408+0000 7f08c963a700 1 --2- 172.21.15.62:0/4134128950 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c4104660 0x7f08c410af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:33.410 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.408+0000 7f08c963a700 1 --2- 172.21.15.62:0/4134128950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08c4103cc0 0x7f08c4104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.408+0000 7f08c963a700 1 -- 172.21.15.62:0/4134128950 >> 172.21.15.62:0/4134128950 conn(0x7f08c40fe060 msgr2=0x7f08c41004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.409+0000 7f08c963a700 1 -- 172.21.15.62:0/4134128950 shutdown_connections 2024-04-25T18:50:33.411 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.409+0000 7f08c963a700 1 -- 172.21.15.62:0/4134128950 wait complete. 2024-04-25T18:50:33.412 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.409+0000 7f08c963a700 1 Processor -- start 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.409+0000 7f08c963a700 1 -- start start 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c963a700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c4102ac0 0x7f08c4198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c963a700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c4103cc0 0x7f08c4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c963a700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08c4104660 0x7f08c41a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c963a700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f08c410e620 con 0x7f08c4102ac0 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c963a700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f08c410e4a0 con 0x7f08c4103cc0 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c963a700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f08c410e7a0 con 0x7f08c4104660 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c4103cc0 0x7f08c4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c27fc700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c4103cc0 0x7f08c4199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:42908/0 (socket says 172.21.15.62:42908) 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c27fc700 1 -- 172.21.15.62:0/2562593420 learned_addr learned my addr 172.21.15.62:0/2562593420 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c37fe700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08c4104660 0x7f08c41a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c27fc700 1 -- 172.21.15.62:0/2562593420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08c4104660 msgr2=0x7f08c41a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:33.413 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c27fc700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08c4104660 0x7f08c41a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c27fc700 1 -- 172.21.15.62:0/2562593420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c4102ac0 msgr2=0x7f08c4198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c2ffd700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c4102ac0 0x7f08c4198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c27fc700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c4102ac0 0x7f08c4198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c27fc700 1 -- 172.21.15.62:0/2562593420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c41a0930 con 0x7f08c4103cc0 2024-04-25T18:50:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.410+0000 7f08c27fc700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c4103cc0 0x7f08c4199220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f08b800d980 tx=0x7f08b800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.411+0000 7f08abfff700 1 -- 172.21.15.62:0/2562593420 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8017070 con 0x7f08c4103cc0 2024-04-25T18:50:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.411+0000 7f08abfff700 1 -- 172.21.15.62:0/2562593420 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b8005020 con 0x7f08c4103cc0 2024-04-25T18:50:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.411+0000 7f08abfff700 1 -- 172.21.15.62:0/2562593420 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8009b50 con 0x7f08c4103cc0 2024-04-25T18:50:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.411+0000 7f08c963a700 1 -- 172.21.15.62:0/2562593420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c41a0c20 con 0x7f08c4103cc0 2024-04-25T18:50:33.414 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.411+0000 7f08c963a700 1 -- 172.21.15.62:0/2562593420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c41a1110 con 0x7f08c4103cc0 2024-04-25T18:50:33.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.412+0000 7f08c963a700 1 -- 172.21.15.62:0/2562593420 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c4069730 con 0x7f08c4103cc0 2024-04-25T18:50:33.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.413+0000 7f08abfff700 1 -- 172.21.15.62:0/2562593420 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f08b80051d0 con 0x7f08c4103cc0 2024-04-25T18:50:33.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.414+0000 7f08abfff700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f08ac0a3cc0 0x7f08ac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:33.415 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.414+0000 7f08abfff700 1 -- 172.21.15.62:0/2562593420 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f08b80cf0d0 con 0x7f08c4103cc0 2024-04-25T18:50:33.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.414+0000 7f08c2ffd700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f08ac0a3cc0 0x7f08ac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:33.416 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.415+0000 7f08c2ffd700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f08ac0a3cc0 0x7f08ac0a6180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08b4006010 tx=0x7f08b4000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:33.420 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.419+0000 7f08abfff700 1 -- 172.21.15.62:0/2562593420 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08b8020020 con 0x7f08c4103cc0 2024-04-25T18:50:33.639 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:33.638+0000 7f08c963a700 1 -- 172.21.15.62:0/2562593420 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f08c4062e80 con 0x7f08ac0a3cc0 2024-04-25T18:50:33.639 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:33 smithi062 ceph-mon[32844]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:33.639 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:33 smithi062 ceph-mon[32844]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T18:50:33.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:33 smithi045 ceph-mon[27776]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:33.672 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:33 smithi045 ceph-mon[27776]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T18:50:33.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:33 smithi079 ceph-mon[33002]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 98 MiB used, 268 GiB / 268 GiB avail 2024-04-25T18:50:33.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:33 smithi079 ceph-mon[33002]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T18:50:34.547 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:34.547 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:50:34.547 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:50:34.547 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:34.547 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi045 2024-04-25T18:50:34.547 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.547 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.547 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:34.548 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:34.548 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi045 to 8547M 2024-04-25T18:50:34.548 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.548 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:34.548 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:34.548 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:34 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:34.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:50:34.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:50:34.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:34.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi045 2024-04-25T18:50:34.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:34.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:34.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi045 to 8547M 2024-04-25T18:50:34.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:34.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:34.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:34 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='client.24224 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi045 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi045 to 8547M 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:34.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:34 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:35.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:35 smithi062 ceph-mon[32844]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:35.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:35 smithi062 ceph-mon[32844]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T18:50:35.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:35 smithi079 ceph-mon[33002]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:35.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:35 smithi079 ceph-mon[33002]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T18:50:35.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:35 smithi045 ceph-mon[27776]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 132 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:35.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:35 smithi045 ceph-mon[27776]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T18:50:36.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:36 smithi062 ceph-mon[32844]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T18:50:36.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:36 smithi062 ceph-mon[32844]: from='client.? 172.21.15.62:0/4097705839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac287cf8-20dd-477f-8877-966a1d4e269b"}]: dispatch 2024-04-25T18:50:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:36 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac287cf8-20dd-477f-8877-966a1d4e269b"}]: dispatch 2024-04-25T18:50:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:36 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac287cf8-20dd-477f-8877-966a1d4e269b"}]': finished 2024-04-25T18:50:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:36 smithi062 ceph-mon[32844]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T18:50:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:36 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:36.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:36 smithi079 ceph-mon[33002]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T18:50:36.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:36 smithi079 ceph-mon[33002]: from='client.? 172.21.15.62:0/4097705839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac287cf8-20dd-477f-8877-966a1d4e269b"}]: dispatch 2024-04-25T18:50:36.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:36 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac287cf8-20dd-477f-8877-966a1d4e269b"}]: dispatch 2024-04-25T18:50:36.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:36 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac287cf8-20dd-477f-8877-966a1d4e269b"}]': finished 2024-04-25T18:50:36.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:36 smithi079 ceph-mon[33002]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T18:50:36.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:36 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:36.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:36 smithi045 ceph-mon[27776]: osdmap e31: 4 total, 4 up, 4 in 2024-04-25T18:50:36.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:36 smithi045 ceph-mon[27776]: from='client.? 172.21.15.62:0/4097705839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac287cf8-20dd-477f-8877-966a1d4e269b"}]: dispatch 2024-04-25T18:50:36.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:36 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac287cf8-20dd-477f-8877-966a1d4e269b"}]: dispatch 2024-04-25T18:50:36.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:36 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac287cf8-20dd-477f-8877-966a1d4e269b"}]': finished 2024-04-25T18:50:36.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:36 smithi045 ceph-mon[27776]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T18:50:36.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:36 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:37.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:37 smithi062 ceph-mon[32844]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T18:50:37.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:37 smithi062 ceph-mon[32844]: from='client.? 172.21.15.62:0/1033042626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:50:37.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:37 smithi079 ceph-mon[33002]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T18:50:37.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:37 smithi079 ceph-mon[33002]: from='client.? 172.21.15.62:0/1033042626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:50:37.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:37 smithi045 ceph-mon[27776]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T18:50:37.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:37 smithi045 ceph-mon[27776]: from='client.? 172.21.15.62:0/1033042626' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:50:39.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:39 smithi062 ceph-mon[32844]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-25T18:50:39.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:39 smithi079 ceph-mon[33002]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-25T18:50:39.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:39 smithi045 ceph-mon[27776]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-04-25T18:50:41.601 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:41 smithi062 ceph-mon[32844]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:50:41.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:41 smithi079 ceph-mon[33002]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:50:41.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:41 smithi045 ceph-mon[27776]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T18:50:43.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:43 smithi062 ceph-mon[32844]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T18:50:43.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:43 smithi079 ceph-mon[33002]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T18:50:43.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:43 smithi045 ceph-mon[27776]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-04-25T18:50:44.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T18:50:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:44 smithi062 ceph-mon[32844]: Deploying daemon osd.4 on smithi062 2024-04-25T18:50:44.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T18:50:44.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:44.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:44 smithi079 ceph-mon[33002]: Deploying daemon osd.4 on smithi062 2024-04-25T18:50:44.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T18:50:44.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:44.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:44 smithi045 ceph-mon[27776]: Deploying daemon osd.4 on smithi062 2024-04-25T18:50:45.858 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:45 smithi062 ceph-mon[32844]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:45.865 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:45 smithi045 ceph-mon[27776]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:45.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:45 smithi079 ceph-mon[33002]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:47 smithi062 ceph-mon[32844]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:47.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:47 smithi079 ceph-mon[33002]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:47.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:47 smithi045 ceph-mon[27776]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:50.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:49 smithi062 ceph-mon[32844]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:50.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:50.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:50.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:50.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:49 smithi079 ceph-mon[33002]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:50.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:50.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:50.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:50.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:49 smithi045 ceph-mon[27776]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:50.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:50.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:50.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:52.018 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:51 smithi062 ceph-mon[32844]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:52.079 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 4 on host 'smithi062' 2024-04-25T18:50:52.080 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.077+0000 7f08abfff700 1 -- 172.21.15.62:0/2562593420 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f08c4062e80 con 0x7f08ac0a3cc0 2024-04-25T18:50:52.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 -- 172.21.15.62:0/2562593420 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f08ac0a3cc0 msgr2=0x7f08ac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:52.081 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f08ac0a3cc0 0x7f08ac0a6180 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08b4006010 tx=0x7f08b4000bc0 comp rx=0 tx=0).stop 2024-04-25T18:50:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 -- 172.21.15.62:0/2562593420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c4103cc0 msgr2=0x7f08c4199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c4103cc0 0x7f08c4199220 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f08b800d980 tx=0x7f08b800dc90 comp rx=0 tx=0).stop 2024-04-25T18:50:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 -- 172.21.15.62:0/2562593420 shutdown_connections 2024-04-25T18:50:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f08ac0a3cc0 0x7f08ac0a6180 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f08c4104660 0x7f08c41a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f08c4102ac0 0x7f08c4198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 --2- 172.21.15.62:0/2562593420 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f08c4103cc0 0x7f08c4199220 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 -- 172.21.15.62:0/2562593420 >> 172.21.15.62:0/2562593420 conn(0x7f08c40fe060 msgr2=0x7f08c4107030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 -- 172.21.15.62:0/2562593420 shutdown_connections 2024-04-25T18:50:52.082 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:50:52.080+0000 7f08c963a700 1 -- 172.21.15.62:0/2562593420 wait complete. 2024-04-25T18:50:52.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:51 smithi079 ceph-mon[33002]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:52.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:51 smithi045 ceph-mon[27776]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:52.734 DEBUG:teuthology.orchestra.run.smithi062:osd.4> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T18:50:52.736 INFO:tasks.cephadm:Deploying osd.5 on smithi079 with /dev/vg_nvme/lv_3... 2024-04-25T18:50:52.737 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T18:50:52.980 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:50:53.291 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.292 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.292 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.292 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.292 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:53 smithi062 ceph-mon[32844]: from='osd.4 [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:50:53.292 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:53 smithi062 ceph-mon[32844]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:50:53.358 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.358 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.358 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.358 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.359 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:53 smithi079 ceph-mon[33002]: from='osd.4 [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:50:53.359 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:53 smithi079 ceph-mon[33002]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:50:53.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:53.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:53 smithi045 ceph-mon[27776]: from='osd.4 [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:50:53.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:53 smithi045 ceph-mon[27776]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T18:50:54.352 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:54 smithi079 ceph-mon[33002]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:54.352 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:54 smithi079 ceph-mon[33002]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T18:50:54.353 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:54 smithi079 ceph-mon[33002]: from='osd.4 [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:50:54.353 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:54 smithi079 ceph-mon[33002]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T18:50:54.353 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:54 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:54.353 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:54 smithi079 ceph-mon[33002]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:50:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:54 smithi062 ceph-mon[32844]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:54 smithi062 ceph-mon[32844]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T18:50:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:54 smithi062 ceph-mon[32844]: from='osd.4 [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:50:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:54 smithi062 ceph-mon[32844]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T18:50:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:54 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:54 smithi062 ceph-mon[32844]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:50:54.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:54 smithi045 ceph-mon[27776]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:54 smithi045 ceph-mon[27776]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T18:50:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:54 smithi045 ceph-mon[27776]: from='osd.4 [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:50:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:54 smithi045 ceph-mon[27776]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T18:50:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:54 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:54 smithi045 ceph-mon[27776]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:50:54.863 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 18:50:54 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4[44345]: 2024-04-25T18:50:54.368+0000 7f48a2838700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T18:50:54.864 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 18:50:54 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4[44345]: 2024-04-25T18:50:54.375+0000 7f489e62b700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:50:54.913 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T18:50:54.913 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T18:50:54.913 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-04-25T18:50:54.913 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-04-25T18:50:54.913 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125335 s, 837 MB/s 2024-04-25T18:50:54.913 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-04-25T18:50:55.477 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:55 smithi079 ceph-mon[33002]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:55.477 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:55 smithi079 ceph-mon[33002]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:50:55.477 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:55 smithi079 ceph-mon[33002]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T18:50:55.477 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:55 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:55.477 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:55 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:55.505 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_3 2024-04-25T18:50:55.685 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:55 smithi062 ceph-mon[32844]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:55.725 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:55 smithi062 ceph-mon[32844]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:50:55.725 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:55 smithi062 ceph-mon[32844]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T18:50:55.725 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:55 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:55.725 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:55 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:55.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:55 smithi045 ceph-mon[27776]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 357 GiB / 358 GiB avail 2024-04-25T18:50:55.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:55 smithi045 ceph-mon[27776]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:50:55.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:55 smithi045 ceph-mon[27776]: osdmap e34: 5 total, 4 up, 5 in 2024-04-25T18:50:55.726 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:55 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:55.727 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:55 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:55.745 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:50:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:50:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:50:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: osd.4 [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] boot 2024-04-25T18:50:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T18:50:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi062 2024-04-25T18:50:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:56.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:56.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi062 to 8544M 2024-04-25T18:50:56.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:56.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:56.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:56.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:56.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:56.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:56 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:50:56.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:50:56.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: osd.4 [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] boot 2024-04-25T18:50:56.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T18:50:56.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:56.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi062 2024-04-25T18:50:56.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi062 to 8544M 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:56.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:56 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:50:56.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:50:56.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: osd.4 [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] boot 2024-04-25T18:50:56.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T18:50:56.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:50:56.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi062 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi062 to 8544M 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:50:56.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:56 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:50:57.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.230+0000 7f2314e34700 1 -- 172.21.15.79:0/1425346246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2310101340 msgr2=0x7f2310101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:57.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.230+0000 7f2314e34700 1 --2- 172.21.15.79:0/1425346246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2310101340 0x7f2310101760 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2300009990 tx=0x7f2300009ca0 comp rx=0 tx=0).stop 2024-04-25T18:50:57.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.231+0000 7f2314e34700 1 -- 172.21.15.79:0/1425346246 shutdown_connections 2024-04-25T18:50:57.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.231+0000 7f2314e34700 1 --2- 172.21.15.79:0/1425346246 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310102660 0x7f2310102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:57.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.231+0000 7f2314e34700 1 --2- 172.21.15.79:0/1425346246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2310101340 0x7f2310101760 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:57.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.231+0000 7f2314e34700 1 --2- 172.21.15.79:0/1425346246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2310101ca0 0x7f2310102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:57.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.231+0000 7f2314e34700 1 -- 172.21.15.79:0/1425346246 >> 172.21.15.79:0/1425346246 conn(0x7f23100fcfa0 msgr2=0x7f23100ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:50:57.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.231+0000 7f2314e34700 1 -- 172.21.15.79:0/1425346246 shutdown_connections 2024-04-25T18:50:57.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.231+0000 7f2314e34700 1 -- 172.21.15.79:0/1425346246 wait complete. 2024-04-25T18:50:57.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.231+0000 7f2314e34700 1 Processor -- start 2024-04-25T18:50:57.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.231+0000 7f2314e34700 1 -- start start 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f2314e34700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2310101340 0x7f231019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f2314e34700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2310101ca0 0x7f231019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f2314e34700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310102660 0x7f23101a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f2314e34700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2310110790 con 0x7f2310101340 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f2314e34700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2310110610 con 0x7f2310102660 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f2314e34700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2310110910 con 0x7f2310101ca0 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f230dd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2310101ca0 0x7f231019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f230dd9b700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2310101ca0 0x7f231019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58658/0 (socket says 172.21.15.79:58658) 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f230dd9b700 1 -- 172.21.15.79:0/1439424972 learned_addr learned my addr 172.21.15.79:0/1439424972 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f230ed9d700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310102660 0x7f23101a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f230dd9b700 1 -- 172.21.15.79:0/1439424972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310102660 msgr2=0x7f23101a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f230dd9b700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310102660 0x7f23101a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f230dd9b700 1 -- 172.21.15.79:0/1439424972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2310101340 msgr2=0x7f231019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:50:57.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f230dd9b700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2310101340 0x7f231019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:50:57.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f230dd9b700 1 -- 172.21.15.79:0/1439424972 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23101a2a10 con 0x7f2310101ca0 2024-04-25T18:50:57.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f230dd9b700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2310101ca0 0x7f231019b2f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f230400b560 tx=0x7f230400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:57.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f22ff7fe700 1 -- 172.21.15.79:0/1439424972 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304015020 con 0x7f2310101ca0 2024-04-25T18:50:57.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f22ff7fe700 1 -- 172.21.15.79:0/1439424972 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2304011ce0 con 0x7f2310101ca0 2024-04-25T18:50:57.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f2314e34700 1 -- 172.21.15.79:0/1439424972 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23101a2d00 con 0x7f2310101ca0 2024-04-25T18:50:57.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f22ff7fe700 1 -- 172.21.15.79:0/1439424972 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304010d20 con 0x7f2310101ca0 2024-04-25T18:50:57.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.232+0000 7f2314e34700 1 -- 172.21.15.79:0/1439424972 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23101a32b0 con 0x7f2310101ca0 2024-04-25T18:50:57.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.233+0000 7f2314e34700 1 -- 172.21.15.79:0/1439424972 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2310069730 con 0x7f2310101ca0 2024-04-25T18:50:57.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.235+0000 7f22ff7fe700 1 -- 172.21.15.79:0/1439424972 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2304018020 con 0x7f2310101ca0 2024-04-25T18:50:57.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.235+0000 7f22ff7fe700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22f80a3cc0 0x7f22f80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:50:57.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.236+0000 7f22ff7fe700 1 -- 172.21.15.79:0/1439424972 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4101+0+0 (secure 0 0 0) 0x7f23040cf080 con 0x7f2310101ca0 2024-04-25T18:50:57.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.236+0000 7f230e59c700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22f80a3cc0 0x7f22f80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:50:57.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.237+0000 7f230e59c700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22f80a3cc0 0x7f22f80a6180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23000098e0 tx=0x7f2300003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:50:57.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.239+0000 7f22ff7fe700 1 -- 172.21.15.79:0/1439424972 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f230401f020 con 0x7f2310101ca0 2024-04-25T18:50:57.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:50:57.444+0000 7f2314e34700 1 -- 172.21.15.79:0/1439424972 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2310062e80 con 0x7f22f80a3cc0 2024-04-25T18:50:57.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:57 smithi079 ceph-mon[33002]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:50:57.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:57 smithi079 ceph-mon[33002]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T18:50:57.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:57 smithi045 ceph-mon[27776]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:50:57.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:57 smithi045 ceph-mon[27776]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T18:50:57.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:57 smithi062 ceph-mon[32844]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:50:57.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:57 smithi062 ceph-mon[32844]: osdmap e36: 5 total, 5 up, 5 in 2024-04-25T18:50:58.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:58 smithi079 ceph-mon[33002]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:50:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:50:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:58.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:58 smithi045 ceph-mon[27776]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:58.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:50:58.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:50:58.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:58.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:58 smithi062 ceph-mon[32844]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:50:58.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:50:58.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:50:58.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:50:59.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:50:59 smithi079 ceph-mon[33002]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:50:59.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:50:59 smithi045 ceph-mon[27776]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:50:59.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:50:59 smithi062 ceph-mon[32844]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:00.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:00 smithi079 ceph-mon[33002]: from='client.? 172.21.15.79:0/3009624113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c258a3e7-ef41-4ad5-a9b0-689eb2761171"}]: dispatch 2024-04-25T18:51:00.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:00 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c258a3e7-ef41-4ad5-a9b0-689eb2761171"}]: dispatch 2024-04-25T18:51:00.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:00 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c258a3e7-ef41-4ad5-a9b0-689eb2761171"}]': finished 2024-04-25T18:51:00.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:00 smithi079 ceph-mon[33002]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T18:51:00.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:00 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:00.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:00 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/3009624113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c258a3e7-ef41-4ad5-a9b0-689eb2761171"}]: dispatch 2024-04-25T18:51:00.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:00 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c258a3e7-ef41-4ad5-a9b0-689eb2761171"}]: dispatch 2024-04-25T18:51:00.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:00 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c258a3e7-ef41-4ad5-a9b0-689eb2761171"}]': finished 2024-04-25T18:51:00.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:00 smithi045 ceph-mon[27776]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T18:51:00.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:00 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:00.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:00 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/3009624113' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c258a3e7-ef41-4ad5-a9b0-689eb2761171"}]: dispatch 2024-04-25T18:51:00.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:00 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c258a3e7-ef41-4ad5-a9b0-689eb2761171"}]: dispatch 2024-04-25T18:51:00.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:00 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c258a3e7-ef41-4ad5-a9b0-689eb2761171"}]': finished 2024-04-25T18:51:00.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:00 smithi062 ceph-mon[32844]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T18:51:00.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:00 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:01.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:01 smithi079 ceph-mon[33002]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:01.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:01 smithi079 ceph-mon[33002]: from='client.? 172.21.15.79:0/179649601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:51:01.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:01 smithi045 ceph-mon[27776]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:01.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:01 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/179649601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:51:01.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:01 smithi062 ceph-mon[32844]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:01.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:01 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/179649601' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:51:03.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:03 smithi079 ceph-mon[33002]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:03.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:03 smithi045 ceph-mon[27776]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:03 smithi062 ceph-mon[32844]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:05.612 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:05 smithi079 ceph-mon[33002]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:05.679 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:05 smithi062 ceph-mon[32844]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:05.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:05 smithi045 ceph-mon[27776]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 159 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:07.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:07 smithi079 ceph-mon[33002]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:07.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:07 smithi045 ceph-mon[27776]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:07 smithi062 ceph-mon[32844]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:08.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:08 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T18:51:08.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:08 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:08.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:08 smithi079 ceph-mon[33002]: Deploying daemon osd.5 on smithi079 2024-04-25T18:51:08.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:08 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T18:51:08.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:08 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:08.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:08 smithi045 ceph-mon[27776]: Deploying daemon osd.5 on smithi079 2024-04-25T18:51:08.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:08 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T18:51:08.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:08 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:08.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:08 smithi062 ceph-mon[32844]: Deploying daemon osd.5 on smithi079 2024-04-25T18:51:09.592 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:09 smithi079 ceph-mon[33002]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:09.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:09 smithi045 ceph-mon[27776]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:09 smithi062 ceph-mon[32844]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:11.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:11 smithi045 ceph-mon[27776]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:11.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:11 smithi062 ceph-mon[32844]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:11.891 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:11 smithi079 ceph-mon[33002]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:14.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:13 smithi079 ceph-mon[33002]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:14.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:14.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:14.101 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:14.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:13 smithi062 ceph-mon[32844]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:14.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:14.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:14.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:14.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:13 smithi045 ceph-mon[27776]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:16.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:15 smithi045 ceph-mon[27776]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:16.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:16.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:16.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:16.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:16.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:16.094 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:15 smithi079 ceph-mon[33002]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:16.094 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:16.094 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:16.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:16.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:16.095 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:16.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:15 smithi062 ceph-mon[32844]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:16.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:16.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:16.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.534+0000 7f22ff7fe700 1 -- 172.21.15.79:0/1439424972 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2310062e80 con 0x7f22f80a3cc0 2024-04-25T18:51:16.536 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 5 on host 'smithi079' 2024-04-25T18:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.536+0000 7f2314e34700 1 -- 172.21.15.79:0/1439424972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22f80a3cc0 msgr2=0x7f22f80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.536+0000 7f2314e34700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22f80a3cc0 0x7f22f80a6180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f23000098e0 tx=0x7f2300003860 comp rx=0 tx=0).stop 2024-04-25T18:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.536+0000 7f2314e34700 1 -- 172.21.15.79:0/1439424972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2310101ca0 msgr2=0x7f231019b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.536+0000 7f2314e34700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2310101ca0 0x7f231019b2f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f230400b560 tx=0x7f230400b870 comp rx=0 tx=0).stop 2024-04-25T18:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.536+0000 7f2314e34700 1 -- 172.21.15.79:0/1439424972 shutdown_connections 2024-04-25T18:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.536+0000 7f2314e34700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22f80a3cc0 0x7f22f80a6180 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.536+0000 7f2314e34700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2310102660 0x7f23101a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.536+0000 7f2314e34700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2310101340 0x7f231019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.536+0000 7f2314e34700 1 --2- 172.21.15.79:0/1439424972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2310101ca0 0x7f231019b2f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.537+0000 7f2314e34700 1 -- 172.21.15.79:0/1439424972 >> 172.21.15.79:0/1439424972 conn(0x7f23100fcfa0 msgr2=0x7f23100feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.537+0000 7f2314e34700 1 -- 172.21.15.79:0/1439424972 shutdown_connections 2024-04-25T18:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:51:16.537+0000 7f2314e34700 1 -- 172.21.15.79:0/1439424972 wait complete. 2024-04-25T18:51:17.124 DEBUG:teuthology.orchestra.run.smithi079:osd.5> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T18:51:17.127 INFO:tasks.cephadm:Deploying osd.6 on smithi045 with /dev/vg_nvme/lv_2... 2024-04-25T18:51:17.127 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T18:51:17.552 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:51:17.837 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:17 smithi045 ceph-mon[27776]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:17.837 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:17.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:17.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:17.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:17 smithi045 ceph-mon[27776]: from='osd.5 [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:51:17.838 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:17 smithi045 ceph-mon[27776]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:51:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:17 smithi062 ceph-mon[32844]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:17 smithi062 ceph-mon[32844]: from='osd.5 [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:51:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:17 smithi062 ceph-mon[32844]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:51:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:17 smithi079 ceph-mon[33002]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:17 smithi079 ceph-mon[33002]: from='osd.5 [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:51:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:17 smithi079 ceph-mon[33002]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T18:51:18.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:18 smithi062 ceph-mon[32844]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T18:51:18.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:18 smithi062 ceph-mon[32844]: from='osd.5 [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:51:18.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:18 smithi062 ceph-mon[32844]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T18:51:18.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:18 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:18.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:18 smithi062 ceph-mon[32844]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:51:18.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:18 smithi079 ceph-mon[33002]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T18:51:18.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:18 smithi079 ceph-mon[33002]: from='osd.5 [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:51:18.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:18 smithi079 ceph-mon[33002]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T18:51:18.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:18 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:18.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:18 smithi079 ceph-mon[33002]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:51:18.935 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 18:51:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T18:51:18.525+0000 7f115c407700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T18:51:18.935 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 18:51:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T18:51:18.532+0000 7f11581fa700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:51:18.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:18 smithi045 ceph-mon[27776]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T18:51:18.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:18 smithi045 ceph-mon[27776]: from='osd.5 [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:51:18.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:18 smithi045 ceph-mon[27776]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T18:51:18.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:18 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:18.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:18 smithi045 ceph-mon[27776]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:51:19.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:19 smithi045 ceph-mon[27776]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:19.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:19 smithi045 ceph-mon[27776]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:51:19.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:19 smithi045 ceph-mon[27776]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T18:51:19.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:19.647 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:19.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:19 smithi045 ceph-mon[27776]: osd.5 [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] boot 2024-04-25T18:51:19.648 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:19 smithi045 ceph-mon[27776]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T18:51:19.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:19 smithi062 ceph-mon[32844]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:19.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:19 smithi062 ceph-mon[32844]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:51:19.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:19 smithi062 ceph-mon[32844]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T18:51:19.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:19.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:19.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:19 smithi062 ceph-mon[32844]: osd.5 [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] boot 2024-04-25T18:51:19.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:19 smithi062 ceph-mon[32844]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T18:51:19.886 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:19 smithi079 ceph-mon[33002]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 447 GiB / 447 GiB avail 2024-04-25T18:51:19.886 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:19 smithi079 ceph-mon[33002]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:51:19.886 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:19 smithi079 ceph-mon[33002]: osdmap e39: 6 total, 5 up, 6 in 2024-04-25T18:51:19.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:19.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:19.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:19 smithi079 ceph-mon[33002]: osd.5 [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] boot 2024-04-25T18:51:19.887 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:19 smithi079 ceph-mon[33002]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T18:51:20.572 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T18:51:20.572 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T18:51:20.572 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-25T18:51:20.572 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-25T18:51:20.572 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0408495 s, 257 MB/s 2024-04-25T18:51:20.573 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-25T18:51:20.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:51:20.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:51:20.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:20.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi079 2024-04-25T18:51:20.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:20.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:20.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi079 to 8544M 2024-04-25T18:51:20.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:20.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:20.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:20 smithi062 ceph-mon[32844]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi079 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi079 to 8544M 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:20.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:20.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:20 smithi079 ceph-mon[33002]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T18:51:20.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi079 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi079 to 8544M 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:20.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:20 smithi045 ceph-mon[27776]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T18:51:21.607 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-04-25T18:51:22.056 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:51:22.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:22 smithi062 ceph-mon[32844]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:22.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:22 smithi062 ceph-mon[32844]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T18:51:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:22 smithi079 ceph-mon[33002]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:22 smithi079 ceph-mon[33002]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T18:51:22.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:22 smithi045 ceph-mon[27776]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:22.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:22 smithi045 ceph-mon[27776]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T18:51:23.793 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:23 smithi045 ceph-mon[27776]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:23.793 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:23 smithi045 ceph-mon[27776]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T18:51:23.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:23 smithi062 ceph-mon[32844]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:23.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:23 smithi062 ceph-mon[32844]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T18:51:23.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:23 smithi079 ceph-mon[33002]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:23.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:23 smithi079 ceph-mon[33002]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T18:51:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.368+0000 7f7a3fe00700 1 -- 172.21.15.45:0/111011633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a38103540 msgr2=0x7f7a38109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.368+0000 7f7a3fe00700 1 --2- 172.21.15.45:0/111011633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a38103540 0x7f7a38109df0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7a3000b0a0 tx=0x7f7a3000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:51:24.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.368+0000 7f7a3fe00700 1 -- 172.21.15.45:0/111011633 shutdown_connections 2024-04-25T18:51:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.368+0000 7f7a3fe00700 1 --2- 172.21.15.45:0/111011633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a381019a0 0x7f7a38101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.368+0000 7f7a3fe00700 1 --2- 172.21.15.45:0/111011633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a38103540 0x7f7a38109df0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.368+0000 7f7a3fe00700 1 --2- 172.21.15.45:0/111011633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a38102ba0 0x7f7a38103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.368+0000 7f7a3fe00700 1 -- 172.21.15.45:0/111011633 >> 172.21.15.45:0/111011633 conn(0x7f7a380fcf40 msgr2=0x7f7a380ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.368+0000 7f7a3fe00700 1 -- 172.21.15.45:0/111011633 shutdown_connections 2024-04-25T18:51:24.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.369+0000 7f7a3fe00700 1 -- 172.21.15.45:0/111011633 wait complete. 2024-04-25T18:51:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.370+0000 7f7a3fe00700 1 Processor -- start 2024-04-25T18:51:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.370+0000 7f7a3fe00700 1 -- start start 2024-04-25T18:51:24.373 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.370+0000 7f7a3fe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a381019a0 0x7f7a38105a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.370+0000 7f7a3fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a38102ba0 0x7f7a38104120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.370+0000 7f7a3fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a38103540 0x7f7a38104660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.370+0000 7f7a3fe00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810d490 con 0x7f7a38103540 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.370+0000 7f7a3fe00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810d310 con 0x7f7a381019a0 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.370+0000 7f7a3fe00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3810d610 con 0x7f7a38102ba0 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.371+0000 7f7a3d400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a381019a0 0x7f7a38105a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.371+0000 7f7a3d400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a381019a0 0x7f7a38105a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41254/0 (socket says 172.21.15.45:41254) 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.371+0000 7f7a3d400700 1 -- 172.21.15.45:0/1076439853 learned_addr learned my addr 172.21.15.45:0/1076439853 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.371+0000 7f7a3de00700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a38103540 0x7f7a38104660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.371+0000 7f7a3ca00700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a38102ba0 0x7f7a38104120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.371+0000 7f7a3d400700 1 -- 172.21.15.45:0/1076439853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a38102ba0 msgr2=0x7f7a38104120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.371+0000 7f7a3d400700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a38102ba0 0x7f7a38104120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:24.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.371+0000 7f7a3d400700 1 -- 172.21.15.45:0/1076439853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a38103540 msgr2=0x7f7a38104660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:24.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.371+0000 7f7a3d400700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a38103540 0x7f7a38104660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:24.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.371+0000 7f7a3d400700 1 -- 172.21.15.45:0/1076439853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a38104f30 con 0x7f7a381019a0 2024-04-25T18:51:24.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.372+0000 7f7a3d400700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a381019a0 0x7f7a38105a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00e950 tx=0x7f7a2c00ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:24.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.372+0000 7f7a36000700 1 -- 172.21.15.45:0/1076439853 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c004230 con 0x7f7a381019a0 2024-04-25T18:51:24.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.372+0000 7f7a3fe00700 1 -- 172.21.15.45:0/1076439853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a381a2d90 con 0x7f7a381019a0 2024-04-25T18:51:24.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.372+0000 7f7a36000700 1 -- 172.21.15.45:0/1076439853 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a2c0043d0 con 0x7f7a381019a0 2024-04-25T18:51:24.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.372+0000 7f7a36000700 1 -- 172.21.15.45:0/1076439853 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c003d60 con 0x7f7a381019a0 2024-04-25T18:51:24.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.372+0000 7f7a3fe00700 1 -- 172.21.15.45:0/1076439853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a381a3190 con 0x7f7a381019a0 2024-04-25T18:51:24.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.373+0000 7f7a3fe00700 1 -- 172.21.15.45:0/1076439853 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a381a3420 con 0x7f7a381019a0 2024-04-25T18:51:24.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.375+0000 7f7a36000700 1 -- 172.21.15.45:0/1076439853 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7a2c00c7e0 con 0x7f7a381019a0 2024-04-25T18:51:24.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.376+0000 7f7a36000700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a200a3c40 0x7f7a200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:24.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.377+0000 7f7a36000700 1 -- 172.21.15.45:0/1076439853 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 4533+0+0 (secure 0 0 0) 0x7f7a2c0cf880 con 0x7f7a381019a0 2024-04-25T18:51:24.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.377+0000 7f7a3ca00700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a200a3c40 0x7f7a200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:24.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.377+0000 7f7a3ca00700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a200a3c40 0x7f7a200a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009c60 tx=0x7f7a24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:24.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.381+0000 7f7a36000700 1 -- 172.21.15.45:0/1076439853 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a2c09d080 con 0x7f7a381019a0 2024-04-25T18:51:24.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:24.628+0000 7f7a3fe00700 1 -- 172.21.15.45:0/1076439853 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7a3806ae30 con 0x7f7a200a3c40 2024-04-25T18:51:24.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:24 smithi062 ceph-mon[32844]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T18:51:24.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:24 smithi079 ceph-mon[33002]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T18:51:24.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:24 smithi045 ceph-mon[27776]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T18:51:25.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:25 smithi062 ceph-mon[32844]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:25.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:25 smithi062 ceph-mon[32844]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:25.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:25 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:51:25.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:25 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:51:25.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:25 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:25.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:25 smithi045 ceph-mon[27776]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:25.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:25 smithi045 ceph-mon[27776]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:25.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:51:25.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:51:25.904 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:25.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:25 smithi079 ceph-mon[33002]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 186 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:25.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:25 smithi079 ceph-mon[33002]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:25.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:51:25.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:51:25.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:27.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:27 smithi045 ceph-mon[27776]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T18:51:27.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:27 smithi062 ceph-mon[32844]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T18:51:27.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:27 smithi079 ceph-mon[33002]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-04-25T18:51:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:29 smithi062 ceph-mon[32844]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T18:51:29.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:29 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2182652259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b1c752c-c246-4c8c-adee-e8948e1c2978"}]: dispatch 2024-04-25T18:51:29.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:29 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b1c752c-c246-4c8c-adee-e8948e1c2978"}]: dispatch 2024-04-25T18:51:29.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:29 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b1c752c-c246-4c8c-adee-e8948e1c2978"}]': finished 2024-04-25T18:51:29.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:29 smithi062 ceph-mon[32844]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T18:51:29.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:29 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:29.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:29 smithi079 ceph-mon[33002]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T18:51:29.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:29 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2182652259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b1c752c-c246-4c8c-adee-e8948e1c2978"}]: dispatch 2024-04-25T18:51:29.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:29 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b1c752c-c246-4c8c-adee-e8948e1c2978"}]: dispatch 2024-04-25T18:51:29.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:29 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b1c752c-c246-4c8c-adee-e8948e1c2978"}]': finished 2024-04-25T18:51:29.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:29 smithi079 ceph-mon[33002]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T18:51:29.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:29 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:29.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:29 smithi045 ceph-mon[27776]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T18:51:29.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:29 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2182652259' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b1c752c-c246-4c8c-adee-e8948e1c2978"}]: dispatch 2024-04-25T18:51:29.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:29 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9b1c752c-c246-4c8c-adee-e8948e1c2978"}]: dispatch 2024-04-25T18:51:29.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:29 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b1c752c-c246-4c8c-adee-e8948e1c2978"}]': finished 2024-04-25T18:51:29.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:29 smithi045 ceph-mon[27776]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T18:51:29.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:29 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:30.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:30 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1764527106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:51:30.930 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:30 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1764527106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:51:30.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:30 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1764527106' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:51:31.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:31 smithi062 ceph-mon[32844]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:51:31.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:31 smithi079 ceph-mon[33002]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:51:31.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:31 smithi045 ceph-mon[27776]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T18:51:33.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:33 smithi045 ceph-mon[27776]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T18:51:33.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:33 smithi062 ceph-mon[32844]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T18:51:33.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:33 smithi079 ceph-mon[33002]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T18:51:35.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:35 smithi062 ceph-mon[32844]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:51:35.868 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:35 smithi045 ceph-mon[27776]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:51:35.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:35 smithi079 ceph-mon[33002]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:51:37.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:37 smithi062 ceph-mon[32844]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:37.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:37 smithi079 ceph-mon[33002]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:37.936 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:37 smithi045 ceph-mon[27776]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:38.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:38 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T18:51:38.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:38 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:38.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:38 smithi062 ceph-mon[32844]: Deploying daemon osd.6 on smithi045 2024-04-25T18:51:38.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:38 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T18:51:38.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:38 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:38.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:38 smithi079 ceph-mon[33002]: Deploying daemon osd.6 on smithi045 2024-04-25T18:51:38.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:38 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T18:51:38.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:38 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:38.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:38 smithi045 ceph-mon[27776]: Deploying daemon osd.6 on smithi045 2024-04-25T18:51:39.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:39 smithi045 ceph-mon[27776]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:39.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:39 smithi062 ceph-mon[32844]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:39.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:39 smithi079 ceph-mon[33002]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:41.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:41 smithi062 ceph-mon[32844]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:41.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:41 smithi079 ceph-mon[33002]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:41.953 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:41 smithi045 ceph-mon[27776]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:43.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:43 smithi045 ceph-mon[27776]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:43.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:43 smithi062 ceph-mon[32844]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:43.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:43 smithi079 ceph-mon[33002]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:45 smithi062 ceph-mon[32844]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:45.866 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:45 smithi045 ceph-mon[27776]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:45.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:45 smithi079 ceph-mon[33002]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:47.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:47 smithi079 ceph-mon[33002]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:47.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:47 smithi045 ceph-mon[27776]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:47.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:47.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:47.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:47.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:47 smithi062 ceph-mon[32844]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:49.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:49 smithi045 ceph-mon[27776]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:49.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:49 smithi079 ceph-mon[33002]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:49.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:49 smithi062 ceph-mon[32844]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:51.534 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 6 on host 'smithi045' 2024-04-25T18:51:51.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.532+0000 7f7a36000700 1 -- 172.21.15.45:0/1076439853 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7a3806ae30 con 0x7f7a200a3c40 2024-04-25T18:51:51.534 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:51 smithi045 ceph-mon[27776]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:51.535 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:51 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:51.535 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:51 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:51.535 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:51 smithi045 ceph-mon[27776]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:51:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 -- 172.21.15.45:0/1076439853 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a200a3c40 msgr2=0x7f7a200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a200a3c40 0x7f7a200a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a24009c60 tx=0x7f7a24009260 comp rx=0 tx=0).stop 2024-04-25T18:51:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 -- 172.21.15.45:0/1076439853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a381019a0 msgr2=0x7f7a38105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a381019a0 0x7f7a38105a80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c00e950 tx=0x7f7a2c00ec60 comp rx=0 tx=0).stop 2024-04-25T18:51:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 -- 172.21.15.45:0/1076439853 shutdown_connections 2024-04-25T18:51:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a200a3c40 0x7f7a200a6100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:51.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a381019a0 0x7f7a38105a80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:51.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a38103540 0x7f7a38104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:51.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 --2- 172.21.15.45:0/1076439853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a38102ba0 0x7f7a38104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:51.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 -- 172.21.15.45:0/1076439853 >> 172.21.15.45:0/1076439853 conn(0x7f7a380fcf40 msgr2=0x7f7a3806a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:51.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 -- 172.21.15.45:0/1076439853 shutdown_connections 2024-04-25T18:51:51.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:51:51.536+0000 7f7a3fe00700 1 -- 172.21.15.45:0/1076439853 wait complete. 2024-04-25T18:51:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:51 smithi062 ceph-mon[32844]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:51.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:51 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:51.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:51 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:51.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:51 smithi062 ceph-mon[32844]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:51:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:51 smithi079 ceph-mon[33002]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:51 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:51 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:51 smithi079 ceph-mon[33002]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T18:51:52.263 DEBUG:teuthology.orchestra.run.smithi045:osd.6> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T18:51:52.274 INFO:tasks.cephadm:Deploying osd.7 on smithi062 with /dev/vg_nvme/lv_2... 2024-04-25T18:51:52.275 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T18:51:52.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:52 smithi045 ceph-mon[27776]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T18:51:52.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:52 smithi045 ceph-mon[27776]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T18:51:52.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:52 smithi045 ceph-mon[27776]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:51:52.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:52 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:52.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:52 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:52.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:52 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:52.528 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:51:52.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:52 smithi062 ceph-mon[32844]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T18:51:52.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:52 smithi062 ceph-mon[32844]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T18:51:52.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:52 smithi062 ceph-mon[32844]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:51:52.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:52 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:52.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:52 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:52.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:52 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:52.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:52 smithi079 ceph-mon[33002]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T18:51:52.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:52 smithi079 ceph-mon[33002]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T18:51:52.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:52 smithi079 ceph-mon[33002]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:51:52.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:52 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:52.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:52 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:52.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:52 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:53.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:53 smithi045 ceph-mon[27776]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:53.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:53 smithi045 ceph-mon[27776]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:51:53.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:53 smithi045 ceph-mon[27776]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T18:51:53.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:53.509 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:53.559 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:53 smithi062 ceph-mon[32844]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:53.559 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:53 smithi062 ceph-mon[32844]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:51:53.559 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:53 smithi062 ceph-mon[32844]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T18:51:53.559 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:53.559 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:53.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:53 smithi079 ceph-mon[33002]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:53.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:53 smithi079 ceph-mon[33002]: from='osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:51:53.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:53 smithi079 ceph-mon[33002]: osdmap e47: 7 total, 6 up, 7 in 2024-04-25T18:51:53.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:53.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:54.237 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T18:51:54.237 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T18:51:54.237 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-25T18:51:54.237 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-25T18:51:54.237 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129908 s, 807 MB/s 2024-04-25T18:51:54.237 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-25T18:51:54.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:54 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:51:54.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:54 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:51:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:54 smithi045 ceph-mon[27776]: osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] boot 2024-04-25T18:51:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:54 smithi045 ceph-mon[27776]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T18:51:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:54 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:54.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:54 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:51:54.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:54 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:51:54.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:54 smithi062 ceph-mon[32844]: osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] boot 2024-04-25T18:51:54.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:54 smithi062 ceph-mon[32844]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T18:51:54.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:54 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:54.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:54 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:51:54.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:54 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:51:54.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:54 smithi079 ceph-mon[33002]: osd.6 [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] boot 2024-04-25T18:51:54.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:54 smithi079 ceph-mon[33002]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T18:51:54.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:54 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:51:54.872 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_2 2024-04-25T18:51:55.118 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:51:55.509 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:55 smithi062 ceph-mon[32844]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:55.510 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:55 smithi062 ceph-mon[32844]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T18:51:55.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:55 smithi079 ceph-mon[33002]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:55.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:55 smithi079 ceph-mon[33002]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T18:51:55.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:55 smithi045 ceph-mon[27776]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 536 GiB / 536 GiB avail 2024-04-25T18:51:55.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:55 smithi045 ceph-mon[27776]: osdmap e49: 7 total, 7 up, 7 in 2024-04-25T18:51:56.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.695+0000 7f14ad351700 1 -- 172.21.15.62:0/3554416793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a8102aa0 msgr2=0x7f14a8102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:56.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.695+0000 7f14ad351700 1 --2- 172.21.15.62:0/3554416793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a8102aa0 0x7f14a8102ec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f14a000b0d0 tx=0x7f14a000b3e0 comp rx=0 tx=0).stop 2024-04-25T18:51:56.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.695+0000 7f14ad351700 1 -- 172.21.15.62:0/3554416793 shutdown_connections 2024-04-25T18:51:56.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.695+0000 7f14ad351700 1 --2- 172.21.15.62:0/3554416793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a8102aa0 0x7f14a8102ec0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:56.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.695+0000 7f14ad351700 1 --2- 172.21.15.62:0/3554416793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14a8104640 0x7f14a810aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:56.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.695+0000 7f14ad351700 1 --2- 172.21.15.62:0/3554416793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14a8103ca0 0x7f14a8104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:56.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.695+0000 7f14ad351700 1 -- 172.21.15.62:0/3554416793 >> 172.21.15.62:0/3554416793 conn(0x7f14a80fe040 msgr2=0x7f14a8100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:51:56.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.695+0000 7f14ad351700 1 -- 172.21.15.62:0/3554416793 shutdown_connections 2024-04-25T18:51:56.699 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.695+0000 7f14ad351700 1 -- 172.21.15.62:0/3554416793 wait complete. 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.695+0000 7f14ad351700 1 Processor -- start 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14ad351700 1 -- start start 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14ad351700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a8102aa0 0x7f14a8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14ad351700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14a8103ca0 0x7f14a8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14ad351700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14a8104640 0x7f14a81a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14ad351700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f14a810e590 con 0x7f14a8104640 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14ad351700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f14a810e410 con 0x7f14a8102aa0 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14ad351700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f14a810e710 con 0x7f14a8103ca0 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14a6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a8102aa0 0x7f14a8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14a6ffd700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a8102aa0 0x7f14a8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:60026/0 (socket says 172.21.15.62:60026) 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14a6ffd700 1 -- 172.21.15.62:0/49258676 learned_addr learned my addr 172.21.15.62:0/49258676 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T18:51:56.700 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14a67fc700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14a8103ca0 0x7f14a8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14a6ffd700 1 -- 172.21.15.62:0/49258676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14a8103ca0 msgr2=0x7f14a8199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14a6ffd700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14a8103ca0 0x7f14a8199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14a6ffd700 1 -- 172.21.15.62:0/49258676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14a8104640 msgr2=0x7f14a81a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14a6ffd700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14a8104640 0x7f14a81a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14a6ffd700 1 -- 172.21.15.62:0/49258676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a8074c70 con 0x7f14a8102aa0 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.696+0000 7f14a67fc700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14a8103ca0 0x7f14a8199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.697+0000 7f14a6ffd700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a8102aa0 0x7f14a8198c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009040 tx=0x7f14a0004e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.697+0000 7f149ffff700 1 -- 172.21.15.62:0/49258676 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a001f030 con 0x7f14a8102aa0 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.697+0000 7f14ad351700 1 -- 172.21.15.62:0/49258676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a8074f00 con 0x7f14a8102aa0 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.697+0000 7f149ffff700 1 -- 172.21.15.62:0/49258676 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14a0007b40 con 0x7f14a8102aa0 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.697+0000 7f149ffff700 1 -- 172.21.15.62:0/49258676 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a0004720 con 0x7f14a8102aa0 2024-04-25T18:51:56.701 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.697+0000 7f14ad351700 1 -- 172.21.15.62:0/49258676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a80753e0 con 0x7f14a8102aa0 2024-04-25T18:51:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.698+0000 7f14ad351700 1 -- 172.21.15.62:0/49258676 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14a8069730 con 0x7f14a8102aa0 2024-04-25T18:51:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.699+0000 7f149ffff700 1 -- 172.21.15.62:0/49258676 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f14a0003a00 con 0x7f14a8102aa0 2024-04-25T18:51:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.700+0000 7f149ffff700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f14940a3c40 0x7f14940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:51:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.700+0000 7f149ffff700 1 -- 172.21.15.62:0/49258676 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4965+0+0 (secure 0 0 0) 0x7f14a000f020 con 0x7f14a8102aa0 2024-04-25T18:51:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.701+0000 7f14a67fc700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f14940a3c40 0x7f14940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:51:56.703 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.701+0000 7f14a67fc700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f14940a3c40 0x7f14940a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1490005fd0 tx=0x7f1490005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:51:56.706 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.705+0000 7f149ffff700 1 -- 172.21.15.62:0/49258676 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14a0029020 con 0x7f14a8102aa0 2024-04-25T18:51:56.922 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:51:56.921+0000 7f14ad351700 1 -- 172.21.15.62:0/49258676 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f14a81099c0 con 0x7f14940a3c40 2024-04-25T18:51:58.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:51:58.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi045 2024-04-25T18:51:58.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi045 to 5698M 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:58.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:58.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:57 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:51:58.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi045 2024-04-25T18:51:58.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi045 to 5698M 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:58.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:58.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:57 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.323 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:51:58.323 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi045 2024-04-25T18:51:58.323 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.323 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.323 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:58.323 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:58.323 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:51:58.323 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi045 to 5698M 2024-04-25T18:51:58.323 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='client.24299 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:51:58.323 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.324 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:51:58.324 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:58.324 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:58.324 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:51:58.324 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:58.324 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:51:58.324 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:51:58.324 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:51:58.324 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:51:58.324 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:57 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:00.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:59 smithi079 ceph-mon[33002]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:00.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:59 smithi079 ceph-mon[33002]: from='client.? 172.21.15.62:0/2778066134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb5a094c-22a9-4e39-9014-57094699afec"}]: dispatch 2024-04-25T18:52:00.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:59 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb5a094c-22a9-4e39-9014-57094699afec"}]: dispatch 2024-04-25T18:52:00.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:59 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb5a094c-22a9-4e39-9014-57094699afec"}]': finished 2024-04-25T18:52:00.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:59 smithi079 ceph-mon[33002]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T18:52:00.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:51:59 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:00.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:59 smithi045 ceph-mon[27776]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:00.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:59 smithi045 ceph-mon[27776]: from='client.? 172.21.15.62:0/2778066134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb5a094c-22a9-4e39-9014-57094699afec"}]: dispatch 2024-04-25T18:52:00.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:59 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb5a094c-22a9-4e39-9014-57094699afec"}]: dispatch 2024-04-25T18:52:00.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:59 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb5a094c-22a9-4e39-9014-57094699afec"}]': finished 2024-04-25T18:52:00.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:59 smithi045 ceph-mon[27776]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T18:52:00.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:51:59 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:00.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:59 smithi062 ceph-mon[32844]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:00.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:59 smithi062 ceph-mon[32844]: from='client.? 172.21.15.62:0/2778066134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb5a094c-22a9-4e39-9014-57094699afec"}]: dispatch 2024-04-25T18:52:00.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:59 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb5a094c-22a9-4e39-9014-57094699afec"}]: dispatch 2024-04-25T18:52:00.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:59 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb5a094c-22a9-4e39-9014-57094699afec"}]': finished 2024-04-25T18:52:00.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:59 smithi062 ceph-mon[32844]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T18:52:00.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:51:59 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:01.173 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:00 smithi079 ceph-mon[33002]: from='client.? 172.21.15.62:0/333225349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:01.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:00 smithi045 ceph-mon[27776]: from='client.? 172.21.15.62:0/333225349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:01.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:00 smithi062 ceph-mon[32844]: from='client.? 172.21.15.62:0/333225349' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:02.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:01 smithi079 ceph-mon[33002]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:02.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:01 smithi045 ceph-mon[27776]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:02.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:01 smithi062 ceph-mon[32844]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:03 smithi079 ceph-mon[33002]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:04.198 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:03 smithi062 ceph-mon[32844]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:03 smithi045 ceph-mon[27776]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:05.973 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:05 smithi062 ceph-mon[32844]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:06.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:05 smithi079 ceph-mon[33002]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:06.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:05 smithi045 ceph-mon[27776]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:08.006 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:07 smithi062 ceph-mon[32844]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:08.006 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:07 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T18:52:08.006 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:07 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:08.007 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:07 smithi062 ceph-mon[32844]: Deploying daemon osd.7 on smithi062 2024-04-25T18:52:08.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:07 smithi079 ceph-mon[33002]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:08.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:07 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T18:52:08.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:07 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:08.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:07 smithi079 ceph-mon[33002]: Deploying daemon osd.7 on smithi062 2024-04-25T18:52:08.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:07 smithi045 ceph-mon[27776]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:08.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:07 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T18:52:08.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:07 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:08.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:07 smithi045 ceph-mon[27776]: Deploying daemon osd.7 on smithi062 2024-04-25T18:52:10.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:09 smithi062 ceph-mon[32844]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:10.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:09 smithi079 ceph-mon[33002]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:10.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:09 smithi045 ceph-mon[27776]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:11.993 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:11 smithi062 ceph-mon[32844]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:12.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:11 smithi045 ceph-mon[27776]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:12.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:11 smithi079 ceph-mon[33002]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:13.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:13.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:13.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:13.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:13 smithi079 ceph-mon[33002]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:13.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:13.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:13.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:13.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:13 smithi045 ceph-mon[27776]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:13.527 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:13.528 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:13.528 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:13.528 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:13 smithi062 ceph-mon[32844]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:15.914 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.913+0000 7f149ffff700 1 -- 172.21.15.62:0/49258676 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f14a81099c0 con 0x7f14940a3c40 2024-04-25T18:52:15.915 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 7 on host 'smithi062' 2024-04-25T18:52:15.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.915+0000 7f14ad351700 1 -- 172.21.15.62:0/49258676 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f14940a3c40 msgr2=0x7f14940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:15.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.915+0000 7f14ad351700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f14940a3c40 0x7f14940a6100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1490005fd0 tx=0x7f1490005f60 comp rx=0 tx=0).stop 2024-04-25T18:52:15.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.915+0000 7f14ad351700 1 -- 172.21.15.62:0/49258676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a8102aa0 msgr2=0x7f14a8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:15.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.915+0000 7f14ad351700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a8102aa0 0x7f14a8198c50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009040 tx=0x7f14a0004e00 comp rx=0 tx=0).stop 2024-04-25T18:52:15.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.915+0000 7f14ad351700 1 -- 172.21.15.62:0/49258676 shutdown_connections 2024-04-25T18:52:15.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.915+0000 7f14ad351700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f14940a3c40 0x7f14940a6100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:15.917 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.915+0000 7f14ad351700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f14a8102aa0 0x7f14a8198c50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.915+0000 7f14ad351700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f14a8104640 0x7f14a81a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.915+0000 7f14ad351700 1 --2- 172.21.15.62:0/49258676 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f14a8103ca0 0x7f14a8199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.915+0000 7f14ad351700 1 -- 172.21.15.62:0/49258676 >> 172.21.15.62:0/49258676 conn(0x7f14a80fe040 msgr2=0x7f14a81070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.916+0000 7f14ad351700 1 -- 172.21.15.62:0/49258676 shutdown_connections 2024-04-25T18:52:15.918 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:52:15.916+0000 7f14ad351700 1 -- 172.21.15.62:0/49258676 wait complete. 2024-04-25T18:52:16.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:15 smithi062 ceph-mon[32844]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:15 smithi079 ceph-mon[33002]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:15 smithi045 ceph-mon[27776]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:16.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:16.500 DEBUG:teuthology.orchestra.run.smithi062:osd.7> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T18:52:16.502 INFO:tasks.cephadm:Deploying osd.8 on smithi079 with /dev/vg_nvme/lv_2... 2024-04-25T18:52:16.502 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T18:52:16.747 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:52:17.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:16 smithi062 ceph-mon[32844]: from='osd.7 [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:52:17.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:16 smithi062 ceph-mon[32844]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:52:17.131 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:16 smithi079 ceph-mon[33002]: from='osd.7 [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:52:17.131 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:16 smithi079 ceph-mon[33002]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:52:17.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:16 smithi045 ceph-mon[27776]: from='osd.7 [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:52:17.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:16 smithi045 ceph-mon[27776]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T18:52:18.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:17 smithi079 ceph-mon[33002]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:18.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:17 smithi079 ceph-mon[33002]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T18:52:18.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:17 smithi079 ceph-mon[33002]: from='osd.7 [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:52:18.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:17 smithi079 ceph-mon[33002]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T18:52:18.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:18.100 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:17 smithi079 ceph-mon[33002]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:52:18.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:17 smithi045 ceph-mon[27776]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:18.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:17 smithi045 ceph-mon[27776]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T18:52:18.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:17 smithi045 ceph-mon[27776]: from='osd.7 [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:52:18.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:17 smithi045 ceph-mon[27776]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T18:52:18.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:18.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:17 smithi045 ceph-mon[27776]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:52:18.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:17 smithi062 ceph-mon[32844]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:18.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:17 smithi062 ceph-mon[32844]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T18:52:18.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:17 smithi062 ceph-mon[32844]: from='osd.7 [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:52:18.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:17 smithi062 ceph-mon[32844]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T18:52:18.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:18.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:17 smithi062 ceph-mon[32844]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:52:18.644 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T18:52:18.645 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T18:52:18.645 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-04-25T18:52:18.645 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-04-25T18:52:18.645 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125475 s, 836 MB/s 2024-04-25T18:52:18.645 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-04-25T18:52:19.079 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:18 smithi062 ceph-mon[32844]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:52:19.079 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:18 smithi062 ceph-mon[32844]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T18:52:19.079 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:18 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:19.079 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:18 smithi062 ceph-mon[32844]: from='osd.7 ' entity='osd.7' 2024-04-25T18:52:19.079 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:18 smithi062 ceph-mon[32844]: osdmap e53: 8 total, 7 up, 8 in 2024-04-25T18:52:19.079 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:18 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:19.079 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 18:52:18 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T18:52:18.895+0000 7f9fe4494700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T18:52:19.080 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 18:52:18 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T18:52:18.969+0000 7f9fdc27f700 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:52:19.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:18 smithi045 ceph-mon[27776]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:52:19.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:18 smithi045 ceph-mon[27776]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T18:52:19.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:18 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:19.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:18 smithi045 ceph-mon[27776]: from='osd.7 ' entity='osd.7' 2024-04-25T18:52:19.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:18 smithi045 ceph-mon[27776]: osdmap e53: 8 total, 7 up, 8 in 2024-04-25T18:52:19.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:18 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:19.258 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:18 smithi079 ceph-mon[33002]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:52:19.258 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:18 smithi079 ceph-mon[33002]: osdmap e52: 8 total, 7 up, 8 in 2024-04-25T18:52:19.258 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:18 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:19.259 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:18 smithi079 ceph-mon[33002]: from='osd.7 ' entity='osd.7' 2024-04-25T18:52:19.259 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:18 smithi079 ceph-mon[33002]: osdmap e53: 8 total, 7 up, 8 in 2024-04-25T18:52:19.259 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:18 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:19.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_2 2024-04-25T18:52:19.527 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:52:20.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:52:20.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:52:20.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:20.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:20.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:20.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:20.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:20.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:20.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: osd.7 [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] boot 2024-04-25T18:52:20.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T18:52:20.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:20.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: osd.7 [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] boot 2024-04-25T18:52:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T18:52:20.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:52:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:52:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:20.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:20.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:20.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:20.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: osd.7 [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] boot 2024-04-25T18:52:20.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T18:52:20.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:52:21.080 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:20 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi062 2024-04-25T18:52:21.080 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:20 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi062 to 5696M 2024-04-25T18:52:21.080 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:20 smithi079 ceph-mon[33002]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T18:52:21.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.078+0000 7f10b8cec700 1 -- 172.21.15.79:0/88200360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10b4101490 msgr2=0x7f10b410b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:21.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.078+0000 7f10b8cec700 1 --2- 172.21.15.79:0/88200360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10b4101490 0x7f10b410b060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00b0a0 tx=0x7f10ac00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:52:21.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.078+0000 7f10b8cec700 1 -- 172.21.15.79:0/88200360 shutdown_connections 2024-04-25T18:52:21.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.078+0000 7f10b8cec700 1 --2- 172.21.15.79:0/88200360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10b4101490 0x7f10b410b060 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:21.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.078+0000 7f10b8cec700 1 --2- 172.21.15.79:0/88200360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4104070 0x7f10b4104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:21.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.078+0000 7f10b8cec700 1 --2- 172.21.15.79:0/88200360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b4100af0 0x7f10b4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:21.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.078+0000 7f10b8cec700 1 -- 172.21.15.79:0/88200360 >> 172.21.15.79:0/88200360 conn(0x7f10b40fc6d0 msgr2=0x7f10b40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:21.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.078+0000 7f10b8cec700 1 -- 172.21.15.79:0/88200360 shutdown_connections 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.078+0000 7f10b8cec700 1 -- 172.21.15.79:0/88200360 wait complete. 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b8cec700 1 Processor -- start 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b8cec700 1 -- start start 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b8cec700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b4100af0 0x7f10b4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b8cec700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4101490 0x7f10b4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b8cec700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10b4104070 0x7f10b41a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b8cec700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10b410e560 con 0x7f10b4101490 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b8cec700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f10b410e3e0 con 0x7f10b4100af0 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b8cec700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f10b410e6e0 con 0x7f10b4104070 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b2d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10b4104070 0x7f10b41a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b2d9d700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10b4104070 0x7f10b41a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54680/0 (socket says 172.21.15.79:54680) 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b2d9d700 1 -- 172.21.15.79:0/1315653147 learned_addr learned my addr 172.21.15.79:0/1315653147 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b2d9d700 1 -- 172.21.15.79:0/1315653147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b4100af0 msgr2=0x7f10b4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:21.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.079+0000 7f10b259c700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b4100af0 0x7f10b4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.080+0000 7f10b2d9d700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b4100af0 0x7f10b4198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.080+0000 7f10b2d9d700 1 -- 172.21.15.79:0/1315653147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4101490 msgr2=0x7f10b4199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.080+0000 7f10b2d9d700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4101490 0x7f10b4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.080+0000 7f10b2d9d700 1 -- 172.21.15.79:0/1315653147 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b41a08a0 con 0x7f10b4104070 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.080+0000 7f10b2d9d700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10b4104070 0x7f10b41a01d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f10ac004fd0 tx=0x7f10ac007770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.080+0000 7f10a37fe700 1 -- 172.21.15.79:0/1315653147 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac021020 con 0x7f10b4104070 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.080+0000 7f10b8cec700 1 -- 172.21.15.79:0/1315653147 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b41a0b30 con 0x7f10b4104070 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.080+0000 7f10a37fe700 1 -- 172.21.15.79:0/1315653147 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac01d920 con 0x7f10b4104070 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.080+0000 7f10a37fe700 1 -- 172.21.15.79:0/1315653147 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac0054c0 con 0x7f10b4104070 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.080+0000 7f10b8cec700 1 -- 172.21.15.79:0/1315653147 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b41a1140 con 0x7f10b4104070 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.081+0000 7f10b8cec700 1 -- 172.21.15.79:0/1315653147 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b4069730 con 0x7f10b4104070 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.082+0000 7f10a37fe700 1 -- 172.21.15.79:0/1315653147 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f10ac024020 con 0x7f10b4104070 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.082+0000 7f10a37fe700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f109c0a3c40 0x7f109c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.083+0000 7f10a37fe700 1 -- 172.21.15.79:0/1315653147 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5426+0+0 (secure 0 0 0) 0x7f10ac0d1830 con 0x7f10b4104070 2024-04-25T18:52:21.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.083+0000 7f10b259c700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f109c0a3c40 0x7f109c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:21.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.084+0000 7f10b259c700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f109c0a3c40 0x7f109c0a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10b419a130 tx=0x7f10a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:21.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.086+0000 7f10a37fe700 1 -- 172.21.15.79:0/1315653147 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10ac0074e0 con 0x7f10b4104070 2024-04-25T18:52:21.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:21.290+0000 7f10b8cec700 1 -- 172.21.15.79:0/1315653147 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f10b4062e80 con 0x7f109c0a3c40 2024-04-25T18:52:21.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:20 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi062 2024-04-25T18:52:21.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:20 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi062 to 5696M 2024-04-25T18:52:21.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:20 smithi062 ceph-mon[32844]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T18:52:21.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:20 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi062 2024-04-25T18:52:21.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:20 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi062 to 5696M 2024-04-25T18:52:21.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:20 smithi045 ceph-mon[27776]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T18:52:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:21 smithi079 ceph-mon[33002]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:21 smithi079 ceph-mon[33002]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-25T18:52:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:21 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:21 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:21 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:21 smithi079 ceph-mon[33002]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T18:52:22.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:21 smithi062 ceph-mon[32844]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:21 smithi062 ceph-mon[32844]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-25T18:52:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:21 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:21 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:21 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:21 smithi062 ceph-mon[32844]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T18:52:22.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:21 smithi045 ceph-mon[27776]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-04-25T18:52:22.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:21 smithi045 ceph-mon[27776]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-04-25T18:52:22.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:21 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:22.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:21 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:22.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:21 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:22.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:21 smithi045 ceph-mon[27776]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T18:52:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:22 smithi079 ceph-mon[33002]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:23 smithi079 ceph-mon[33002]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T18:52:23.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:23 smithi062 ceph-mon[32844]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:23.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:23 smithi062 ceph-mon[32844]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T18:52:23.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:22 smithi045 ceph-mon[27776]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:23.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:22 smithi045 ceph-mon[27776]: osdmap e57: 8 total, 8 up, 8 in 2024-04-25T18:52:24.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:24 smithi062 ceph-mon[32844]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:24.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:24 smithi062 ceph-mon[32844]: osdmap e58: 8 total, 8 up, 8 in 2024-04-25T18:52:24.388 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:24 smithi079 ceph-mon[33002]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:24.389 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:24 smithi079 ceph-mon[33002]: osdmap e58: 8 total, 8 up, 8 in 2024-04-25T18:52:24.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:24 smithi045 ceph-mon[27776]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:24.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:24 smithi045 ceph-mon[27776]: osdmap e58: 8 total, 8 up, 8 in 2024-04-25T18:52:25.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:25 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/2151861036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9e1829d-cafc-47cf-ba9b-1dc60e0c6421"}]: dispatch 2024-04-25T18:52:25.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:25 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9e1829d-cafc-47cf-ba9b-1dc60e0c6421"}]: dispatch 2024-04-25T18:52:25.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:25 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9e1829d-cafc-47cf-ba9b-1dc60e0c6421"}]': finished 2024-04-25T18:52:25.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:25 smithi062 ceph-mon[32844]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T18:52:25.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:25 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:25.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:25 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/451805240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:25 smithi079 ceph-mon[33002]: from='client.? 172.21.15.79:0/2151861036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9e1829d-cafc-47cf-ba9b-1dc60e0c6421"}]: dispatch 2024-04-25T18:52:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:25 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9e1829d-cafc-47cf-ba9b-1dc60e0c6421"}]: dispatch 2024-04-25T18:52:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:25 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9e1829d-cafc-47cf-ba9b-1dc60e0c6421"}]': finished 2024-04-25T18:52:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:25 smithi079 ceph-mon[33002]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T18:52:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:25 smithi079 ceph-mon[33002]: from='client.? 172.21.15.79:0/451805240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:25.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:25 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/2151861036' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9e1829d-cafc-47cf-ba9b-1dc60e0c6421"}]: dispatch 2024-04-25T18:52:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:25 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b9e1829d-cafc-47cf-ba9b-1dc60e0c6421"}]: dispatch 2024-04-25T18:52:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:25 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b9e1829d-cafc-47cf-ba9b-1dc60e0c6421"}]': finished 2024-04-25T18:52:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:25 smithi045 ceph-mon[27776]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T18:52:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:25 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/451805240' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:26.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:26 smithi062 ceph-mon[32844]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:26 smithi079 ceph-mon[33002]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:26.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:26 smithi045 ceph-mon[27776]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:27.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:27 smithi062 ceph-mon[32844]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:27.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:27 smithi062 ceph-mon[32844]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-25T18:52:27.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:27 smithi062 ceph-mon[32844]: Cluster is now healthy 2024-04-25T18:52:27.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:27 smithi079 ceph-mon[33002]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:27.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:27 smithi079 ceph-mon[33002]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-25T18:52:27.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:27 smithi079 ceph-mon[33002]: Cluster is now healthy 2024-04-25T18:52:27.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:27 smithi045 ceph-mon[27776]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:27.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:27 smithi045 ceph-mon[27776]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-04-25T18:52:27.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:27 smithi045 ceph-mon[27776]: Cluster is now healthy 2024-04-25T18:52:29.552 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:29 smithi079 ceph-mon[33002]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:29.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:29 smithi062 ceph-mon[32844]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:29.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:29 smithi045 ceph-mon[27776]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:31.521 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:31 smithi079 ceph-mon[33002]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:31.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:31 smithi062 ceph-mon[32844]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:31.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:31 smithi045 ceph-mon[27776]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:32.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T18:52:32.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:32.508 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:32 smithi079 ceph-mon[33002]: Deploying daemon osd.8 on smithi079 2024-04-25T18:52:32.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T18:52:32.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:32.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:32 smithi062 ceph-mon[32844]: Deploying daemon osd.8 on smithi079 2024-04-25T18:52:32.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T18:52:32.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:32.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:32 smithi045 ceph-mon[27776]: Deploying daemon osd.8 on smithi079 2024-04-25T18:52:33.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:33 smithi079 ceph-mon[33002]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:33.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:33 smithi062 ceph-mon[32844]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:33.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:33 smithi045 ceph-mon[27776]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:35.505 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:35 smithi079 ceph-mon[33002]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:35.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:35 smithi062 ceph-mon[32844]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:35.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:35 smithi045 ceph-mon[27776]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:37.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:37 smithi062 ceph-mon[32844]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:37.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:37 smithi079 ceph-mon[33002]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:37.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:37.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:37.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:37.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:37 smithi045 ceph-mon[27776]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:37.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:37.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:37.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:39.305 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:39 smithi079 ceph-mon[33002]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:39.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:39 smithi062 ceph-mon[32844]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:39.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:39 smithi045 ceph-mon[27776]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:40.983 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 8 on host 'smithi079' 2024-04-25T18:52:40.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.982+0000 7f10a37fe700 1 -- 172.21.15.79:0/1315653147 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f10b4062e80 con 0x7f109c0a3c40 2024-04-25T18:52:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.985+0000 7f10b8cec700 1 -- 172.21.15.79:0/1315653147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f109c0a3c40 msgr2=0x7f109c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.985+0000 7f10b8cec700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f109c0a3c40 0x7f109c0a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f10b419a130 tx=0x7f10a4008040 comp rx=0 tx=0).stop 2024-04-25T18:52:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.985+0000 7f10b8cec700 1 -- 172.21.15.79:0/1315653147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10b4104070 msgr2=0x7f10b41a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.985+0000 7f10b8cec700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10b4104070 0x7f10b41a01d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f10ac004fd0 tx=0x7f10ac007770 comp rx=0 tx=0).stop 2024-04-25T18:52:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.986+0000 7f10b8cec700 1 -- 172.21.15.79:0/1315653147 shutdown_connections 2024-04-25T18:52:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.986+0000 7f10b8cec700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f109c0a3c40 0x7f109c0a6100 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.986+0000 7f10b8cec700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f10b4100af0 0x7f10b4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.986+0000 7f10b8cec700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10b4104070 0x7f10b41a01d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.986+0000 7f10b8cec700 1 --2- 172.21.15.79:0/1315653147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4101490 0x7f10b4199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:40.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.986+0000 7f10b8cec700 1 -- 172.21.15.79:0/1315653147 >> 172.21.15.79:0/1315653147 conn(0x7f10b40fc6d0 msgr2=0x7f10b41071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:40.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.986+0000 7f10b8cec700 1 -- 172.21.15.79:0/1315653147 shutdown_connections 2024-04-25T18:52:40.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:52:40.986+0000 7f10b8cec700 1 -- 172.21.15.79:0/1315653147 wait complete. 2024-04-25T18:52:41.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:41.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:41.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:41.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.437 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.437 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.437 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:41.437 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: from='osd.8 [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:52:41.437 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:41 smithi079 ceph-mon[33002]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:52:41.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:41.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:41.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:41.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:41.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: from='osd.8 [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:52:41.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:41 smithi045 ceph-mon[27776]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: from='osd.8 [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:52:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:41 smithi062 ceph-mon[32844]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T18:52:41.642 DEBUG:teuthology.orchestra.run.smithi079:osd.8> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T18:52:41.644 INFO:tasks.cephadm:Deploying osd.9 on smithi045 with /dev/vg_nvme/lv_1... 2024-04-25T18:52:41.644 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T18:52:42.106 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:52:43.091 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:42 smithi079 ceph-mon[33002]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T18:52:43.091 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:42 smithi079 ceph-mon[33002]: from='osd.8 [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:52:43.091 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:42 smithi079 ceph-mon[33002]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T18:52:43.091 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:42 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:43.091 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:42 smithi079 ceph-mon[33002]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:52:43.091 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 18:52:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T18:52:42.962+0000 7fedaaf52700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T18:52:43.091 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 18:52:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T18:52:42.975+0000 7feda6d45700 -1 osd.8 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:52:43.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:42 smithi045 ceph-mon[27776]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T18:52:43.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:42 smithi045 ceph-mon[27776]: from='osd.8 [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:52:43.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:42 smithi045 ceph-mon[27776]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T18:52:43.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:42 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:43.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:42 smithi045 ceph-mon[27776]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:52:43.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:42 smithi062 ceph-mon[32844]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T18:52:43.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:42 smithi062 ceph-mon[32844]: from='osd.8 [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:52:43.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:42 smithi062 ceph-mon[32844]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T18:52:43.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:42 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:43.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:42 smithi062 ceph-mon[32844]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:52:44.015 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:43 smithi045 ceph-mon[27776]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:44.015 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:43 smithi045 ceph-mon[27776]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:52:44.015 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:43 smithi045 ceph-mon[27776]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T18:52:44.015 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:44.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:43 smithi079 ceph-mon[33002]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:44.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:43 smithi079 ceph-mon[33002]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:52:44.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:43 smithi079 ceph-mon[33002]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T18:52:44.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:44.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:43 smithi062 ceph-mon[32844]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-04-25T18:52:44.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:43 smithi062 ceph-mon[32844]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:52:44.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:43 smithi062 ceph-mon[32844]: osdmap e61: 9 total, 8 up, 9 in 2024-04-25T18:52:44.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:45.033 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T18:52:45.033 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T18:52:45.033 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-25T18:52:45.034 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-25T18:52:45.034 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0363425 s, 289 MB/s 2024-04-25T18:52:45.034 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-25T18:52:45.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: osd.8 [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] boot 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:44 smithi062 ceph-mon[32844]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T18:52:45.365 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: osd.8 [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] boot 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.366 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:44 smithi045 ceph-mon[27776]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T18:52:45.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:52:45.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:52:45.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: osd.8 [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] boot 2024-04-25T18:52:45.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T18:52:45.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:52:45.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:45.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:45.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:52:45.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:45.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:52:45.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:52:45.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:44 smithi079 ceph-mon[33002]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T18:52:45.861 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-04-25T18:52:46.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:45 smithi045 ceph-mon[27776]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:46.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:45 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi079 2024-04-25T18:52:46.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:45 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi079 to 5696M 2024-04-25T18:52:46.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:45 smithi045 ceph-mon[27776]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T18:52:46.325 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:52:46.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:45 smithi062 ceph-mon[32844]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:46.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:45 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi079 2024-04-25T18:52:46.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:45 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi079 to 5696M 2024-04-25T18:52:46.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:45 smithi062 ceph-mon[32844]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T18:52:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:45 smithi079 ceph-mon[33002]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:45 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi079 2024-04-25T18:52:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:45 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi079 to 5696M 2024-04-25T18:52:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:45 smithi079 ceph-mon[33002]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T18:52:48.131 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:48 smithi045 ceph-mon[27776]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:48.132 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:48 smithi045 ceph-mon[27776]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T18:52:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:48 smithi062 ceph-mon[32844]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:48 smithi062 ceph-mon[32844]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T18:52:48.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:48 smithi079 ceph-mon[33002]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:48.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:48 smithi079 ceph-mon[33002]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T18:52:48.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.471+0000 7fc30ea00700 1 -- 172.21.15.45:0/3937328716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308103cd0 msgr2=0x7fc308104130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:48.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.471+0000 7fc30ea00700 1 --2- 172.21.15.45:0/3937328716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308103cd0 0x7fc308104130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009920 tx=0x7fc2f0009c30 comp rx=0 tx=0).stop 2024-04-25T18:52:48.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.471+0000 7fc30ea00700 1 -- 172.21.15.45:0/3937328716 shutdown_connections 2024-04-25T18:52:48.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.471+0000 7fc30ea00700 1 --2- 172.21.15.45:0/3937328716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc308104670 0x7fc30810af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:48.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.471+0000 7fc30ea00700 1 --2- 172.21.15.45:0/3937328716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc308102ad0 0x7fc308102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:48.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.471+0000 7fc30ea00700 1 --2- 172.21.15.45:0/3937328716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308103cd0 0x7fc308104130 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:48.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.471+0000 7fc30ea00700 1 -- 172.21.15.45:0/3937328716 >> 172.21.15.45:0/3937328716 conn(0x7fc3080fe090 msgr2=0x7fc3081004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:52:48.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.471+0000 7fc30ea00700 1 -- 172.21.15.45:0/3937328716 shutdown_connections 2024-04-25T18:52:48.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.471+0000 7fc30ea00700 1 -- 172.21.15.45:0/3937328716 wait complete. 2024-04-25T18:52:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.473+0000 7fc30ea00700 1 Processor -- start 2024-04-25T18:52:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.473+0000 7fc30ea00700 1 -- start start 2024-04-25T18:52:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.473+0000 7fc30ea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc308102ad0 0x7fc30819aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.473+0000 7fc30ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308103cd0 0x7fc30819b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.473+0000 7fc30ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc308104670 0x7fc3081a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.473+0000 7fc30ea00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810e5b0 con 0x7fc308102ad0 2024-04-25T18:52:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.473+0000 7fc30ea00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810e430 con 0x7fc308103cd0 2024-04-25T18:52:48.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.473+0000 7fc30ea00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc30810e730 con 0x7fc308104670 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc307e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc308102ad0 0x7fc30819aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308103cd0 0x7fc30819b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc30ca00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc308104670 0x7fc3081a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc30ca00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc308104670 0x7fc3081a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:52528/0 (socket says 172.21.15.45:52528) 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc30ca00700 1 -- 172.21.15.45:0/2139095368 learned_addr learned my addr 172.21.15.45:0/2139095368 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc2ff400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308103cd0 0x7fc30819b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:34304/0 (socket says 172.21.15.45:34304) 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc2ff400700 1 -- 172.21.15.45:0/2139095368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc308104670 msgr2=0x7fc3081a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc2ff400700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc308104670 0x7fc3081a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc2ff400700 1 -- 172.21.15.45:0/2139095368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc308102ad0 msgr2=0x7fc30819aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc2ff400700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc308102ad0 0x7fc30819aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc2ff400700 1 -- 172.21.15.45:0/2139095368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3081a2af0 con 0x7fc308103cd0 2024-04-25T18:52:48.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.474+0000 7fc2ff400700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308103cd0 0x7fc30819b420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009230 tx=0x7fc2f0005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.475+0000 7fc305600700 1 -- 172.21.15.45:0/2139095368 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0027070 con 0x7fc308103cd0 2024-04-25T18:52:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.475+0000 7fc305600700 1 -- 172.21.15.45:0/2139095368 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f000ba10 con 0x7fc308103cd0 2024-04-25T18:52:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.475+0000 7fc305600700 1 -- 172.21.15.45:0/2139095368 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0004930 con 0x7fc308103cd0 2024-04-25T18:52:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.475+0000 7fc30ea00700 1 -- 172.21.15.45:0/2139095368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3081a2d80 con 0x7fc308103cd0 2024-04-25T18:52:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.475+0000 7fc30ea00700 1 -- 172.21.15.45:0/2139095368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081a3180 con 0x7fc308103cd0 2024-04-25T18:52:48.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.476+0000 7fc30ea00700 1 -- 172.21.15.45:0/2139095368 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3081a3350 con 0x7fc308103cd0 2024-04-25T18:52:48.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.478+0000 7fc305600700 1 -- 172.21.15.45:0/2139095368 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc2f002a070 con 0x7fc308103cd0 2024-04-25T18:52:48.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.480+0000 7fc305600700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc2ec0a3cc0 0x7fc2ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:52:48.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.480+0000 7fc305600700 1 -- 172.21.15.45:0/2139095368 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5829+0+0 (secure 0 0 0) 0x7fc2f00d0480 con 0x7fc308103cd0 2024-04-25T18:52:48.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.481+0000 7fc307e00700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc2ec0a3cc0 0x7fc2ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:52:48.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.481+0000 7fc307e00700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc2ec0a3cc0 0x7fc2ec0a6180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc308103b30 tx=0x7fc2f800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:52:48.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.485+0000 7fc305600700 1 -- 172.21.15.45:0/2139095368 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc2f005eff0 con 0x7fc308103cd0 2024-04-25T18:52:48.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:52:48.727+0000 7fc30ea00700 1 -- 172.21.15.45:0/2139095368 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc30810ac10 con 0x7fc2ec0a3cc0 2024-04-25T18:52:49.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:49 smithi062 ceph-mon[32844]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T18:52:49.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:49.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:49.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:49.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:49 smithi079 ceph-mon[33002]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T18:52:49.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:49.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:49.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:49.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:49 smithi045 ceph-mon[27776]: osdmap e66: 9 total, 9 up, 9 in 2024-04-25T18:52:49.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:52:49.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:52:49.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:52:50.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:50 smithi062 ceph-mon[32844]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:50.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:50 smithi062 ceph-mon[32844]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:50 smithi079 ceph-mon[33002]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:50 smithi079 ceph-mon[33002]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:50 smithi045 ceph-mon[27776]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:50 smithi045 ceph-mon[27776]: from='client.24338 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:52:52.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:52 smithi062 ceph-mon[32844]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:52.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:52 smithi079 ceph-mon[33002]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:52.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:52 smithi045 ceph-mon[27776]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:52:53.345 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:53 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2832172515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f783937a-cdd7-4310-a6b4-00cf934db047"}]: dispatch 2024-04-25T18:52:53.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:53 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2832172515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f783937a-cdd7-4310-a6b4-00cf934db047"}]': finished 2024-04-25T18:52:53.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:53 smithi045 ceph-mon[27776]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T18:52:53.346 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:52:53.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:53 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2832172515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f783937a-cdd7-4310-a6b4-00cf934db047"}]: dispatch 2024-04-25T18:52:53.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:53 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2832172515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f783937a-cdd7-4310-a6b4-00cf934db047"}]': finished 2024-04-25T18:52:53.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:53 smithi062 ceph-mon[32844]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T18:52:53.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:52:53.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:53 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2832172515' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f783937a-cdd7-4310-a6b4-00cf934db047"}]: dispatch 2024-04-25T18:52:53.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:53 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2832172515' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f783937a-cdd7-4310-a6b4-00cf934db047"}]': finished 2024-04-25T18:52:53.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:53 smithi079 ceph-mon[33002]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T18:52:53.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:52:54.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:54 smithi062 ceph-mon[32844]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T18:52:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:54 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3189372981' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:54.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:54 smithi079 ceph-mon[33002]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T18:52:54.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:54 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3189372981' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:54.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:54 smithi045 ceph-mon[27776]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T18:52:54.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:54 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3189372981' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:52:56.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:56 smithi062 ceph-mon[32844]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T18:52:56.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:56 smithi079 ceph-mon[33002]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T18:52:56.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:56 smithi045 ceph-mon[27776]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T18:52:58.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:52:58 smithi045 ceph-mon[27776]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T18:52:58.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:52:58 smithi062 ceph-mon[32844]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T18:52:58.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:52:58 smithi079 ceph-mon[33002]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T18:53:00.086 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:00 smithi045 ceph-mon[27776]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:53:00.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:00 smithi062 ceph-mon[32844]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:53:00.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:00 smithi079 ceph-mon[33002]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:53:02.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:02 smithi062 ceph-mon[32844]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:53:02.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:02 smithi079 ceph-mon[33002]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:53:02.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:02 smithi045 ceph-mon[27776]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T18:53:03.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T18:53:03.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:03.419 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T18:53:03.419 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:03.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T18:53:03.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:04.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:04 smithi062 ceph-mon[32844]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:04.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:04 smithi062 ceph-mon[32844]: Deploying daemon osd.9 on smithi045 2024-04-25T18:53:04.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:04 smithi079 ceph-mon[33002]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:04.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:04 smithi079 ceph-mon[33002]: Deploying daemon osd.9 on smithi045 2024-04-25T18:53:04.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:04 smithi045 ceph-mon[27776]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:04.606 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:04 smithi045 ceph-mon[27776]: Deploying daemon osd.9 on smithi045 2024-04-25T18:53:06.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:06 smithi062 ceph-mon[32844]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:06.433 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:06 smithi045 ceph-mon[27776]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:06.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:06 smithi079 ceph-mon[33002]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:08.181 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:08 smithi045 ceph-mon[27776]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:08.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:08 smithi062 ceph-mon[32844]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:08.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:08 smithi079 ceph-mon[33002]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:10.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:10 smithi045 ceph-mon[27776]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:10.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:10 smithi062 ceph-mon[32844]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:10.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:10 smithi079 ceph-mon[33002]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:11 smithi079 ceph-mon[33002]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:11 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:11 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:11 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:12.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:11 smithi045 ceph-mon[27776]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:12.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:11 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:12.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:11 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:12.202 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:11 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:12.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:11 smithi062 ceph-mon[32844]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:11 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:11 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:11 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:14.130 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:13 smithi045 ceph-mon[27776]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:14.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:13 smithi079 ceph-mon[33002]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:14.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:13 smithi062 ceph-mon[32844]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:15.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:15 smithi045 ceph-mon[27776]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:15.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:15.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:15 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:15.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:15 smithi045 ceph-mon[27776]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T18:53:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:15 smithi062 ceph-mon[32844]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:15 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:16.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:15 smithi062 ceph-mon[32844]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T18:53:16.125 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 9 on host 'smithi045' 2024-04-25T18:53:16.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.122+0000 7fc305600700 1 -- 172.21.15.45:0/2139095368 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc30810ac10 con 0x7fc2ec0a3cc0 2024-04-25T18:53:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.125+0000 7fc30ea00700 1 -- 172.21.15.45:0/2139095368 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc2ec0a3cc0 msgr2=0x7fc2ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.125+0000 7fc30ea00700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc2ec0a3cc0 0x7fc2ec0a6180 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc308103b30 tx=0x7fc2f800b040 comp rx=0 tx=0).stop 2024-04-25T18:53:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.126+0000 7fc30ea00700 1 -- 172.21.15.45:0/2139095368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308103cd0 msgr2=0x7fc30819b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.126+0000 7fc30ea00700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308103cd0 0x7fc30819b420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009230 tx=0x7fc2f0005270 comp rx=0 tx=0).stop 2024-04-25T18:53:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.126+0000 7fc30ea00700 1 -- 172.21.15.45:0/2139095368 shutdown_connections 2024-04-25T18:53:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.126+0000 7fc30ea00700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc2ec0a3cc0 0x7fc2ec0a6180 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.126+0000 7fc30ea00700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc308104670 0x7fc3081a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.126+0000 7fc30ea00700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc308102ad0 0x7fc30819aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:16.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.126+0000 7fc30ea00700 1 --2- 172.21.15.45:0/2139095368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc308103cd0 0x7fc30819b420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.126+0000 7fc30ea00700 1 -- 172.21.15.45:0/2139095368 >> 172.21.15.45:0/2139095368 conn(0x7fc3080fe090 msgr2=0x7fc3081094f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.126+0000 7fc30ea00700 1 -- 172.21.15.45:0/2139095368 shutdown_connections 2024-04-25T18:53:16.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:53:16.126+0000 7fc30ea00700 1 -- 172.21.15.45:0/2139095368 wait complete. 2024-04-25T18:53:16.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:15 smithi079 ceph-mon[33002]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:15 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:15 smithi079 ceph-mon[33002]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T18:53:16.993 DEBUG:teuthology.orchestra.run.smithi045:osd.9> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T18:53:16.998 INFO:tasks.cephadm:Deploying osd.10 on smithi062 with /dev/vg_nvme/lv_1... 2024-04-25T18:53:16.999 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T18:53:17.026 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:16 smithi062 ceph-mon[32844]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T18:53:17.026 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:16 smithi062 ceph-mon[32844]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T18:53:17.027 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:16 smithi062 ceph-mon[32844]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:53:17.027 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:17.027 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:17.027 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:17.056 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:16 smithi045 ceph-mon[27776]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T18:53:17.056 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:16 smithi045 ceph-mon[27776]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T18:53:17.056 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:16 smithi045 ceph-mon[27776]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:53:17.056 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:17.056 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:17.056 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:16 smithi079 ceph-mon[33002]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T18:53:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:16 smithi079 ceph-mon[33002]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T18:53:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:16 smithi079 ceph-mon[33002]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-25T18:53:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:17.255 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:53:17.981 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:17 smithi045 ceph-mon[27776]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:17.981 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:17 smithi045 ceph-mon[27776]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:53:17.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:17 smithi045 ceph-mon[27776]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T18:53:17.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:17.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:17.982 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:17 smithi045 ceph-mon[27776]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' 2024-04-25T18:53:17.982 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 18:53:17 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T18:53:17.714+0000 7f273ea00700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T18:53:17.982 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 18:53:17 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T18:53:17.743+0000 7f2734800700 -1 osd.9 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:53:18.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:17 smithi062 ceph-mon[32844]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:18.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:17 smithi062 ceph-mon[32844]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:53:18.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:17 smithi062 ceph-mon[32844]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T18:53:18.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:18.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:18.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:17 smithi062 ceph-mon[32844]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' 2024-04-25T18:53:18.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:17 smithi079 ceph-mon[33002]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:18.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:17 smithi079 ceph-mon[33002]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-25T18:53:18.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:17 smithi079 ceph-mon[33002]: osdmap e69: 10 total, 9 up, 10 in 2024-04-25T18:53:18.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:18.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:18.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:17 smithi079 ceph-mon[33002]: from='osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689]' entity='osd.9' 2024-04-25T18:53:18.809 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:18 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:53:18.809 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:18 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:53:18.809 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:18 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:18.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:18 smithi045 ceph-mon[27776]: osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] boot 2024-04-25T18:53:18.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:18 smithi045 ceph-mon[27776]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T18:53:18.810 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:18 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:18 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:53:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:18 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:53:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:18 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:18 smithi062 ceph-mon[32844]: osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] boot 2024-04-25T18:53:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:18 smithi062 ceph-mon[32844]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T18:53:19.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:18 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:18 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:53:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:18 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:53:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:18 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:18 smithi079 ceph-mon[33002]: osd.9 [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] boot 2024-04-25T18:53:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:18 smithi079 ceph-mon[33002]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T18:53:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:18 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:53:19.276 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T18:53:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T18:53:19.277 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 10+0 records in 2024-04-25T18:53:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:10+0 records out 2024-04-25T18:53:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112687 s, 931 MB/s 2024-04-25T18:53:19.277 INFO:teuthology.orchestra.run.smithi062.stderr: stderr: 2024-04-25T18:53:19.277 INFO:teuthology.orchestra.run.smithi062.stderr:--> Zapping successful for: 2024-04-25T18:53:19.934 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:19 smithi062 ceph-mon[32844]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:19.961 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi062:vg_nvme/lv_1 2024-04-25T18:53:19.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:19 smithi045 ceph-mon[27776]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:20.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:19 smithi079 ceph-mon[33002]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 274 MiB used, 804 GiB / 805 GiB avail 2024-04-25T18:53:20.254 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T18:53:21.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:20 smithi062 ceph-mon[32844]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T18:53:21.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:20 smithi079 ceph-mon[33002]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T18:53:21.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:20 smithi045 ceph-mon[27776]: osdmap e71: 10 total, 10 up, 10 in 2024-04-25T18:53:21.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.714+0000 7f1ac43d8700 1 -- 172.21.15.62:0/1201167966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1abc105ee0 msgr2=0x7f1abc1082d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:21.717 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.714+0000 7f1ac43d8700 1 --2- 172.21.15.62:0/1201167966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1abc105ee0 0x7f1abc1082d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009990 tx=0x7f1ab8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.714+0000 7f1ac43d8700 1 -- 172.21.15.62:0/1201167966 shutdown_connections 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.714+0000 7f1ac43d8700 1 --2- 172.21.15.62:0/1201167966 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1abc108810 0x7f1abc10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.714+0000 7f1ac43d8700 1 --2- 172.21.15.62:0/1201167966 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc1035b0 0x7f1abc1059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.714+0000 7f1ac43d8700 1 --2- 172.21.15.62:0/1201167966 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1abc105ee0 0x7f1abc1082d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.714+0000 7f1ac43d8700 1 -- 172.21.15.62:0/1201167966 >> 172.21.15.62:0/1201167966 conn(0x7f1abc0fcf40 msgr2=0x7f1abc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.714+0000 7f1ac43d8700 1 -- 172.21.15.62:0/1201167966 shutdown_connections 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.714+0000 7f1ac43d8700 1 -- 172.21.15.62:0/1201167966 wait complete. 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac43d8700 1 Processor -- start 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac43d8700 1 -- start start 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac43d8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc1035b0 0x7f1abc19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac43d8700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1abc105ee0 0x7f1abc19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac43d8700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1abc108810 0x7f1abc1a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:21.718 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac43d8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc10e5d0 con 0x7f1abc1035b0 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac43d8700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc10e450 con 0x7f1abc108810 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac43d8700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc10e750 con 0x7f1abc105ee0 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac2975700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1abc108810 0x7f1abc1a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac2975700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1abc108810 0x7f1abc1a2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.62:52796/0 (socket says 172.21.15.62:52796) 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac2975700 1 -- 172.21.15.62:0/3110784844 learned_addr learned my addr 172.21.15.62:0/3110784844 (peer_addr_for_me v2:172.21.15.62:0/0) 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac1973700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1abc105ee0 0x7f1abc19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac2975700 1 -- 172.21.15.62:0/3110784844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1abc105ee0 msgr2=0x7f1abc19b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac2975700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1abc105ee0 0x7f1abc19b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac2975700 1 -- 172.21.15.62:0/3110784844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc1035b0 msgr2=0x7f1abc19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac2975700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc1035b0 0x7f1abc19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac2975700 1 -- 172.21.15.62:0/3110784844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc1a2af0 con 0x7f1abc108810 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.715+0000 7f1ac2975700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1abc108810 0x7f1abc1a2450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40094f0 tx=0x7f1ab4007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:21.719 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.716+0000 7f1aaf7fe700 1 -- 172.21.15.62:0/3110784844 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4004db0 con 0x7f1abc108810 2024-04-25T18:53:21.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.716+0000 7f1aaf7fe700 1 -- 172.21.15.62:0/3110784844 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ab400db60 con 0x7f1abc108810 2024-04-25T18:53:21.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.716+0000 7f1ac43d8700 1 -- 172.21.15.62:0/3110784844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc1a2de0 con 0x7f1abc108810 2024-04-25T18:53:21.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.716+0000 7f1aaf7fe700 1 -- 172.21.15.62:0/3110784844 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4015400 con 0x7f1abc108810 2024-04-25T18:53:21.720 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.716+0000 7f1ac43d8700 1 -- 172.21.15.62:0/3110784844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc1a3310 con 0x7f1abc108810 2024-04-25T18:53:21.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.717+0000 7f1ac43d8700 1 -- 172.21.15.62:0/3110784844 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc069730 con 0x7f1abc108810 2024-04-25T18:53:21.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.718+0000 7f1aaf7fe700 1 -- 172.21.15.62:0/3110784844 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1ab400dd10 con 0x7f1abc108810 2024-04-25T18:53:21.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.718+0000 7f1aaf7fe700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1aa80a3c40 0x7f1aa80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:21.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.719+0000 7f1aaf7fe700 1 -- 172.21.15.62:0/3110784844 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6261+0+0 (secure 0 0 0) 0x7f1ab40cfe70 con 0x7f1abc108810 2024-04-25T18:53:21.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.719+0000 7f1ac2174700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1aa80a3c40 0x7f1aa80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:21.721 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.720+0000 7f1ac2174700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1aa80a3c40 0x7f1aa80a6100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00097b0 tx=0x7f1ab0006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:21.723 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.722+0000 7f1aaf7fe700 1 -- 172.21.15.62:0/3110784844 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ab4010070 con 0x7f1abc108810 2024-04-25T18:53:21.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:21 smithi062 ceph-mon[32844]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:21.942 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:21.941+0000 7f1ac43d8700 1 -- 172.21.15.62:0/3110784844 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1abc062f40 con 0x7f1aa80a3c40 2024-04-25T18:53:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:21 smithi079 ceph-mon[33002]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:22.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:21 smithi045 ceph-mon[27776]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:22.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:22.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:22.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:22.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:22.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:23.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:23.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:23.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:23.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:23.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:23.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:23.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:23.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:23.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:23.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:23.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='client.24365 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi062:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:23.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:23.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:23.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:23.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:23.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:23.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:24.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:23 smithi062 ceph-mon[32844]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:24.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:23 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi045 2024-04-25T18:53:24.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:23 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-25T18:53:24.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:23 smithi079 ceph-mon[33002]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:24.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:23 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi045 2024-04-25T18:53:24.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:23 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-25T18:53:24.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:23 smithi045 ceph-mon[27776]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:24.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:23 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi045 2024-04-25T18:53:24.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:23 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-25T18:53:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:24 smithi062 ceph-mon[32844]: from='client.? 172.21.15.62:0/3544780787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2abd19-2916-4e72-ace4-4fc3a72ef06c"}]: dispatch 2024-04-25T18:53:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:24 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2abd19-2916-4e72-ace4-4fc3a72ef06c"}]: dispatch 2024-04-25T18:53:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:24 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf2abd19-2916-4e72-ace4-4fc3a72ef06c"}]': finished 2024-04-25T18:53:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:24 smithi062 ceph-mon[32844]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T18:53:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:24 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:25.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:24 smithi079 ceph-mon[33002]: from='client.? 172.21.15.62:0/3544780787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2abd19-2916-4e72-ace4-4fc3a72ef06c"}]: dispatch 2024-04-25T18:53:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:24 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2abd19-2916-4e72-ace4-4fc3a72ef06c"}]: dispatch 2024-04-25T18:53:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:24 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf2abd19-2916-4e72-ace4-4fc3a72ef06c"}]': finished 2024-04-25T18:53:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:24 smithi079 ceph-mon[33002]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T18:53:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:25.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:24 smithi045 ceph-mon[27776]: from='client.? 172.21.15.62:0/3544780787' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2abd19-2916-4e72-ace4-4fc3a72ef06c"}]: dispatch 2024-04-25T18:53:25.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:24 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf2abd19-2916-4e72-ace4-4fc3a72ef06c"}]: dispatch 2024-04-25T18:53:25.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:24 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf2abd19-2916-4e72-ace4-4fc3a72ef06c"}]': finished 2024-04-25T18:53:25.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:24 smithi045 ceph-mon[27776]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T18:53:25.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:26.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:25 smithi062 ceph-mon[32844]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:26.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:25 smithi062 ceph-mon[32844]: from='client.? 172.21.15.62:0/816683107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:26.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:25 smithi079 ceph-mon[33002]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:26.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:25 smithi079 ceph-mon[33002]: from='client.? 172.21.15.62:0/816683107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:26.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:25 smithi045 ceph-mon[27776]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:25 smithi045 ceph-mon[27776]: from='client.? 172.21.15.62:0/816683107' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:28.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:27 smithi062 ceph-mon[32844]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:28.182 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:27 smithi045 ceph-mon[27776]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:28.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:27 smithi079 ceph-mon[33002]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:30.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:29 smithi062 ceph-mon[32844]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:30.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:29 smithi079 ceph-mon[33002]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:30.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:29 smithi045 ceph-mon[27776]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:32.099 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:31 smithi062 ceph-mon[32844]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:32.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:31 smithi079 ceph-mon[33002]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:32.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:31 smithi045 ceph-mon[27776]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:33.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T18:53:33.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:32 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:33.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T18:53:33.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:32 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:33.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T18:53:33.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:32 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:34.075 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:33 smithi062 ceph-mon[32844]: Deploying daemon osd.10 on smithi062 2024-04-25T18:53:34.076 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:33 smithi062 ceph-mon[32844]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:34.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:33 smithi079 ceph-mon[33002]: Deploying daemon osd.10 on smithi062 2024-04-25T18:53:34.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:33 smithi079 ceph-mon[33002]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:34.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:33 smithi045 ceph-mon[27776]: Deploying daemon osd.10 on smithi062 2024-04-25T18:53:34.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:33 smithi045 ceph-mon[27776]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:36.068 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:35 smithi062 ceph-mon[32844]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:36.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:35 smithi079 ceph-mon[33002]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:36.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:35 smithi045 ceph-mon[27776]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:37 smithi062 ceph-mon[32844]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:38.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:38.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:38.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:38.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:37 smithi079 ceph-mon[33002]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:38.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:38.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:38.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:38.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:37 smithi045 ceph-mon[27776]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:38.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:38.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:38.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:40.036 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:39 smithi062 ceph-mon[32844]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:40.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:39 smithi079 ceph-mon[33002]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:40.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:39 smithi045 ceph-mon[27776]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:41.580 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.579+0000 7f1aaf7fe700 1 -- 172.21.15.62:0/3110784844 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1abc062f40 con 0x7f1aa80a3c40 2024-04-25T18:53:41.580 INFO:teuthology.orchestra.run.smithi062.stdout:Created osd(s) 10 on host 'smithi062' 2024-04-25T18:53:41.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 -- 172.21.15.62:0/3110784844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1aa80a3c40 msgr2=0x7f1aa80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:41.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1aa80a3c40 0x7f1aa80a6100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00097b0 tx=0x7f1ab0006d20 comp rx=0 tx=0).stop 2024-04-25T18:53:41.582 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 -- 172.21.15.62:0/3110784844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1abc108810 msgr2=0x7f1abc1a2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1abc108810 0x7f1abc1a2450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40094f0 tx=0x7f1ab4007f90 comp rx=0 tx=0).stop 2024-04-25T18:53:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 -- 172.21.15.62:0/3110784844 shutdown_connections 2024-04-25T18:53:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1aa80a3c40 0x7f1aa80a6100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1abc108810 0x7f1abc1a2450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1abc1035b0 0x7f1abc19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 --2- 172.21.15.62:0/3110784844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1abc105ee0 0x7f1abc19b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 -- 172.21.15.62:0/3110784844 >> 172.21.15.62:0/3110784844 conn(0x7f1abc0fcf40 msgr2=0x7f1abc102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 -- 172.21.15.62:0/3110784844 shutdown_connections 2024-04-25T18:53:41.583 INFO:teuthology.orchestra.run.smithi062.stderr:2024-04-25T18:53:41.581+0000 7f1ac43d8700 1 -- 172.21.15.62:0/3110784844 wait complete. 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: from='osd.10 [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:53:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:41 smithi062 ceph-mon[32844]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:53:42.181 DEBUG:teuthology.orchestra.run.smithi062:osd.10> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: from='osd.10 [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:53:42.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:41 smithi079 ceph-mon[33002]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:53:42.185 INFO:tasks.cephadm:Deploying osd.11 on smithi079 with /dev/vg_nvme/lv_1... 2024-04-25T18:53:42.185 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T18:53:42.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:42.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:42.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:42.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:42.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:53:42.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: from='osd.10 [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:53:42.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:41 smithi045 ceph-mon[27776]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T18:53:42.430 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:53:43.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:43 smithi079 ceph-mon[33002]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:43.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:43 smithi079 ceph-mon[33002]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T18:53:43.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:43 smithi079 ceph-mon[33002]: from='osd.10 [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:53:43.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:43 smithi079 ceph-mon[33002]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T18:53:43.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:43.806 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:43 smithi079 ceph-mon[33002]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:53:43.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:43 smithi062 ceph-mon[32844]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:43.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:43 smithi062 ceph-mon[32844]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T18:53:43.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:43 smithi062 ceph-mon[32844]: from='osd.10 [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:53:43.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:43 smithi062 ceph-mon[32844]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T18:53:43.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:43.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:43 smithi062 ceph-mon[32844]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:53:43.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:43 smithi045 ceph-mon[27776]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:43.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:43 smithi045 ceph-mon[27776]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T18:53:43.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:43 smithi045 ceph-mon[27776]: from='osd.10 [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:53:43.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:43 smithi045 ceph-mon[27776]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T18:53:43.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:43.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:43 smithi045 ceph-mon[27776]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]: dispatch 2024-04-25T18:53:44.503 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T18:53:44.503 INFO:teuthology.orchestra.run.smithi079.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T18:53:44.504 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10+0 records in 2024-04-25T18:53:44.504 INFO:teuthology.orchestra.run.smithi079.stderr:10+0 records out 2024-04-25T18:53:44.504 INFO:teuthology.orchestra.run.smithi079.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130288 s, 805 MB/s 2024-04-25T18:53:44.504 INFO:teuthology.orchestra.run.smithi079.stderr:--> Zapping successful for: 2024-04-25T18:53:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:44 smithi079 ceph-mon[33002]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:53:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:44 smithi079 ceph-mon[33002]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T18:53:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:44 smithi079 ceph-mon[33002]: from='osd.10 ' entity='osd.10' 2024-04-25T18:53:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:44 smithi079 ceph-mon[33002]: osdmap e75: 11 total, 10 up, 11 in 2024-04-25T18:53:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:44 smithi062 ceph-mon[32844]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:53:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:44 smithi062 ceph-mon[32844]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T18:53:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:44 smithi062 ceph-mon[32844]: from='osd.10 ' entity='osd.10' 2024-04-25T18:53:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:44 smithi062 ceph-mon[32844]: osdmap e75: 11 total, 10 up, 11 in 2024-04-25T18:53:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:44.865 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 18:53:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T18:53:44.491+0000 7f597a2fa700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T18:53:44.865 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 18:53:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T18:53:44.537+0000 7f59718e4700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:53:44.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:44 smithi045 ceph-mon[27776]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi062", "root=default"]}]': finished 2024-04-25T18:53:44.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:44 smithi045 ceph-mon[27776]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T18:53:44.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:44.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:44.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:44 smithi045 ceph-mon[27776]: from='osd.10 ' entity='osd.10' 2024-04-25T18:53:44.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:44 smithi045 ceph-mon[27776]: osdmap e75: 11 total, 10 up, 11 in 2024-04-25T18:53:44.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:45.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi079:vg_nvme/lv_1 2024-04-25T18:53:45.377 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:53:45.756 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.757 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: osd.10 [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] boot 2024-04-25T18:53:45.758 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T18:53:45.758 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: osd.10 [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] boot 2024-04-25T18:53:45.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T18:53:45.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:53:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:53:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:53:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:45.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:53:45.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:53:45.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: osd.10 [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] boot 2024-04-25T18:53:45.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T18:53:45.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:53:47.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.021+0000 7f917397e700 1 -- 172.21.15.79:0/710825153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916c103ca0 msgr2=0x7f916c104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.021+0000 7f917397e700 1 --2- 172.21.15.79:0/710825153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916c103ca0 0x7f916c104100 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9168009990 tx=0x7f9168009ca0 comp rx=0 tx=0).stop 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.021+0000 7f917397e700 1 -- 172.21.15.79:0/710825153 shutdown_connections 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.021+0000 7f917397e700 1 --2- 172.21.15.79:0/710825153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f916c102aa0 0x7f916c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.021+0000 7f917397e700 1 --2- 172.21.15.79:0/710825153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f916c104640 0x7f916c10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.021+0000 7f917397e700 1 --2- 172.21.15.79:0/710825153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916c103ca0 0x7f916c104100 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.021+0000 7f917397e700 1 -- 172.21.15.79:0/710825153 >> 172.21.15.79:0/710825153 conn(0x7f916c0fe040 msgr2=0x7f916c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.021+0000 7f917397e700 1 -- 172.21.15.79:0/710825153 shutdown_connections 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.021+0000 7f917397e700 1 -- 172.21.15.79:0/710825153 wait complete. 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f917397e700 1 Processor -- start 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f917397e700 1 -- start start 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f917397e700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f916c102aa0 0x7f916c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f917397e700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916c103ca0 0x7f916c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f917397e700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f916c104640 0x7f916c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:47.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f917397e700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10e630 con 0x7f916c102aa0 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f917397e700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10e4b0 con 0x7f916c104640 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f917397e700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f916c10e7b0 con 0x7f916c103ca0 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f9170f19700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916c103ca0 0x7f916c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f9170f19700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916c103ca0 0x7f916c1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44774/0 (socket says 172.21.15.79:44774) 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f9170f19700 1 -- 172.21.15.79:0/4201446308 learned_addr learned my addr 172.21.15.79:0/4201446308 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.022+0000 7f9171f1b700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f916c104640 0x7f916c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f9170f19700 1 -- 172.21.15.79:0/4201446308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f916c104640 msgr2=0x7f916c1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f9170f19700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f916c104640 0x7f916c1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f9170f19700 1 -- 172.21.15.79:0/4201446308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f916c102aa0 msgr2=0x7f916c198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f9170f19700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f916c102aa0 0x7f916c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f9170f19700 1 -- 172.21.15.79:0/4201446308 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f916c1a0930 con 0x7f916c103ca0 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f9170f19700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916c103ca0 0x7f916c1991f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91680036f0 tx=0x7f9168003b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f91627fc700 1 -- 172.21.15.79:0/4201446308 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168027070 con 0x7f916c103ca0 2024-04-25T18:53:47.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f91627fc700 1 -- 172.21.15.79:0/4201446308 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f916800bd90 con 0x7f916c103ca0 2024-04-25T18:53:47.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f917397e700 1 -- 172.21.15.79:0/4201446308 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f916c1a0bc0 con 0x7f916c103ca0 2024-04-25T18:53:47.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f91627fc700 1 -- 172.21.15.79:0/4201446308 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916800f460 con 0x7f916c103ca0 2024-04-25T18:53:47.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.023+0000 7f917397e700 1 -- 172.21.15.79:0/4201446308 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f916c1a1010 con 0x7f916c103ca0 2024-04-25T18:53:47.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.024+0000 7f917397e700 1 -- 172.21.15.79:0/4201446308 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f916c02df80 con 0x7f916c103ca0 2024-04-25T18:53:47.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.025+0000 7f91627fc700 1 -- 172.21.15.79:0/4201446308 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f916802a070 con 0x7f916c103ca0 2024-04-25T18:53:47.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.026+0000 7f91627fc700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91580a3c40 0x7f91580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:53:47.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.026+0000 7f91627fc700 1 -- 172.21.15.79:0/4201446308 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 6723+0+0 (secure 0 0 0) 0x7f91680d1750 con 0x7f916c103ca0 2024-04-25T18:53:47.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.026+0000 7f917171a700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91580a3c40 0x7f91580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:53:47.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.027+0000 7f917171a700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91580a3c40 0x7f91580a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f916c199b90 tx=0x7f915c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:53:47.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.029+0000 7f91627fc700 1 -- 172.21.15.79:0/4201446308 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9168031020 con 0x7f916c103ca0 2024-04-25T18:53:47.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:53:47.243+0000 7f917397e700 1 -- 172.21.15.79:0/4201446308 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f916c062e80 con 0x7f91580a3c40 2024-04-25T18:53:47.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:47 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi062 2024-04-25T18:53:47.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:47 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi062 to 4272M 2024-04-25T18:53:47.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:47 smithi062 ceph-mon[32844]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T18:53:47.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:47 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi062 2024-04-25T18:53:47.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:47 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi062 to 4272M 2024-04-25T18:53:47.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:47 smithi079 ceph-mon[33002]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T18:53:47.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:47 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi062 2024-04-25T18:53:47.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:47 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi062 to 4272M 2024-04-25T18:53:47.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:47 smithi045 ceph-mon[27776]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T18:53:48.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:48 smithi079 ceph-mon[33002]: pgmap v262: 1 pgs: 1 remapped+peering; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:48.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:48 smithi079 ceph-mon[33002]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T18:53:48.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:48 smithi079 ceph-mon[33002]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:48.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:48 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:48.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:48 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:48.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:48 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:48.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:48 smithi045 ceph-mon[27776]: pgmap v262: 1 pgs: 1 remapped+peering; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:48.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:48 smithi045 ceph-mon[27776]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T18:53:48.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:48 smithi045 ceph-mon[27776]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:48.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:48 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:48.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:48 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:48.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:48 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:48.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:48 smithi062 ceph-mon[32844]: pgmap v262: 1 pgs: 1 remapped+peering; 577 KiB data, 309 MiB used, 894 GiB / 894 GiB avail 2024-04-25T18:53:48.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:48 smithi062 ceph-mon[32844]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T18:53:48.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:48 smithi062 ceph-mon[32844]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi079:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:53:48.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:48 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T18:53:48.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:48 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T18:53:48.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:48 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:49.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:49 smithi079 ceph-mon[33002]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T18:53:49.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:49 smithi079 ceph-mon[33002]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:53:49.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:49 smithi045 ceph-mon[27776]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T18:53:49.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:49 smithi045 ceph-mon[27776]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:53:49.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:49 smithi062 ceph-mon[32844]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T18:53:49.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:49 smithi062 ceph-mon[32844]: pgmap v265: 1 pgs: 1 remapped+peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:53:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:50 smithi079 ceph-mon[33002]: osdmap e80: 11 total, 11 up, 11 in 2024-04-25T18:53:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:50 smithi079 ceph-mon[33002]: from='client.? 172.21.15.79:0/36497283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197c940c-1050-4bff-b88c-86b6a4da0cf5"}]: dispatch 2024-04-25T18:53:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:50 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197c940c-1050-4bff-b88c-86b6a4da0cf5"}]: dispatch 2024-04-25T18:53:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:50 smithi079 ceph-mon[33002]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "197c940c-1050-4bff-b88c-86b6a4da0cf5"}]': finished 2024-04-25T18:53:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:50 smithi079 ceph-mon[33002]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T18:53:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:50 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:53:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:50 smithi045 ceph-mon[27776]: osdmap e80: 11 total, 11 up, 11 in 2024-04-25T18:53:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:50 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/36497283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197c940c-1050-4bff-b88c-86b6a4da0cf5"}]: dispatch 2024-04-25T18:53:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:50 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197c940c-1050-4bff-b88c-86b6a4da0cf5"}]: dispatch 2024-04-25T18:53:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:50 smithi045 ceph-mon[27776]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "197c940c-1050-4bff-b88c-86b6a4da0cf5"}]': finished 2024-04-25T18:53:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:50 smithi045 ceph-mon[27776]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T18:53:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:50 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:53:50.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:50 smithi062 ceph-mon[32844]: osdmap e80: 11 total, 11 up, 11 in 2024-04-25T18:53:50.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:50 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/36497283' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197c940c-1050-4bff-b88c-86b6a4da0cf5"}]: dispatch 2024-04-25T18:53:50.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:50 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "197c940c-1050-4bff-b88c-86b6a4da0cf5"}]: dispatch 2024-04-25T18:53:50.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:50 smithi062 ceph-mon[32844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "197c940c-1050-4bff-b88c-86b6a4da0cf5"}]': finished 2024-04-25T18:53:50.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:50 smithi062 ceph-mon[32844]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T18:53:50.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:50 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:53:51.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:51 smithi079 ceph-mon[33002]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:53:51.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:51 smithi079 ceph-mon[33002]: from='client.? 172.21.15.79:0/3743939557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:51.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:51 smithi045 ceph-mon[27776]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:53:51.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:51 smithi045 ceph-mon[27776]: from='client.? 172.21.15.79:0/3743939557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:51 smithi062 ceph-mon[32844]: pgmap v268: 1 pgs: 1 remapped+peering; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:53:51.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:51 smithi062 ceph-mon[32844]: from='client.? 172.21.15.79:0/3743939557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T18:53:53.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:53 smithi062 ceph-mon[32844]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-25T18:53:53.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:53 smithi079 ceph-mon[33002]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-25T18:53:53.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:53 smithi045 ceph-mon[27776]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-25T18:53:55.407 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:55 smithi079 ceph-mon[33002]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T18:53:55.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:55 smithi062 ceph-mon[32844]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T18:53:55.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:55 smithi045 ceph-mon[27776]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 983 GiB / 983 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-04-25T18:53:57.391 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:57 smithi079 ceph-mon[33002]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:53:57.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:57 smithi062 ceph-mon[32844]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:53:57.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:57 smithi045 ceph-mon[27776]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T18:53:58.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T18:53:58.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:58.360 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:58 smithi079 ceph-mon[33002]: Deploying daemon osd.11 on smithi079 2024-04-25T18:53:58.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T18:53:58.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:58.565 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:58 smithi045 ceph-mon[27776]: Deploying daemon osd.11 on smithi079 2024-04-25T18:53:58.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T18:53:58.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:53:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:58 smithi062 ceph-mon[32844]: Deploying daemon osd.11 on smithi079 2024-04-25T18:53:59.609 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:53:59 smithi079 ceph-mon[33002]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T18:53:59.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:53:59 smithi062 ceph-mon[32844]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T18:53:59.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:53:59 smithi045 ceph-mon[27776]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T18:54:01.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:01 smithi062 ceph-mon[32844]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T18:54:01.646 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:01 smithi079 ceph-mon[33002]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T18:54:01.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:01 smithi045 ceph-mon[27776]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-04-25T18:54:04.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:03 smithi062 ceph-mon[32844]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T18:54:04.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:04.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:04.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:03 smithi079 ceph-mon[33002]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T18:54:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:04.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:03 smithi045 ceph-mon[27776]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-25T18:54:04.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:04.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:04.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:06.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:05 smithi062 ceph-mon[32844]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:06.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:05 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:06.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:05 smithi079 ceph-mon[33002]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:06.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:05 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:06.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:05 smithi045 ceph-mon[27776]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:06.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:06.836 INFO:teuthology.orchestra.run.smithi079.stdout:Created osd(s) 11 on host 'smithi079' 2024-04-25T18:54:06.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.835+0000 7f91627fc700 1 -- 172.21.15.79:0/4201446308 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f916c062e80 con 0x7f91580a3c40 2024-04-25T18:54:06.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 -- 172.21.15.79:0/4201446308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91580a3c40 msgr2=0x7f91580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:06.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91580a3c40 0x7f91580a6100 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f916c199b90 tx=0x7f915c006d20 comp rx=0 tx=0).stop 2024-04-25T18:54:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 -- 172.21.15.79:0/4201446308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916c103ca0 msgr2=0x7f916c1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916c103ca0 0x7f916c1991f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f91680036f0 tx=0x7f9168003b50 comp rx=0 tx=0).stop 2024-04-25T18:54:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 -- 172.21.15.79:0/4201446308 shutdown_connections 2024-04-25T18:54:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f91580a3c40 0x7f91580a6100 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f916c104640 0x7f916c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f916c102aa0 0x7f916c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 --2- 172.21.15.79:0/4201446308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f916c103ca0 0x7f916c1991f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 -- 172.21.15.79:0/4201446308 >> 172.21.15.79:0/4201446308 conn(0x7f916c0fe040 msgr2=0x7f916c1070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 -- 172.21.15.79:0/4201446308 shutdown_connections 2024-04-25T18:54:06.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:06.837+0000 7f917397e700 1 -- 172.21.15.79:0/4201446308 wait complete. 2024-04-25T18:54:07.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:07.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:07.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:06 smithi062 ceph-mon[32844]: from='osd.11 [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:54:07.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:06 smithi062 ceph-mon[32844]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:54:07.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:07.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:07.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:07.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:06 smithi079 ceph-mon[33002]: from='osd.11 [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:54:07.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:06 smithi079 ceph-mon[33002]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:54:07.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:07.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:07.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:07.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:06 smithi045 ceph-mon[27776]: from='osd.11 [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:54:07.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:06 smithi045 ceph-mon[27776]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T18:54:07.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:07.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:07.763 DEBUG:teuthology.orchestra.run.smithi079:osd.11> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T18:54:07.765 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T18:54:07.765 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T18:54:07.996 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:07 smithi079 ceph-mon[33002]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:07.997 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:07 smithi079 ceph-mon[33002]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T18:54:07.997 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:07 smithi079 ceph-mon[33002]: from='osd.11 [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:54:07.997 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:07 smithi079 ceph-mon[33002]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T18:54:07.997 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:07 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:07.997 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:07 smithi079 ceph-mon[33002]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:54:08.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:07 smithi045 ceph-mon[27776]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:08.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:07 smithi045 ceph-mon[27776]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T18:54:08.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:07 smithi045 ceph-mon[27776]: from='osd.11 [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:54:08.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:07 smithi045 ceph-mon[27776]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T18:54:08.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:07 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:08.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:07 smithi045 ceph-mon[27776]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:54:08.241 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:08.274 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:07 smithi062 ceph-mon[32844]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:08.274 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:07 smithi062 ceph-mon[32844]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T18:54:08.274 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:07 smithi062 ceph-mon[32844]: from='osd.11 [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:54:08.274 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:07 smithi062 ceph-mon[32844]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T18:54:08.274 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:07 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:08.274 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:07 smithi062 ceph-mon[32844]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]: dispatch 2024-04-25T18:54:08.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:08 smithi045 ceph-mon[27776]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:54:08.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:08 smithi045 ceph-mon[27776]: osdmap e83: 12 total, 11 up, 12 in 2024-04-25T18:54:08.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:08 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:08.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:08 smithi045 ceph-mon[27776]: from='osd.11 ' entity='osd.11' 2024-04-25T18:54:08.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:08 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:09.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:08 smithi079 ceph-mon[33002]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:54:09.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:08 smithi079 ceph-mon[33002]: osdmap e83: 12 total, 11 up, 12 in 2024-04-25T18:54:09.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:08 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:09.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:08 smithi079 ceph-mon[33002]: from='osd.11 ' entity='osd.11' 2024-04-25T18:54:09.055 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:08 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:09.055 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 18:54:08 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T18:54:08.815+0000 7f430af05700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T18:54:09.055 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 18:54:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T18:54:09.001+0000 7f4302cf0700 -1 osd.11 83 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T18:54:09.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:08 smithi062 ceph-mon[32844]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi079", "root=default"]}]': finished 2024-04-25T18:54:09.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:08 smithi062 ceph-mon[32844]: osdmap e83: 12 total, 11 up, 12 in 2024-04-25T18:54:09.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:08 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:09.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:08 smithi062 ceph-mon[32844]: from='osd.11 ' entity='osd.11' 2024-04-25T18:54:09.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:08 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:10.124 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:09 smithi079 ceph-mon[33002]: purged_snaps scrub starts 2024-04-25T18:54:10.124 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:09 smithi079 ceph-mon[33002]: purged_snaps scrub ok 2024-04-25T18:54:10.124 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:09 smithi079 ceph-mon[33002]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:10.124 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:09 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:10.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:09 smithi045 ceph-mon[27776]: purged_snaps scrub starts 2024-04-25T18:54:10.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:09 smithi045 ceph-mon[27776]: purged_snaps scrub ok 2024-04-25T18:54:10.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:09 smithi045 ceph-mon[27776]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:10.153 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:09 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:10.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:09 smithi062 ceph-mon[32844]: purged_snaps scrub starts 2024-04-25T18:54:10.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:09 smithi062 ceph-mon[32844]: purged_snaps scrub ok 2024-04-25T18:54:10.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:09 smithi062 ceph-mon[32844]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:10.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:09 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.636+0000 7ff140200700 1 -- 172.21.15.45:0/355594527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1380754a0 msgr2=0x7ff138075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:10.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.636+0000 7ff140200700 1 --2- 172.21.15.45:0/355594527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1380754a0 0x7ff138075920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff1240098f0 tx=0x7ff124009c00 comp rx=0 tx=0).stop 2024-04-25T18:54:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.637+0000 7ff140200700 1 -- 172.21.15.45:0/355594527 shutdown_connections 2024-04-25T18:54:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.637+0000 7ff140200700 1 --2- 172.21.15.45:0/355594527 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff138076a70 0x7ff138074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.637+0000 7ff140200700 1 --2- 172.21.15.45:0/355594527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff13810a980 0x7ff13810ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.637+0000 7ff140200700 1 --2- 172.21.15.45:0/355594527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1380754a0 0x7ff138075920 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.637+0000 7ff140200700 1 -- 172.21.15.45:0/355594527 >> 172.21.15.45:0/355594527 conn(0x7ff1380fe010 msgr2=0x7ff138100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.637+0000 7ff140200700 1 -- 172.21.15.45:0/355594527 shutdown_connections 2024-04-25T18:54:10.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.637+0000 7ff140200700 1 -- 172.21.15.45:0/355594527 wait complete. 2024-04-25T18:54:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.638+0000 7ff140200700 1 Processor -- start 2024-04-25T18:54:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.638+0000 7ff140200700 1 -- start start 2024-04-25T18:54:10.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.638+0000 7ff140200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1380754a0 0x7ff13819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.638+0000 7ff140200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff138076a70 0x7ff13819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.638+0000 7ff140200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff13810a980 0x7ff1381a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.638+0000 7ff140200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff138110740 con 0x7ff138076a70 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.638+0000 7ff140200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381105c0 con 0x7ff13810a980 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.638+0000 7ff140200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff1381108c0 con 0x7ff1380754a0 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13ce00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff138076a70 0x7ff13819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1380754a0 0x7ff13819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13ce00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff138076a70 0x7ff13819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55550/0 (socket says 172.21.15.45:55550) 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13ce00700 1 -- 172.21.15.45:0/2982897999 learned_addr learned my addr 172.21.15.45:0/2982897999 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13e200700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff13810a980 0x7ff1381a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13d800700 1 -- 172.21.15.45:0/2982897999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff13810a980 msgr2=0x7ff1381a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13d800700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff13810a980 0x7ff1381a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:10.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13d800700 1 -- 172.21.15.45:0/2982897999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff138076a70 msgr2=0x7ff13819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13d800700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff138076a70 0x7ff13819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13d800700 1 -- 172.21.15.45:0/2982897999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1381a2a80 con 0x7ff1380754a0 2024-04-25T18:54:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.639+0000 7ff13ce00700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff138076a70 0x7ff13819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.640+0000 7ff13d800700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1380754a0 0x7ff13819ae20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00e9f0 tx=0x7ff12c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.640+0000 7ff136000700 1 -- 172.21.15.45:0/2982897999 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00c9c0 con 0x7ff1380754a0 2024-04-25T18:54:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.640+0000 7ff140200700 1 -- 172.21.15.45:0/2982897999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1381a2d70 con 0x7ff1380754a0 2024-04-25T18:54:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.640+0000 7ff136000700 1 -- 172.21.15.45:0/2982897999 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c010430 con 0x7ff1380754a0 2024-04-25T18:54:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.640+0000 7ff136000700 1 -- 172.21.15.45:0/2982897999 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12c018680 con 0x7ff1380754a0 2024-04-25T18:54:10.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.640+0000 7ff140200700 1 -- 172.21.15.45:0/2982897999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1381a3210 con 0x7ff1380754a0 2024-04-25T18:54:10.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.641+0000 7ff140200700 1 -- 172.21.15.45:0/2982897999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff138069730 con 0x7ff1380754a0 2024-04-25T18:54:10.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.645+0000 7ff136000700 1 -- 172.21.15.45:0/2982897999 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff12c0108e0 con 0x7ff1380754a0 2024-04-25T18:54:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.646+0000 7ff136000700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1200a3c40 0x7ff1200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.646+0000 7ff136000700 1 -- 172.21.15.45:0/2982897999 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7ff12c0d0070 con 0x7ff1380754a0 2024-04-25T18:54:10.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.646+0000 7ff13ce00700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1200a3c40 0x7ff1200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:10.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.647+0000 7ff13ce00700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1200a3c40 0x7ff1200a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff124009fd0 tx=0x7ff124019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:10.653 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.650+0000 7ff136000700 1 -- 172.21.15.45:0/2982897999 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff12c09ce50 con 0x7ff1380754a0 2024-04-25T18:54:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.883+0000 7ff140200700 1 -- 172.21.15.45:0/2982897999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff138063680 con 0x7ff1380754a0 2024-04-25T18:54:10.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.884+0000 7ff136000700 1 -- 172.21.15.45:0/2982897999 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v84) ==== 74+0+133 (secure 0 0 0) 0x7ff12c09c750 con 0x7ff1380754a0 2024-04-25T18:54:10.887 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:54:10.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.886+0000 7ff140200700 1 -- 172.21.15.45:0/2982897999 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1200a3c40 msgr2=0x7ff1200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:10.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.886+0000 7ff140200700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1200a3c40 0x7ff1200a6100 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff124009fd0 tx=0x7ff124019040 comp rx=0 tx=0).stop 2024-04-25T18:54:10.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.886+0000 7ff140200700 1 -- 172.21.15.45:0/2982897999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1380754a0 msgr2=0x7ff13819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.886+0000 7ff140200700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1380754a0 0x7ff13819ae20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00e9f0 tx=0x7ff12c00ed00 comp rx=0 tx=0).stop 2024-04-25T18:54:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.887+0000 7ff140200700 1 -- 172.21.15.45:0/2982897999 shutdown_connections 2024-04-25T18:54:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.887+0000 7ff140200700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1200a3c40 0x7ff1200a6100 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.887+0000 7ff140200700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff13810a980 0x7ff1381a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.887+0000 7ff140200700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1380754a0 0x7ff13819ae20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:10.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.887+0000 7ff140200700 1 --2- 172.21.15.45:0/2982897999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff138076a70 0x7ff13819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.887+0000 7ff140200700 1 -- 172.21.15.45:0/2982897999 >> 172.21.15.45:0/2982897999 conn(0x7ff1380fe010 msgr2=0x7ff138106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.887+0000 7ff140200700 1 -- 172.21.15.45:0/2982897999 shutdown_connections 2024-04-25T18:54:10.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:10.887+0000 7ff140200700 1 -- 172.21.15.45:0/2982897999 wait complete. 2024-04-25T18:54:11.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: osd.11 [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] boot 2024-04-25T18:54:11.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T18:54:11.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:11.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:11.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:11.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:10 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2982897999' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: osd.11 [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] boot 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:11.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.216 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:10 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2982897999' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: osd.11 [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] boot 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:11.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:11.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:11.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:10 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2982897999' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T18:54:11.745 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":84,"num_osds":12,"num_up_osds":12,"osd_up_since":1714071249,"num_in_osds":12,"osd_in_since":1714071229,"num_remapped_pgs":0} 2024-04-25T18:54:11.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:54:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:11 smithi079 ceph-mon[33002]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:11 smithi079 ceph-mon[33002]: Detected new or changed devices on smithi079 2024-04-25T18:54:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:11 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi079 to 4272M 2024-04-25T18:54:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:11 smithi079 ceph-mon[33002]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T18:54:12.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:11 smithi045 ceph-mon[27776]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:12.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:11 smithi045 ceph-mon[27776]: Detected new or changed devices on smithi079 2024-04-25T18:54:12.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:11 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi079 to 4272M 2024-04-25T18:54:12.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:11 smithi045 ceph-mon[27776]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T18:54:12.227 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:12.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:11 smithi062 ceph-mon[32844]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 983 GiB / 983 GiB avail 2024-04-25T18:54:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:11 smithi062 ceph-mon[32844]: Detected new or changed devices on smithi079 2024-04-25T18:54:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:11 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi079 to 4272M 2024-04-25T18:54:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:11 smithi062 ceph-mon[32844]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T18:54:14.160 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:13 smithi045 ceph-mon[27776]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.330+0000 7fdfce200700 1 -- 172.21.15.45:0/2688155449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc8074c70 msgr2=0x7fdfc8075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.330+0000 7fdfce200700 1 --2- 172.21.15.45:0/2688155449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc8074c70 0x7fdfc8075050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8009990 tx=0x7fdfb8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.330+0000 7fdfce200700 1 -- 172.21.15.45:0/2688155449 shutdown_connections 2024-04-25T18:54:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.330+0000 7fdfce200700 1 --2- 172.21.15.45:0/2688155449 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc810f440 0x7fdfc8111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.330+0000 7fdfce200700 1 --2- 172.21.15.45:0/2688155449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc8074c70 0x7fdfc8075050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.330+0000 7fdfce200700 1 --2- 172.21.15.45:0/2688155449 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc8075590 0x7fdfc810ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.330+0000 7fdfce200700 1 -- 172.21.15.45:0/2688155449 >> 172.21.15.45:0/2688155449 conn(0x7fdfc8079ba0 msgr2=0x7fdfc8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.330+0000 7fdfce200700 1 -- 172.21.15.45:0/2688155449 shutdown_connections 2024-04-25T18:54:14.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.330+0000 7fdfce200700 1 -- 172.21.15.45:0/2688155449 wait complete. 2024-04-25T18:54:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.332+0000 7fdfce200700 1 Processor -- start 2024-04-25T18:54:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.332+0000 7fdfce200700 1 -- start start 2024-04-25T18:54:14.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.332+0000 7fdfce200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc8074c70 0x7fdfc819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:14.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfce200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc8075590 0x7fdfc819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:14.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfce200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc810f440 0x7fdfc819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:14.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfce200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8114e40 con 0x7fdfc810f440 2024-04-25T18:54:14.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfce200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8114cc0 con 0x7fdfc8074c70 2024-04-25T18:54:14.335 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfce200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc8114fc0 con 0x7fdfc8075590 2024-04-25T18:54:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfc7e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc810f440 0x7fdfc819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfc7e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc810f440 0x7fdfc819fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55566/0 (socket says 172.21.15.45:55566) 2024-04-25T18:54:14.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfc7e00700 1 -- 172.21.15.45:0/3903909200 learned_addr learned my addr 172.21.15.45:0/3903909200 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfbea00700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc8075590 0x7fdfc819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfc7400700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc8074c70 0x7fdfc819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfc7e00700 1 -- 172.21.15.45:0/3903909200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc8075590 msgr2=0x7fdfc819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfc7e00700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc8075590 0x7fdfc819b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfc7e00700 1 -- 172.21.15.45:0/3903909200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc8074c70 msgr2=0x7fdfc819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfc7e00700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc8074c70 0x7fdfc819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfc7e00700 1 -- 172.21.15.45:0/3903909200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc81a02e0 con 0x7fdfc810f440 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.333+0000 7fdfbea00700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc8075590 0x7fdfc819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.334+0000 7fdfc7e00700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc810f440 0x7fdfc819fba0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000ef90 tx=0x7fdfc000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.334+0000 7fdfc4c00700 1 -- 172.21.15.45:0/3903909200 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc00092e0 con 0x7fdfc810f440 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.334+0000 7fdfc4c00700 1 -- 172.21.15.45:0/3903909200 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfc0004bb0 con 0x7fdfc810f440 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.334+0000 7fdfce200700 1 -- 172.21.15.45:0/3903909200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc81a0570 con 0x7fdfc810f440 2024-04-25T18:54:14.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.334+0000 7fdfc4c00700 1 -- 172.21.15.45:0/3903909200 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc0004280 con 0x7fdfc810f440 2024-04-25T18:54:14.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.334+0000 7fdfce200700 1 -- 172.21.15.45:0/3903909200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc81a7e50 con 0x7fdfc810f440 2024-04-25T18:54:14.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.335+0000 7fdfce200700 1 -- 172.21.15.45:0/3903909200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc8076360 con 0x7fdfc810f440 2024-04-25T18:54:14.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.337+0000 7fdfc4c00700 1 -- 172.21.15.45:0/3903909200 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdfc001b030 con 0x7fdfc810f440 2024-04-25T18:54:14.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.340+0000 7fdfc4c00700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfac0a3c40 0x7fdfac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:14.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.340+0000 7fdfc4c00700 1 -- 172.21.15.45:0/3903909200 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fdfc00cfaa0 con 0x7fdfc810f440 2024-04-25T18:54:14.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.340+0000 7fdfc7400700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfac0a3c40 0x7fdfac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:14.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.341+0000 7fdfc7400700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfac0a3c40 0x7fdfac0a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006010 tx=0x7fdfb800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:14.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.347+0000 7fdfc4c00700 1 -- 172.21.15.45:0/3903909200 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfc0022020 con 0x7fdfc810f440 2024-04-25T18:54:14.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:13 smithi062 ceph-mon[32844]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:14.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:13 smithi079 ceph-mon[33002]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:14.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.584+0000 7fdfce200700 1 -- 172.21.15.45:0/3903909200 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfc802df80 con 0x7fdfc810f440 2024-04-25T18:54:14.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.587+0000 7fdfc4c00700 1 -- 172.21.15.45:0/3903909200 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+18361 (secure 0 0 0) 0x7fdfc0010040 con 0x7fdfc810f440 2024-04-25T18:54:14.590 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:54:14.590 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":85,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T18:54:11.530366+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.590+0000 7fdfce200700 1 -- 172.21.15.45:0/3903909200 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfac0a3c40 msgr2=0x7fdfac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.590+0000 7fdfce200700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfac0a3c40 0x7fdfac0a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006010 tx=0x7fdfb800b260 comp rx=0 tx=0).stop 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.590+0000 7fdfce200700 1 -- 172.21.15.45:0/3903909200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc810f440 msgr2=0x7fdfc819fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.590+0000 7fdfce200700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc810f440 0x7fdfc819fba0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdfc000ef90 tx=0x7fdfc000c3a0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.591+0000 7fdfce200700 1 -- 172.21.15.45:0/3903909200 shutdown_connections 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.591+0000 7fdfce200700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfac0a3c40 0x7fdfac0a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.591+0000 7fdfce200700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdfc8074c70 0x7fdfc819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.591+0000 7fdfce200700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfc810f440 0x7fdfc819fba0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.591+0000 7fdfce200700 1 --2- 172.21.15.45:0/3903909200 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdfc8075590 0x7fdfc819b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.591+0000 7fdfce200700 1 -- 172.21.15.45:0/3903909200 >> 172.21.15.45:0/3903909200 conn(0x7fdfc8079ba0 msgr2=0x7fdfc80ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.591+0000 7fdfce200700 1 -- 172.21.15.45:0/3903909200 shutdown_connections 2024-04-25T18:54:14.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:14.591+0000 7fdfce200700 1 -- 172.21.15.45:0/3903909200 wait complete. 2024-04-25T18:54:15.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:14 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3903909200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:54:15.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:14 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3903909200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:54:15.375 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T18:50:00.259702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T18:54:15.376 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T18:54:15.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:14 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3903909200' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:54:15.845 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:16.206 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:15 smithi045 ceph-mon[27776]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:16.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:15 smithi062 ceph-mon[32844]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:16.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:15 smithi079 ceph-mon[33002]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:17.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.893+0000 7f1b45a00700 1 -- 172.21.15.45:0/3228736553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b40108950 msgr2=0x7f1b40108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:17.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.893+0000 7f1b45a00700 1 --2- 172.21.15.45:0/3228736553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b40108950 0x7f1b40108d30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009990 tx=0x7f1b30009ca0 comp rx=0 tx=0).stop 2024-04-25T18:54:17.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.893+0000 7f1b45a00700 1 -- 172.21.15.45:0/3228736553 shutdown_connections 2024-04-25T18:54:17.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.893+0000 7f1b45a00700 1 --2- 172.21.15.45:0/3228736553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b40103310 0x7f1b4010f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:17.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.893+0000 7f1b45a00700 1 --2- 172.21.15.45:0/3228736553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b40108950 0x7f1b40108d30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:17.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.893+0000 7f1b45a00700 1 --2- 172.21.15.45:0/3228736553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b40102950 0x7f1b40102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:17.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.893+0000 7f1b45a00700 1 -- 172.21.15.45:0/3228736553 >> 172.21.15.45:0/3228736553 conn(0x7f1b400fe460 msgr2=0x7f1b40100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:17.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.893+0000 7f1b45a00700 1 -- 172.21.15.45:0/3228736553 shutdown_connections 2024-04-25T18:54:17.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.893+0000 7f1b45a00700 1 -- 172.21.15.45:0/3228736553 wait complete. 2024-04-25T18:54:17.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.894+0000 7f1b45a00700 1 Processor -- start 2024-04-25T18:54:17.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.894+0000 7f1b45a00700 1 -- start start 2024-04-25T18:54:17.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b45a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b40102950 0x7f1b40074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:17.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b45a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b40103310 0x7f1b40075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:17.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b45a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b40108950 0x7f1b400757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:17.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b45a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40112b30 con 0x7f1b40108950 2024-04-25T18:54:17.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b45a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401129b0 con 0x7f1b40103310 2024-04-25T18:54:17.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b45a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40112cb0 con 0x7f1b40102950 2024-04-25T18:54:17.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b45000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b40108950 0x7f1b400757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:17.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b3f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b40103310 0x7f1b40075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:17.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b45000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b40108950 0x7f1b400757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37982/0 (socket says 172.21.15.45:37982) 2024-04-25T18:54:17.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b3f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b40103310 0x7f1b40075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36214/0 (socket says 172.21.15.45:36214) 2024-04-25T18:54:17.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.895+0000 7f1b45000700 1 -- 172.21.15.45:0/216439362 learned_addr learned my addr 172.21.15.45:0/216439362 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:54:17.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.896+0000 7f1b3fe00700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b40102950 0x7f1b40074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:17.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.896+0000 7f1b3f400700 1 -- 172.21.15.45:0/216439362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b40102950 msgr2=0x7f1b40074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:17.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.896+0000 7f1b3f400700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b40102950 0x7f1b40074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:17.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.896+0000 7f1b3f400700 1 -- 172.21.15.45:0/216439362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b40108950 msgr2=0x7f1b400757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:17.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.896+0000 7f1b3f400700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b40108950 0x7f1b400757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.896+0000 7f1b3f400700 1 -- 172.21.15.45:0/216439362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b401a7570 con 0x7f1b40103310 2024-04-25T18:54:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.896+0000 7f1b45000700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b40108950 0x7f1b400757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.896+0000 7f1b3f400700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b40103310 0x7f1b40075280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e9f0 tx=0x7f1b2c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.897+0000 7f1b3cc00700 1 -- 172.21.15.45:0/216439362 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c00c9c0 con 0x7f1b40103310 2024-04-25T18:54:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.897+0000 7f1b45a00700 1 -- 172.21.15.45:0/216439362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b401a7860 con 0x7f1b40103310 2024-04-25T18:54:17.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.897+0000 7f1b3cc00700 1 -- 172.21.15.45:0/216439362 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b2c00cb60 con 0x7f1b40103310 2024-04-25T18:54:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.897+0000 7f1b3cc00700 1 -- 172.21.15.45:0/216439362 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c018680 con 0x7f1b40103310 2024-04-25T18:54:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.897+0000 7f1b45a00700 1 -- 172.21.15.45:0/216439362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b401a7df0 con 0x7f1b40103310 2024-04-25T18:54:17.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.898+0000 7f1b45a00700 1 -- 172.21.15.45:0/216439362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b4010aee0 con 0x7f1b40103310 2024-04-25T18:54:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.900+0000 7f1b3cc00700 1 -- 172.21.15.45:0/216439362 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1b2c010430 con 0x7f1b40103310 2024-04-25T18:54:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.901+0000 7f1b3cc00700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1b240a3c40 0x7f1b240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.902+0000 7f1b3cc00700 1 -- 172.21.15.45:0/216439362 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f1b2c0d0280 con 0x7f1b40103310 2024-04-25T18:54:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.902+0000 7f1b3fe00700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1b240a3c40 0x7f1b240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:17.907 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.902+0000 7f1b3fe00700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1b240a3c40 0x7f1b240a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000b2e0 tx=0x7f1b30005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:17.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:17.904+0000 7f1b3cc00700 1 -- 172.21.15.45:0/216439362 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b2c09d060 con 0x7f1b40103310 2024-04-25T18:54:18.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:17 smithi062 ceph-mon[32844]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:18.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.134+0000 7f1b45a00700 1 -- 172.21.15.45:0/216439362 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b40069730 con 0x7f1b40103310 2024-04-25T18:54:18.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:17 smithi045 ceph-mon[27776]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:18.137 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.135+0000 7f1b3cc00700 1 -- 172.21.15.45:0/216439362 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) ==== 93+0+10 (secure 0 0 0) 0x7f1b2c09c960 con 0x7f1b40103310 2024-04-25T18:54:18.138 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T18:54:18.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 -- 172.21.15.45:0/216439362 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1b240a3c40 msgr2=0x7f1b240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:18.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1b240a3c40 0x7f1b240a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000b2e0 tx=0x7f1b30005960 comp rx=0 tx=0).stop 2024-04-25T18:54:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 -- 172.21.15.45:0/216439362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b40103310 msgr2=0x7f1b40075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b40103310 0x7f1b40075280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00e9f0 tx=0x7f1b2c00ed00 comp rx=0 tx=0).stop 2024-04-25T18:54:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 -- 172.21.15.45:0/216439362 shutdown_connections 2024-04-25T18:54:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f1b240a3c40 0x7f1b240a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1b40108950 0x7f1b400757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b40102950 0x7f1b40074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 --2- 172.21.15.45:0/216439362 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1b40103310 0x7f1b40075280 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:18.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 -- 172.21.15.45:0/216439362 >> 172.21.15.45:0/216439362 conn(0x7f1b400fe460 msgr2=0x7f1b400fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:18.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 -- 172.21.15.45:0/216439362 shutdown_connections 2024-04-25T18:54:18.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:18.138+0000 7f1b45a00700 1 -- 172.21.15.45:0/216439362 wait complete. 2024-04-25T18:54:18.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:17 smithi079 ceph-mon[33002]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:18.840 INFO:tasks.cephadm:Adding mds.a on smithi045 2024-04-25T18:54:18.840 INFO:tasks.cephadm:Adding mds.d on smithi045 2024-04-25T18:54:18.840 INFO:tasks.cephadm:Adding mds.g on smithi045 2024-04-25T18:54:18.840 INFO:tasks.cephadm:Adding mds.j on smithi045 2024-04-25T18:54:18.840 INFO:tasks.cephadm:Adding mds.b on smithi062 2024-04-25T18:54:18.840 INFO:tasks.cephadm:Adding mds.e on smithi062 2024-04-25T18:54:18.840 INFO:tasks.cephadm:Adding mds.h on smithi062 2024-04-25T18:54:18.841 INFO:tasks.cephadm:Adding mds.k on smithi062 2024-04-25T18:54:18.841 INFO:tasks.cephadm:Adding mds.c on smithi079 2024-04-25T18:54:18.841 INFO:tasks.cephadm:Adding mds.f on smithi079 2024-04-25T18:54:18.841 INFO:tasks.cephadm:Adding mds.i on smithi079 2024-04-25T18:54:18.841 INFO:tasks.cephadm:Adding mds.l on smithi079 2024-04-25T18:54:18.841 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l' 2024-04-25T18:54:19.094 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T18:54:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:18 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/216439362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:54:19.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:18 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/216439362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:54:19.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:18 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/216439362' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T18:54:20.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:19 smithi045 ceph-mon[27776]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:20.296 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:19 smithi079 ceph-mon[33002]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:19 smithi062 ceph-mon[32844]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.641+0000 7fdffd511700 1 -- 172.21.15.79:0/763210977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdff8105e80 msgr2=0x7fdff8108270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.641+0000 7fdffd511700 1 --2- 172.21.15.79:0/763210977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdff8105e80 0x7fdff8108270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdfec009990 tx=0x7fdfec009ca0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.641+0000 7fdffd511700 1 -- 172.21.15.79:0/763210977 shutdown_connections 2024-04-25T18:54:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.641+0000 7fdffd511700 1 --2- 172.21.15.79:0/763210977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff81087b0 0x7fdff810ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.641+0000 7fdffd511700 1 --2- 172.21.15.79:0/763210977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdff8103550 0x7fdff8105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.641+0000 7fdffd511700 1 --2- 172.21.15.79:0/763210977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdff8105e80 0x7fdff8108270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.641+0000 7fdffd511700 1 -- 172.21.15.79:0/763210977 >> 172.21.15.79:0/763210977 conn(0x7fdff80fcfa0 msgr2=0x7fdff80ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.641+0000 7fdffd511700 1 -- 172.21.15.79:0/763210977 shutdown_connections 2024-04-25T18:54:20.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.641+0000 7fdffd511700 1 -- 172.21.15.79:0/763210977 wait complete. 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.642+0000 7fdffd511700 1 Processor -- start 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.642+0000 7fdffd511700 1 -- start start 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.642+0000 7fdffd511700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdff8103550 0x7fdff8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.642+0000 7fdffd511700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff8105e80 0x7fdff81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.642+0000 7fdffd511700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdff81087b0 0x7fdff81a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.642+0000 7fdffd511700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdff810e640 con 0x7fdff81087b0 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.642+0000 7fdffd511700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdff810e4c0 con 0x7fdff8105e80 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.642+0000 7fdffd511700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdff810e7c0 con 0x7fdff8103550 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.642+0000 7fdff6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdff8103550 0x7fdff8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdff6ffd700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdff8103550 0x7fdff8198db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57278/0 (socket says 172.21.15.79:57278) 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdff6ffd700 1 -- 172.21.15.79:0/3581273558 learned_addr learned my addr 172.21.15.79:0/3581273558 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdff67fc700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff8105e80 0x7fdff81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdff6ffd700 1 -- 172.21.15.79:0/3581273558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff8105e80 msgr2=0x7fdff81992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdff6ffd700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff8105e80 0x7fdff81992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdff6ffd700 1 -- 172.21.15.79:0/3581273558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdff81087b0 msgr2=0x7fdff81a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:20.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdff6ffd700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdff81087b0 0x7fdff81a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdff6ffd700 1 -- 172.21.15.79:0/3581273558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff81a09c0 con 0x7fdff8103550 2024-04-25T18:54:20.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdff6ffd700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdff8103550 0x7fdff8198db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b560 tx=0x7fdfe800b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:20.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdfdffff700 1 -- 172.21.15.79:0/3581273558 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8015020 con 0x7fdff8103550 2024-04-25T18:54:20.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdffd511700 1 -- 172.21.15.79:0/3581273558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff81a0cb0 con 0x7fdff8103550 2024-04-25T18:54:20.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdfdffff700 1 -- 172.21.15.79:0/3581273558 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfe8011ce0 con 0x7fdff8103550 2024-04-25T18:54:20.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdfdffff700 1 -- 172.21.15.79:0/3581273558 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8010d20 con 0x7fdff8103550 2024-04-25T18:54:20.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.643+0000 7fdffd511700 1 -- 172.21.15.79:0/3581273558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff81a1210 con 0x7fdff8103550 2024-04-25T18:54:20.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.644+0000 7fdffd511700 1 -- 172.21.15.79:0/3581273558 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff802df80 con 0x7fdff8103550 2024-04-25T18:54:20.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.645+0000 7fdfdffff700 1 -- 172.21.15.79:0/3581273558 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdfe8018020 con 0x7fdff8103550 2024-04-25T18:54:20.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.646+0000 7fdfdffff700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfe00a3c40 0x7fdfe00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:20.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.646+0000 7fdfdffff700 1 -- 172.21.15.79:0/3581273558 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fdfe80cf6e0 con 0x7fdff8103550 2024-04-25T18:54:20.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.646+0000 7fdff67fc700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfe00a3c40 0x7fdfe00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:20.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.647+0000 7fdff67fc700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfe00a3c40 0x7fdfe00a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00b5c0 tx=0x7fdfec005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:20.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.650+0000 7fdfdffff700 1 -- 172.21.15.79:0/3581273558 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfe801f020 con 0x7fdff8103550 2024-04-25T18:54:20.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.857+0000 7fdffd511700 1 -- 172.21.15.79:0/3581273558 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}) -- 0x7fdff8062f80 con 0x7fdfe00a3c40 2024-04-25T18:54:20.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.873+0000 7fdfdffff700 1 -- 172.21.15.79:0/3581273558 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fdff8062f80 con 0x7fdfe00a3c40 2024-04-25T18:54:20.875 INFO:teuthology.orchestra.run.smithi079.stdout:Scheduled mds.all update... 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 -- 172.21.15.79:0/3581273558 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfe00a3c40 msgr2=0x7fdfe00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfe00a3c40 0x7fdfe00a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00b5c0 tx=0x7fdfec005fb0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 -- 172.21.15.79:0/3581273558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdff8103550 msgr2=0x7fdff8198db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdff8103550 0x7fdff8198db0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b560 tx=0x7fdfe800b870 comp rx=0 tx=0).stop 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 -- 172.21.15.79:0/3581273558 shutdown_connections 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdfe00a3c40 0x7fdfe00a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdff81087b0 0x7fdff81a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdff8103550 0x7fdff8198db0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 --2- 172.21.15.79:0/3581273558 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdff8105e80 0x7fdff81992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 -- 172.21.15.79:0/3581273558 >> 172.21.15.79:0/3581273558 conn(0x7fdff80fcfa0 msgr2=0x7fdff8103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 -- 172.21.15.79:0/3581273558 shutdown_connections 2024-04-25T18:54:20.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-25T18:54:20.876+0000 7fdffd511700 1 -- 172.21.15.79:0/3581273558 wait complete. 2024-04-25T18:54:21.487 DEBUG:teuthology.orchestra.run.smithi045:mds.a> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T18:54:21.495 DEBUG:teuthology.orchestra.run.smithi045:mds.d> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T18:54:21.541 DEBUG:teuthology.orchestra.run.smithi045:mds.g> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T18:54:21.588 DEBUG:teuthology.orchestra.run.smithi045:mds.j> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T18:54:21.634 DEBUG:teuthology.orchestra.run.smithi062:mds.b> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T18:54:21.636 DEBUG:teuthology.orchestra.run.smithi062:mds.e> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T18:54:21.680 DEBUG:teuthology.orchestra.run.smithi062:mds.h> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T18:54:21.723 DEBUG:teuthology.orchestra.run.smithi062:mds.k> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T18:54:21.766 DEBUG:teuthology.orchestra.run.smithi079:mds.c> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T18:54:21.768 DEBUG:teuthology.orchestra.run.smithi079:mds.f> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T18:54:21.811 DEBUG:teuthology.orchestra.run.smithi079:mds.i> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T18:54:21.854 DEBUG:teuthology.orchestra.run.smithi079:mds.l> sudo journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T18:54:21.897 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T18:54:21.897 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:54:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:21 smithi062 ceph-mon[32844]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:22.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:21 smithi062 ceph-mon[32844]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:54:22.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:21 smithi062 ceph-mon[32844]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-04-25T18:54:22.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:21 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:22.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:21 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:21 smithi079 ceph-mon[33002]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:21 smithi079 ceph-mon[33002]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:54:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:21 smithi079 ceph-mon[33002]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-04-25T18:54:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:21 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:21 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:22.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:21 smithi045 ceph-mon[27776]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:22.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:21 smithi045 ceph-mon[27776]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:54:22.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:21 smithi045 ceph-mon[27776]: Saving service mds.all spec with placement smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-04-25T18:54:22.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:21 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:22.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:21 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:54:22.347 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:23.037 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:23.038 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:23.038 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:23.038 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:23.038 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:23.038 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:22 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:23.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:23.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:23.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:23.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:23.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:23.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:22 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:23.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:23.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:54:23.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:23.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:23.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:23.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:22 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:23 smithi045 ceph-mon[27776]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:24.143 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:23 smithi045 ceph-mon[27776]: Deploying daemon mds.l on smithi079 2024-04-25T18:54:24.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:23 smithi079 ceph-mon[33002]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:24.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:23 smithi079 ceph-mon[33002]: Deploying daemon mds.l on smithi079 2024-04-25T18:54:24.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:23 smithi062 ceph-mon[32844]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:24.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:23 smithi062 ceph-mon[32844]: Deploying daemon mds.l on smithi079 2024-04-25T18:54:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.438+0000 7f55c9c00700 1 -- 172.21.15.45:0/3810310994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c4108a50 msgr2=0x7f55c4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.438+0000 7f55c9c00700 1 --2- 172.21.15.45:0/3810310994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c4108a50 0x7f55c4108e30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009990 tx=0x7f55b4009ca0 comp rx=0 tx=0).stop 2024-04-25T18:54:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.438+0000 7f55c9c00700 1 -- 172.21.15.45:0/3810310994 shutdown_connections 2024-04-25T18:54:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.438+0000 7f55c9c00700 1 --2- 172.21.15.45:0/3810310994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c4108a50 0x7f55c4108e30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.438+0000 7f55c9c00700 1 --2- 172.21.15.45:0/3810310994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4103450 0x7f55c410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.438+0000 7f55c9c00700 1 --2- 172.21.15.45:0/3810310994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4102a90 0x7f55c4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:24.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.438+0000 7f55c9c00700 1 -- 172.21.15.45:0/3810310994 >> 172.21.15.45:0/3810310994 conn(0x7f55c4079ba0 msgr2=0x7f55c4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.439+0000 7f55c9c00700 1 -- 172.21.15.45:0/3810310994 shutdown_connections 2024-04-25T18:54:24.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.439+0000 7f55c9c00700 1 -- 172.21.15.45:0/3810310994 wait complete. 2024-04-25T18:54:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.440+0000 7f55c9c00700 1 Processor -- start 2024-04-25T18:54:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.440+0000 7f55c9c00700 1 -- start start 2024-04-25T18:54:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.440+0000 7f55c9c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4102a90 0x7f55c4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.440+0000 7f55c9c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c4103450 0x7f55c4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.440+0000 7f55c9c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4108a50 0x7f55c40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.440+0000 7f55c9c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4112be0 con 0x7f55c4102a90 2024-04-25T18:54:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.440+0000 7f55c9c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4112a60 con 0x7f55c4103450 2024-04-25T18:54:24.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c9c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f55c4112d60 con 0x7f55c4108a50 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4102a90 0x7f55c4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c2000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c4103450 0x7f55c4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4102a90 0x7f55c4074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37998/0 (socket says 172.21.15.45:37998) 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c2a00700 1 -- 172.21.15.45:0/1516494570 learned_addr learned my addr 172.21.15.45:0/1516494570 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c3400700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4108a50 0x7f55c40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c2000700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c4103450 0x7f55c4075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36246/0 (socket says 172.21.15.45:36246) 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c2000700 1 -- 172.21.15.45:0/1516494570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4108a50 msgr2=0x7f55c40757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c2000700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4108a50 0x7f55c40757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c2000700 1 -- 172.21.15.45:0/1516494570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4102a90 msgr2=0x7f55c4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c2000700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4102a90 0x7f55c4074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:24.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.441+0000 7f55c2000700 1 -- 172.21.15.45:0/1516494570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c41a7700 con 0x7f55c4103450 2024-04-25T18:54:24.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.442+0000 7f55c2000700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c4103450 0x7f55c4075280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e9f0 tx=0x7f55b000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:24.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.442+0000 7f55bb400700 1 -- 172.21.15.45:0/1516494570 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b000c9c0 con 0x7f55c4103450 2024-04-25T18:54:24.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.442+0000 7f55bb400700 1 -- 172.21.15.45:0/1516494570 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55b000cb60 con 0x7f55c4103450 2024-04-25T18:54:24.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.442+0000 7f55bb400700 1 -- 172.21.15.45:0/1516494570 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0018680 con 0x7f55c4103450 2024-04-25T18:54:24.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.442+0000 7f55c9c00700 1 -- 172.21.15.45:0/1516494570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c41a79f0 con 0x7f55c4103450 2024-04-25T18:54:24.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.442+0000 7f55c9c00700 1 -- 172.21.15.45:0/1516494570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41a7ee0 con 0x7f55c4103450 2024-04-25T18:54:24.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.443+0000 7f55c9c00700 1 -- 172.21.15.45:0/1516494570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c4069730 con 0x7f55c4103450 2024-04-25T18:54:24.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.446+0000 7f55bb400700 1 -- 172.21.15.45:0/1516494570 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f55b0018a30 con 0x7f55c4103450 2024-04-25T18:54:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.447+0000 7f55bb400700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f55a80a3cc0 0x7f55a80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.447+0000 7f55bb400700 1 -- 172.21.15.45:0/1516494570 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f55b00d02e0 con 0x7f55c4103450 2024-04-25T18:54:24.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.448+0000 7f55c2a00700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f55a80a3cc0 0x7f55a80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:24.450 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.448+0000 7f55c2a00700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f55a80a3cc0 0x7f55a80a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f55b400b2e0 tx=0x7f55b4009510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:24.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.451+0000 7f55bb400700 1 -- 172.21.15.45:0/1516494570 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f55b009d0c0 con 0x7f55c4103450 2024-04-25T18:54:24.578 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 18:54:24 smithi079 systemd[1]: Starting Ceph mds.l for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:54:24.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.736+0000 7f55c9c00700 1 -- 172.21.15.45:0/1516494570 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55c402e140 con 0x7f55c4103450 2024-04-25T18:54:24.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.737+0000 7f55bb400700 1 -- 172.21.15.45:0/1516494570 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f55b009c9c0 con 0x7f55c4103450 2024-04-25T18:54:24.739 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:54:24.739 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T18:54:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 -- 172.21.15.45:0/1516494570 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f55a80a3cc0 msgr2=0x7f55a80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f55a80a3cc0 0x7f55a80a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f55b400b2e0 tx=0x7f55b4009510 comp rx=0 tx=0).stop 2024-04-25T18:54:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 -- 172.21.15.45:0/1516494570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c4103450 msgr2=0x7f55c4075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c4103450 0x7f55c4075280 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f55b000e9f0 tx=0x7f55b000ed00 comp rx=0 tx=0).stop 2024-04-25T18:54:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 -- 172.21.15.45:0/1516494570 shutdown_connections 2024-04-25T18:54:24.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f55a80a3cc0 0x7f55a80a6180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f55c4108a50 0x7f55c40757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f55c4102a90 0x7f55c4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 --2- 172.21.15.45:0/1516494570 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f55c4103450 0x7f55c4075280 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 -- 172.21.15.45:0/1516494570 >> 172.21.15.45:0/1516494570 conn(0x7f55c4079ba0 msgr2=0x7f55c4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 -- 172.21.15.45:0/1516494570 shutdown_connections 2024-04-25T18:54:24.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:24.740+0000 7f55c9c00700 1 -- 172.21.15.45:0/1516494570 wait complete. 2024-04-25T18:54:24.744 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 1 2024-04-25T18:54:24.830 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 18:54:24 smithi079 podman[63354]: 2024-04-25 18:54:24.480318609 +0000 UTC m=+0.017488411 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:54:24.831 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 18:54:24 smithi079 podman[63354]: 2024-04-25 18:54:24.593186308 +0000 UTC m=+0.130356086 container create b7ee805eec8b7d6560eed48e49d66ff50548c4ce2983e0dc4b6ff9a1225760f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T18:54:24.831 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 18:54:24 smithi079 podman[63354]: 2024-04-25 18:54:24.804096542 +0000 UTC m=+0.341266329 container init b7ee805eec8b7d6560eed48e49d66ff50548c4ce2983e0dc4b6ff9a1225760f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T18:54:24.831 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 18:54:24 smithi079 podman[63354]: 2024-04-25 18:54:24.806645507 +0000 UTC m=+0.343815294 container start b7ee805eec8b7d6560eed48e49d66ff50548c4ce2983e0dc4b6ff9a1225760f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:54:25.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:24 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1516494570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:25.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:24 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.79:6832/990407858,v1:172.21.15.79:6833/990407858] up:boot 2024-04-25T18:54:25.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:24 smithi045 ceph-mon[27776]: fsmap 1 up:standby 2024-04-25T18:54:25.137 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:54:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:24 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1516494570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:24 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.79:6832/990407858,v1:172.21.15.79:6833/990407858] up:boot 2024-04-25T18:54:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:24 smithi079 ceph-mon[33002]: fsmap 1 up:standby 2024-04-25T18:54:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:54:25.184 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 18:54:24 smithi079 ceph-mds[63373]: -- 172.21.15.79:0/282854680 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5602c82b3a40 con 0x5602c8286c00 2024-04-25T18:54:25.184 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 18:54:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-l[63369]: starting mds.l at 2024-04-25T18:54:25.185 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 18:54:24 smithi079 bash[63354]: b7ee805eec8b7d6560eed48e49d66ff50548c4ce2983e0dc4b6ff9a1225760f9 2024-04-25T18:54:25.185 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 18:54:24 smithi079 systemd[1]: Started Ceph mds.l for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:54:25.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:24 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1516494570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:25.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:24 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.79:6832/990407858,v1:172.21.15.79:6833/990407858] up:boot 2024-04-25T18:54:25.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:24 smithi062 ceph-mon[32844]: fsmap 1 up:standby 2024-04-25T18:54:25.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:24 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:54:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:25 smithi045 ceph-mon[27776]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:25 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:26.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:25 smithi045 ceph-mon[27776]: Deploying daemon mds.k on smithi062 2024-04-25T18:54:26.342 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:25 smithi062 ceph-mon[32844]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:26.342 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:25 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:26.342 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:25 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:26.342 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:25 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:26.342 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:25 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:26.342 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:25 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:26.342 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:25 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:26.342 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:25 smithi062 ceph-mon[32844]: Deploying daemon mds.k on smithi062 2024-04-25T18:54:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:25 smithi079 ceph-mon[33002]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:26.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:25 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:26.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:25 smithi079 ceph-mon[33002]: Deploying daemon mds.k on smithi062 2024-04-25T18:54:27.364 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 18:54:27 smithi062 systemd[1]: Starting Ceph mds.k for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:54:27.552 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:54:27.774 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 18:54:27 smithi062 podman[62950]: 2024-04-25 18:54:27.502916016 +0000 UTC m=+0.103910890 container create 7e4a287086ff13620f40b51bf4921fd3e5bf6b19a1e4812e1ce6f9dce612d9ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T18:54:27.774 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 18:54:27 smithi062 podman[62950]: 2024-04-25 18:54:27.417465383 +0000 UTC m=+0.018460269 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:54:27.774 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 18:54:27 smithi062 podman[62950]: 2024-04-25 18:54:27.747158201 +0000 UTC m=+0.348153078 container init 7e4a287086ff13620f40b51bf4921fd3e5bf6b19a1e4812e1ce6f9dce612d9ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T18:54:27.775 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 18:54:27 smithi062 podman[62950]: 2024-04-25 18:54:27.749720431 +0000 UTC m=+0.350715314 container start 7e4a287086ff13620f40b51bf4921fd3e5bf6b19a1e4812e1ce6f9dce612d9ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-k, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T18:54:28.021 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:28.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:27 smithi062 ceph-mon[32844]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:28.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:28.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:28.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:28.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:28.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:28.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:27 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:28.115 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 18:54:27 smithi062 ceph-mds[62969]: -- 172.21.15.62:0/2493295179 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559307a39a40 con 0x559307a0cc00 2024-04-25T18:54:28.115 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 18:54:27 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-k[62965]: starting mds.k at 2024-04-25T18:54:28.115 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 18:54:27 smithi062 bash[62950]: 7e4a287086ff13620f40b51bf4921fd3e5bf6b19a1e4812e1ce6f9dce612d9ec 2024-04-25T18:54:28.115 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 18:54:27 smithi062 systemd[1]: Started Ceph mds.k for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:54:28.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:27 smithi045 ceph-mon[27776]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:28.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:28.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:28.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:28.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:28.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:28.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:27 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:27 smithi079 ceph-mon[33002]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:27 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:29.270 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:28 smithi045 ceph-mon[27776]: Deploying daemon mds.j on smithi045 2024-04-25T18:54:29.270 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:28 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.62:6832/1072807299,v1:172.21.15.62:6833/1072807299] up:boot 2024-04-25T18:54:29.270 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:28 smithi045 ceph-mon[27776]: fsmap 2 up:standby 2024-04-25T18:54:29.271 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:28 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:54:29.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:28 smithi062 ceph-mon[32844]: Deploying daemon mds.j on smithi045 2024-04-25T18:54:29.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:28 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.62:6832/1072807299,v1:172.21.15.62:6833/1072807299] up:boot 2024-04-25T18:54:29.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:28 smithi062 ceph-mon[32844]: fsmap 2 up:standby 2024-04-25T18:54:29.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:28 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:54:29.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:28 smithi079 ceph-mon[33002]: Deploying daemon mds.j on smithi045 2024-04-25T18:54:29.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:28 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.62:6832/1072807299,v1:172.21.15.62:6833/1072807299] up:boot 2024-04-25T18:54:29.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:28 smithi079 ceph-mon[33002]: fsmap 2 up:standby 2024-04-25T18:54:29.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:28 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:54:30.157 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:29 smithi062 ceph-mon[32844]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:30.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:29 smithi045 ceph-mon[27776]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:29 smithi079 ceph-mon[33002]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.948+0000 7fa642c00700 1 -- 172.21.15.45:0/775148689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa63c06b0f0 msgr2=0x7fa63c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.948+0000 7fa642c00700 1 --2- 172.21.15.45:0/775148689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa63c06b0f0 0x7fa63c10f900 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa63400b0a0 tx=0x7fa63400b3b0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.948+0000 7fa642c00700 1 -- 172.21.15.45:0/775148689 shutdown_connections 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.948+0000 7fa642c00700 1 --2- 172.21.15.45:0/775148689 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa63c06b0f0 0x7fa63c10f900 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.948+0000 7fa642c00700 1 --2- 172.21.15.45:0/775148689 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa63c108a30 0x7fa63c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.948+0000 7fa642c00700 1 --2- 172.21.15.45:0/775148689 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63c06a750 0x7fa63c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.948+0000 7fa642c00700 1 -- 172.21.15.45:0/775148689 >> 172.21.15.45:0/775148689 conn(0x7fa63c0788f0 msgr2=0x7fa63c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.948+0000 7fa642c00700 1 -- 172.21.15.45:0/775148689 shutdown_connections 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.948+0000 7fa642c00700 1 -- 172.21.15.45:0/775148689 wait complete. 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.949+0000 7fa642c00700 1 Processor -- start 2024-04-25T18:54:31.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.949+0000 7fa642c00700 1 -- start start 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa642c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63c06a750 0x7fa63c19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa642c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa63c06b0f0 0x7fa63c19b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa642c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa63c108a30 0x7fa63c19fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa642c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c112c60 con 0x7fa63c108a30 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa642c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c112ae0 con 0x7fa63c06b0f0 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa642c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa63c112de0 con 0x7fa63c06a750 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa63b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa63c06b0f0 0x7fa63c19b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa63b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa63c06b0f0 0x7fa63c19b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:54048/0 (socket says 172.21.15.45:54048) 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa63b400700 1 -- 172.21.15.45:0/632481936 learned_addr learned my addr 172.21.15.45:0/632481936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa640c00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa63c108a30 0x7fa63c19fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.950+0000 7fa63be00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63c06a750 0x7fa63c19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa63b400700 1 -- 172.21.15.45:0/632481936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63c06a750 msgr2=0x7fa63c19b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa63b400700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63c06a750 0x7fa63c19b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa63b400700 1 -- 172.21.15.45:0/632481936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa63c108a30 msgr2=0x7fa63c19fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:31.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa63b400700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa63c108a30 0x7fa63c19fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa63b400700 1 -- 172.21.15.45:0/632481936 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63c1a0310 con 0x7fa63c06b0f0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa63be00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63c06a750 0x7fa63c19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa63b400700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa63c06b0f0 0x7fa63c19b930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa62800e9c0 tx=0x7fa62800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa638c00700 1 -- 172.21.15.45:0/632481936 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62800cbd0 con 0x7fa63c06b0f0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa638c00700 1 -- 172.21.15.45:0/632481936 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62800cd70 con 0x7fa63c06b0f0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa638c00700 1 -- 172.21.15.45:0/632481936 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628005610 con 0x7fa63c06b0f0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.951+0000 7fa642c00700 1 -- 172.21.15.45:0/632481936 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63c1a0600 con 0x7fa63c06b0f0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.952+0000 7fa642c00700 1 -- 172.21.15.45:0/632481936 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa63c1a7e80 con 0x7fa63c06b0f0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.953+0000 7fa642c00700 1 -- 172.21.15.45:0/632481936 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa63c069730 con 0x7fa63c06b0f0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.955+0000 7fa638c00700 1 -- 172.21.15.45:0/632481936 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa628004750 con 0x7fa63c06b0f0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.956+0000 7fa638c00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa6200a3cc0 0x7fa6200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.956+0000 7fa638c00700 1 -- 172.21.15.45:0/632481936 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fa6280d02d0 con 0x7fa63c06b0f0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.956+0000 7fa63be00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa6200a3cc0 0x7fa6200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:31.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.957+0000 7fa63be00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa6200a3cc0 0x7fa6200a6180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009ca0 tx=0x7fa62c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:31.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:30.960+0000 7fa638c00700 1 -- 172.21.15.45:0/632481936 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa62809d0b0 con 0x7fa63c06b0f0 2024-04-25T18:54:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.284+0000 7fa642c00700 1 -- 172.21.15.45:0/632481936 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa63c02df80 con 0x7fa63c06b0f0 2024-04-25T18:54:31.288 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.285+0000 7fa638c00700 1 -- 172.21.15.45:0/632481936 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2215 (secure 0 0 0) 0x7fa62809c9b0 con 0x7fa63c06b0f0 2024-04-25T18:54:31.288 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:54:31.288 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T18:54:31.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.288+0000 7fa642c00700 1 -- 172.21.15.45:0/632481936 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa6200a3cc0 msgr2=0x7fa6200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:31.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.288+0000 7fa642c00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa6200a3cc0 0x7fa6200a6180 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009ca0 tx=0x7fa62c009260 comp rx=0 tx=0).stop 2024-04-25T18:54:31.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.288+0000 7fa642c00700 1 -- 172.21.15.45:0/632481936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa63c06b0f0 msgr2=0x7fa63c19b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:31.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.288+0000 7fa642c00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa63c06b0f0 0x7fa63c19b930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa62800e9c0 tx=0x7fa62800ecd0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.288+0000 7fa642c00700 1 -- 172.21.15.45:0/632481936 shutdown_connections 2024-04-25T18:54:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.288+0000 7fa642c00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa6200a3cc0 0x7fa6200a6180 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.288+0000 7fa642c00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa63c108a30 0x7fa63c19fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.288+0000 7fa642c00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa63c06a750 0x7fa63c19b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.288+0000 7fa642c00700 1 --2- 172.21.15.45:0/632481936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa63c06b0f0 0x7fa63c19b930 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.288+0000 7fa642c00700 1 -- 172.21.15.45:0/632481936 >> 172.21.15.45:0/632481936 conn(0x7fa63c0788f0 msgr2=0x7fa63c10df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.289+0000 7fa642c00700 1 -- 172.21.15.45:0/632481936 shutdown_connections 2024-04-25T18:54:31.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:31.289+0000 7fa642c00700 1 -- 172.21.15.45:0/632481936 wait complete. 2024-04-25T18:54:31.292 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 3 2024-04-25T18:54:32.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:31 smithi045 ceph-mon[27776]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:32.145 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:31 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/632481936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:32.236 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:31 smithi079 ceph-mon[33002]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:32.236 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:31 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/632481936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:32.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:31 smithi062 ceph-mon[32844]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:31 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/632481936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:32.801 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 18:54:32 smithi045 podman[65405]: 2024-04-25 18:54:32.543850326 +0000 UTC m=+0.160177795 container create 9f0469c61ac39a388a1913f761cec7b8e8e080d2c444c71928662ebe6ed7e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T18:54:32.801 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 18:54:32 smithi045 podman[65405]: 2024-04-25 18:54:32.797836549 +0000 UTC m=+0.414164025 container init 9f0469c61ac39a388a1913f761cec7b8e8e080d2c444c71928662ebe6ed7e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T18:54:33.213 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 18:54:32 smithi045 podman[65405]: 2024-04-25 18:54:32.809356403 +0000 UTC m=+0.425683906 container start 9f0469c61ac39a388a1913f761cec7b8e8e080d2c444c71928662ebe6ed7e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T18:54:33.214 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 18:54:32 smithi045 ceph-mds[65424]: -- 172.21.15.45:0/3463772256 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e0d0afa40 con 0x559e0d082800 2024-04-25T18:54:33.214 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 18:54:32 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-j[65420]: starting mds.j at 2024-04-25T18:54:33.214 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 18:54:32 smithi045 bash[65405]: 9f0469c61ac39a388a1913f761cec7b8e8e080d2c444c71928662ebe6ed7e08e 2024-04-25T18:54:33.214 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 18:54:32 smithi045 systemd[1]: Started Ceph mds.j for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:54:34.220 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:54:34.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:33 smithi045 ceph-mon[27776]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:34.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:33 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.45:6834/4023030153,v1:172.21.15.45:6835/4023030153] up:boot 2024-04-25T18:54:34.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:33 smithi045 ceph-mon[27776]: fsmap 3 up:standby 2024-04-25T18:54:34.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:33 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:54:34.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:33 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:34.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:33 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:34.304 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:33 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:34.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:33 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:34.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:33 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:34.305 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:33 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:34.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:33 smithi062 ceph-mon[32844]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:33 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.45:6834/4023030153,v1:172.21.15.45:6835/4023030153] up:boot 2024-04-25T18:54:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:33 smithi062 ceph-mon[32844]: fsmap 3 up:standby 2024-04-25T18:54:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:33 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:54:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:33 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:33 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:33 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:33 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:33 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:33 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:33 smithi079 ceph-mon[33002]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:33 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.45:6834/4023030153,v1:172.21.15.45:6835/4023030153] up:boot 2024-04-25T18:54:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:33 smithi079 ceph-mon[33002]: fsmap 3 up:standby 2024-04-25T18:54:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:33 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:54:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:33 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:34.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:33 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:34.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:33 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:34.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:33 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:34.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:33 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:34.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:33 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:35.199 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:34 smithi045 ceph-mon[27776]: Deploying daemon mds.d on smithi045 2024-04-25T18:54:35.293 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:35.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:34 smithi062 ceph-mon[32844]: Deploying daemon mds.d on smithi045 2024-04-25T18:54:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:34 smithi079 ceph-mon[33002]: Deploying daemon mds.d on smithi045 2024-04-25T18:54:36.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:35 smithi045 ceph-mon[27776]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:36.214 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 18:54:36 smithi045 systemd[1]: Starting Ceph mds.d for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:54:36.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:35 smithi062 ceph-mon[32844]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:35 smithi079 ceph-mon[33002]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:36.964 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 18:54:36 smithi045 podman[65798]: 2024-04-25 18:54:36.749535823 +0000 UTC m=+0.038702880 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:54:36.964 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 18:54:36 smithi045 podman[65798]: 2024-04-25 18:54:36.89027099 +0000 UTC m=+0.179438085 container create 571d4c0dbe360ab9f3a68dd2e40dd30297ac44eaef901a81ed93760b891a0f55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T18:54:37.609 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 18:54:37 smithi045 podman[65798]: 2024-04-25 18:54:37.362096994 +0000 UTC m=+0.651264052 container init 571d4c0dbe360ab9f3a68dd2e40dd30297ac44eaef901a81ed93760b891a0f55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T18:54:37.609 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 18:54:37 smithi045 podman[65798]: 2024-04-25 18:54:37.372298132 +0000 UTC m=+0.661465197 container start 571d4c0dbe360ab9f3a68dd2e40dd30297ac44eaef901a81ed93760b891a0f55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-d, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T18:54:37.609 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 18:54:37 smithi045 ceph-mds[65840]: -- 172.21.15.45:0/2673238530 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d6a17c5a40 con 0x55d6a1798800 2024-04-25T18:54:37.609 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 18:54:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-d[65832]: starting mds.d at 2024-04-25T18:54:37.609 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 18:54:37 smithi045 bash[65798]: 571d4c0dbe360ab9f3a68dd2e40dd30297ac44eaef901a81ed93760b891a0f55 2024-04-25T18:54:37.609 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 18:54:37 smithi045 systemd[1]: Started Ceph mds.d for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:54:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.782+0000 7fa55f200700 1 -- 172.21.15.45:0/3642587007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa560108a20 msgr2=0x7fa560108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:37.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.782+0000 7fa55f200700 1 --2- 172.21.15.45:0/3642587007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa560108a20 0x7fa560108e00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa55000b0a0 tx=0x7fa55000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:54:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.783+0000 7fa55f200700 1 -- 172.21.15.45:0/3642587007 shutdown_connections 2024-04-25T18:54:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.783+0000 7fa55f200700 1 --2- 172.21.15.45:0/3642587007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa56006b970 0x7fa56010f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.783+0000 7fa55f200700 1 --2- 172.21.15.45:0/3642587007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa560108a20 0x7fa560108e00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.783+0000 7fa55f200700 1 --2- 172.21.15.45:0/3642587007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56006afb0 0x7fa56006b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.783+0000 7fa55f200700 1 -- 172.21.15.45:0/3642587007 >> 172.21.15.45:0/3642587007 conn(0x7fa560078780 msgr2=0x7fa560078b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.783+0000 7fa55f200700 1 -- 172.21.15.45:0/3642587007 shutdown_connections 2024-04-25T18:54:37.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.783+0000 7fa55f200700 1 -- 172.21.15.45:0/3642587007 wait complete. 2024-04-25T18:54:37.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.784+0000 7fa55f200700 1 Processor -- start 2024-04-25T18:54:37.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.784+0000 7fa55f200700 1 -- start start 2024-04-25T18:54:37.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.784+0000 7fa55f200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa56006afb0 0x7fa56019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:37.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.784+0000 7fa55f200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56006b970 0x7fa56019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:37.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.784+0000 7fa55f200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa560108a20 0x7fa56019fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:37.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55f200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa560112ba0 con 0x7fa56006b970 2024-04-25T18:54:37.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55f200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa560112a20 con 0x7fa560108a20 2024-04-25T18:54:37.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55f200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa560112d20 con 0x7fa56006afb0 2024-04-25T18:54:37.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa555400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56006b970 0x7fa56019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:37.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55e800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa560108a20 0x7fa56019fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:37.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55e800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa560108a20 0x7fa56019fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41516/0 (socket says 172.21.15.45:41516) 2024-04-25T18:54:37.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55e800700 1 -- 172.21.15.45:0/1580587082 learned_addr learned my addr 172.21.15.45:0/1580587082 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:54:37.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa555400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56006b970 0x7fa56019b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57824/0 (socket says 172.21.15.45:57824) 2024-04-25T18:54:37.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55de00700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa56006afb0 0x7fa56019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:37.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55e800700 1 -- 172.21.15.45:0/1580587082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa56006afb0 msgr2=0x7fa56019b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:37.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55e800700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa56006afb0 0x7fa56019b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:37.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55e800700 1 -- 172.21.15.45:0/1580587082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56006b970 msgr2=0x7fa56019b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:37.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55e800700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56006b970 0x7fa56019b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:37.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55e800700 1 -- 172.21.15.45:0/1580587082 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5601a01b0 con 0x7fa560108a20 2024-04-25T18:54:37.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa55de00700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa56006afb0 0x7fa56019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:37.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.785+0000 7fa555400700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56006b970 0x7fa56019b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:54:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.786+0000 7fa55e800700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa560108a20 0x7fa56019fa70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa55800e970 tx=0x7fa55800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.786+0000 7fa557400700 1 -- 172.21.15.45:0/1580587082 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558004a30 con 0x7fa560108a20 2024-04-25T18:54:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.786+0000 7fa557400700 1 -- 172.21.15.45:0/1580587082 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5580054d0 con 0x7fa560108a20 2024-04-25T18:54:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.786+0000 7fa557400700 1 -- 172.21.15.45:0/1580587082 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558010ca0 con 0x7fa560108a20 2024-04-25T18:54:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.786+0000 7fa55f200700 1 -- 172.21.15.45:0/1580587082 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5601a04a0 con 0x7fa560108a20 2024-04-25T18:54:37.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.786+0000 7fa55f200700 1 -- 172.21.15.45:0/1580587082 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5601a7d20 con 0x7fa560108a20 2024-04-25T18:54:37.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.787+0000 7fa55f200700 1 -- 172.21.15.45:0/1580587082 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa560069730 con 0x7fa560108a20 2024-04-25T18:54:37.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.789+0000 7fa557400700 1 -- 172.21.15.45:0/1580587082 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa558004fe0 con 0x7fa560108a20 2024-04-25T18:54:37.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.790+0000 7fa557400700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5440a3c40 0x7fa5440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:37.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.791+0000 7fa557400700 1 -- 172.21.15.45:0/1580587082 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fa5580d00d0 con 0x7fa560108a20 2024-04-25T18:54:37.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.791+0000 7fa55de00700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5440a3c40 0x7fa5440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:37.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.791+0000 7fa55de00700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5440a3c40 0x7fa5440a6100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa5500062a0 tx=0x7fa550006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:37.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:37.795+0000 7fa557400700 1 -- 172.21.15.45:0/1580587082 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa55809ceb0 con 0x7fa560108a20 2024-04-25T18:54:38.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.078+0000 7fa55f200700 1 -- 172.21.15.45:0/1580587082 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa56019c500 con 0x7fa560108a20 2024-04-25T18:54:38.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.079+0000 7fa557400700 1 -- 172.21.15.45:0/1580587082 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3862 (secure 0 0 0) 0x7fa55809c7b0 con 0x7fa560108a20 2024-04-25T18:54:38.081 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:54:38.082 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[]} 2024-04-25T18:54:38.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 -- 172.21.15.45:0/1580587082 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5440a3c40 msgr2=0x7fa5440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:38.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5440a3c40 0x7fa5440a6100 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa5500062a0 tx=0x7fa550006210 comp rx=0 tx=0).stop 2024-04-25T18:54:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 -- 172.21.15.45:0/1580587082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa560108a20 msgr2=0x7fa56019fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa560108a20 0x7fa56019fa70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa55800e970 tx=0x7fa55800ec80 comp rx=0 tx=0).stop 2024-04-25T18:54:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 -- 172.21.15.45:0/1580587082 shutdown_connections 2024-04-25T18:54:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5440a3c40 0x7fa5440a6100 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa560108a20 0x7fa56019fa70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa56006afb0 0x7fa56019b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 --2- 172.21.15.45:0/1580587082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa56006b970 0x7fa56019b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 -- 172.21.15.45:0/1580587082 >> 172.21.15.45:0/1580587082 conn(0x7fa560078780 msgr2=0x7fa56010dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 -- 172.21.15.45:0/1580587082 shutdown_connections 2024-04-25T18:54:38.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:38.082+0000 7fa55f200700 1 -- 172.21.15.45:0/1580587082 wait complete. 2024-04-25T18:54:38.086 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 5 2024-04-25T18:54:38.353 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:37 smithi062 ceph-mon[32844]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:38.353 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:38.353 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:38.353 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:38.353 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:38.353 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:38.353 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:37 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:37 smithi079 ceph-mon[33002]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:37 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:37 smithi045 ceph-mon[27776]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:37 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:39.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:39 smithi045 ceph-mon[27776]: Deploying daemon mds.e on smithi062 2024-04-25T18:54:39.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:39 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.45:6836/496487622,v1:172.21.15.45:6837/496487622] up:boot 2024-04-25T18:54:39.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:39 smithi045 ceph-mon[27776]: fsmap 4 up:standby 2024-04-25T18:54:39.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:39 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:54:39.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:39 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1580587082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:39.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:39 smithi062 ceph-mon[32844]: Deploying daemon mds.e on smithi062 2024-04-25T18:54:39.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:39 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.45:6836/496487622,v1:172.21.15.45:6837/496487622] up:boot 2024-04-25T18:54:39.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:39 smithi062 ceph-mon[32844]: fsmap 4 up:standby 2024-04-25T18:54:39.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:39 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:54:39.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:39 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1580587082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:39.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:39 smithi079 ceph-mon[33002]: Deploying daemon mds.e on smithi062 2024-04-25T18:54:39.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:39 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.45:6836/496487622,v1:172.21.15.45:6837/496487622] up:boot 2024-04-25T18:54:39.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:39 smithi079 ceph-mon[33002]: fsmap 4 up:standby 2024-04-25T18:54:39.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:39 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:54:39.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:39 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1580587082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:40.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:40 smithi062 ceph-mon[32844]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:40.114 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 18:54:40 smithi062 systemd[1]: Starting Ceph mds.e for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:54:40.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:40 smithi079 ceph-mon[33002]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:40.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:40 smithi045 ceph-mon[27776]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:40.520 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 18:54:40 smithi062 podman[63272]: 2024-04-25 18:54:40.249988231 +0000 UTC m=+0.100901864 container create 8de2efef5e02bfb3f2e9a68068688e7cd0c489dcb0be453de1e116bae53c0b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T18:54:40.521 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 18:54:40 smithi062 podman[63272]: 2024-04-25 18:54:40.167550231 +0000 UTC m=+0.018463867 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:54:40.521 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 18:54:40 smithi062 podman[63272]: 2024-04-25 18:54:40.494102301 +0000 UTC m=+0.345015937 container init 8de2efef5e02bfb3f2e9a68068688e7cd0c489dcb0be453de1e116bae53c0b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T18:54:40.521 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 18:54:40 smithi062 podman[63272]: 2024-04-25 18:54:40.4968523 +0000 UTC m=+0.347765936 container start 8de2efef5e02bfb3f2e9a68068688e7cd0c489dcb0be453de1e116bae53c0b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T18:54:40.863 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 18:54:40 smithi062 ceph-mds[63291]: -- 172.21.15.62:0/612716871 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a90279da40 con 0x55a902770800 2024-04-25T18:54:40.864 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 18:54:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-e[63287]: starting mds.e at 2024-04-25T18:54:40.864 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 18:54:40 smithi062 bash[63272]: 8de2efef5e02bfb3f2e9a68068688e7cd0c489dcb0be453de1e116bae53c0b81 2024-04-25T18:54:40.864 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 18:54:40 smithi062 systemd[1]: Started Ceph mds.e for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:54:40.877 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:54:41.372 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:41.779 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:41.783 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:41.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:41.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:41.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:41.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:41.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:41.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: Deploying daemon mds.f on smithi079 2024-04-25T18:54:41.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.62:6834/3037892802,v1:172.21.15.62:6835/3037892802] up:boot 2024-04-25T18:54:41.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: fsmap 5 up:standby 2024-04-25T18:54:41.784 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:41 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:54:41.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:41.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:41.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:41.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:41.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:41.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:41.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:41.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: Deploying daemon mds.f on smithi079 2024-04-25T18:54:41.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.62:6834/3037892802,v1:172.21.15.62:6835/3037892802] up:boot 2024-04-25T18:54:41.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: fsmap 5 up:standby 2024-04-25T18:54:41.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:41 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:54:42.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: Deploying daemon mds.f on smithi079 2024-04-25T18:54:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.62:6834/3037892802,v1:172.21.15.62:6835/3037892802] up:boot 2024-04-25T18:54:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: fsmap 5 up:standby 2024-04-25T18:54:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:41 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:54:43.132 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 18:54:42 smithi079 systemd[1]: Starting Ceph mds.f for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:54:43.391 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 18:54:43 smithi079 podman[63671]: 2024-04-25 18:54:43.034652026 +0000 UTC m=+0.018160366 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:54:43.391 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 18:54:43 smithi079 podman[63671]: 2024-04-25 18:54:43.136177274 +0000 UTC m=+0.119685610 container create 1de7d6a87493e0cc9d27efac7d99bb3aab0250c489b92fa5f0a096b2b767daa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T18:54:43.391 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 18:54:43 smithi079 podman[63671]: 2024-04-25 18:54:43.363688139 +0000 UTC m=+0.347196474 container init 1de7d6a87493e0cc9d27efac7d99bb3aab0250c489b92fa5f0a096b2b767daa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T18:54:43.391 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 18:54:43 smithi079 podman[63671]: 2024-04-25 18:54:43.366297907 +0000 UTC m=+0.349806251 container start 1de7d6a87493e0cc9d27efac7d99bb3aab0250c489b92fa5f0a096b2b767daa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T18:54:43.391 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 18:54:43 smithi079 ceph-mds[63690]: -- 172.21.15.79:0/3742532608 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e45041a40 con 0x563e45014800 2024-04-25T18:54:43.583 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.580+0000 7f7cae800700 1 -- 172.21.15.45:0/3708919225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ca81044b0 msgr2=0x7f7ca810c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:43.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.580+0000 7f7cae800700 1 --2- 172.21.15.45:0/3708919225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ca81044b0 0x7f7ca810c9a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009990 tx=0x7f7c94009ca0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.580+0000 7f7cae800700 1 -- 172.21.15.45:0/3708919225 shutdown_connections 2024-04-25T18:54:43.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.580+0000 7f7cae800700 1 --2- 172.21.15.45:0/3708919225 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ca810cf70 0x7f7ca810f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.580+0000 7f7cae800700 1 --2- 172.21.15.45:0/3708919225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca8103b90 0x7f7ca8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.580+0000 7f7cae800700 1 --2- 172.21.15.45:0/3708919225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ca81044b0 0x7f7ca810c9a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.580+0000 7f7cae800700 1 -- 172.21.15.45:0/3708919225 >> 172.21.15.45:0/3708919225 conn(0x7f7ca80fd360 msgr2=0x7f7ca80ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:43.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.581+0000 7f7cae800700 1 -- 172.21.15.45:0/3708919225 shutdown_connections 2024-04-25T18:54:43.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.581+0000 7f7cae800700 1 -- 172.21.15.45:0/3708919225 wait complete. 2024-04-25T18:54:43.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.582+0000 7f7cae800700 1 Processor -- start 2024-04-25T18:54:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.582+0000 7f7cae800700 1 -- start start 2024-04-25T18:54:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.582+0000 7f7cae800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ca8103b90 0x7f7ca81a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.582+0000 7f7cae800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ca81044b0 0x7f7ca81a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.582+0000 7f7cae800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca810cf70 0x7f7ca810bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.582+0000 7f7cae800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8112b70 con 0x7f7ca810cf70 2024-04-25T18:54:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.582+0000 7f7cae800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca81129f0 con 0x7f7ca81044b0 2024-04-25T18:54:43.585 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.582+0000 7f7cae800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca8112cf0 con 0x7f7ca8103b90 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7cad400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ca8103b90 0x7f7ca81a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7cade00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca810cf70 0x7f7ca810bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7cad400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ca8103b90 0x7f7ca81a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48656/0 (socket says 172.21.15.45:48656) 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7cad400700 1 -- 172.21.15.45:0/3104025943 learned_addr learned my addr 172.21.15.45:0/3104025943 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7caca00700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ca81044b0 0x7f7ca81a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7cad400700 1 -- 172.21.15.45:0/3104025943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ca81044b0 msgr2=0x7f7ca81a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7cad400700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ca81044b0 0x7f7ca81a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7cad400700 1 -- 172.21.15.45:0/3104025943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca810cf70 msgr2=0x7f7ca810bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7cad400700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca810cf70 0x7f7ca810bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7cad400700 1 -- 172.21.15.45:0/3104025943 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca810c480 con 0x7f7ca8103b90 2024-04-25T18:54:43.586 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.583+0000 7f7cade00700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca810cf70 0x7f7ca810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:54:43.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.584+0000 7f7cad400700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ca8103b90 0x7f7ca81a42d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9a0 tx=0x7f7c9c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:43.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.584+0000 7f7ca6000700 1 -- 172.21.15.45:0/3104025943 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c00cbd0 con 0x7f7ca8103b90 2024-04-25T18:54:43.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.584+0000 7f7ca6000700 1 -- 172.21.15.45:0/3104025943 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c9c00cd70 con 0x7f7ca8103b90 2024-04-25T18:54:43.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.584+0000 7f7cae800700 1 -- 172.21.15.45:0/3104025943 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca810c770 con 0x7f7ca8103b90 2024-04-25T18:54:43.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.584+0000 7f7ca6000700 1 -- 172.21.15.45:0/3104025943 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c018680 con 0x7f7ca8103b90 2024-04-25T18:54:43.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.584+0000 7f7cae800700 1 -- 172.21.15.45:0/3104025943 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca81ac3d0 con 0x7f7ca8103b90 2024-04-25T18:54:43.587 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.585+0000 7f7cae800700 1 -- 172.21.15.45:0/3104025943 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca806afa0 con 0x7f7ca8103b90 2024-04-25T18:54:43.589 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.587+0000 7f7ca6000700 1 -- 172.21.15.45:0/3104025943 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7c9c010430 con 0x7f7ca8103b90 2024-04-25T18:54:43.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.589+0000 7f7ca6000700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7c900a3cc0 0x7f7c900a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:43.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.589+0000 7f7ca6000700 1 -- 172.21.15.45:0/3104025943 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f7c9c0d01c0 con 0x7f7ca8103b90 2024-04-25T18:54:43.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.589+0000 7f7caca00700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7c900a3cc0 0x7f7c900a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:43.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.590+0000 7f7caca00700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7c900a3cc0 0x7f7c900a6180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7ca81a0870 tx=0x7f7c9400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:43.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.591+0000 7f7ca6000700 1 -- 172.21.15.45:0/3104025943 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c9c09cfa0 con 0x7f7ca8103b90 2024-04-25T18:54:43.662 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 18:54:43 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f[63686]: starting mds.f at 2024-04-25T18:54:43.662 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 18:54:43 smithi079 bash[63671]: 1de7d6a87493e0cc9d27efac7d99bb3aab0250c489b92fa5f0a096b2b767daa0 2024-04-25T18:54:43.662 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 18:54:43 smithi079 systemd[1]: Started Ceph mds.f for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:54:43.878 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.876+0000 7f7cae800700 1 -- 172.21.15.45:0/3104025943 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ca81a0a50 con 0x7f7ca8103b90 2024-04-25T18:54:43.879 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:43 smithi045 ceph-mon[27776]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:43.879 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:43.879 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:43.879 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:43.879 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:43.879 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:43.879 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:43 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:43.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.877+0000 7f7ca6000700 1 -- 172.21.15.45:0/3104025943 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5512 (secure 0 0 0) 0x7f7c9c09c8a0 con 0x7f7ca8103b90 2024-04-25T18:54:43.880 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:54:43.880 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T18:54:43.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.881+0000 7f7cae800700 1 -- 172.21.15.45:0/3104025943 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7c900a3cc0 msgr2=0x7f7c900a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:43.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.881+0000 7f7cae800700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7c900a3cc0 0x7f7c900a6180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7ca81a0870 tx=0x7f7c9400b260 comp rx=0 tx=0).stop 2024-04-25T18:54:43.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.881+0000 7f7cae800700 1 -- 172.21.15.45:0/3104025943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ca8103b90 msgr2=0x7f7ca81a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:43.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.881+0000 7f7cae800700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ca8103b90 0x7f7ca81a42d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c00e9a0 tx=0x7f7c9c00ecb0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.882+0000 7f7cae800700 1 -- 172.21.15.45:0/3104025943 shutdown_connections 2024-04-25T18:54:43.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.882+0000 7f7cae800700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7c900a3cc0 0x7f7c900a6180 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.882+0000 7f7cae800700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7ca810cf70 0x7f7ca810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.882+0000 7f7cae800700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ca8103b90 0x7f7ca81a42d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.882+0000 7f7cae800700 1 --2- 172.21.15.45:0/3104025943 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7ca81044b0 0x7f7ca81a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:43.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.882+0000 7f7cae800700 1 -- 172.21.15.45:0/3104025943 >> 172.21.15.45:0/3104025943 conn(0x7f7ca80fd360 msgr2=0x7f7ca80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:43.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.882+0000 7f7cae800700 1 -- 172.21.15.45:0/3104025943 shutdown_connections 2024-04-25T18:54:43.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:43.882+0000 7f7cae800700 1 -- 172.21.15.45:0/3104025943 wait complete. 2024-04-25T18:54:43.885 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 7 2024-04-25T18:54:43.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:43 smithi079 ceph-mon[33002]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:43.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:43.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:43.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:43.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:43.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:43.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:43 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:44.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:43 smithi062 ceph-mon[32844]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:44.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:44.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:44.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:44.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:44.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:44.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:43 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:44.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:44 smithi079 ceph-mon[33002]: Deploying daemon mds.c on smithi079 2024-04-25T18:54:44.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:44 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.79:6834/1201787807,v1:172.21.15.79:6835/1201787807] up:boot 2024-04-25T18:54:44.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:44 smithi079 ceph-mon[33002]: fsmap 6 up:standby 2024-04-25T18:54:44.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:54:44.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:44 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3104025943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:44.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:44 smithi045 ceph-mon[27776]: Deploying daemon mds.c on smithi079 2024-04-25T18:54:44.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:44 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.79:6834/1201787807,v1:172.21.15.79:6835/1201787807] up:boot 2024-04-25T18:54:44.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:44 smithi045 ceph-mon[27776]: fsmap 6 up:standby 2024-04-25T18:54:44.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:54:44.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:44 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3104025943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:45.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:44 smithi062 ceph-mon[32844]: Deploying daemon mds.c on smithi079 2024-04-25T18:54:45.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:44 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.79:6834/1201787807,v1:172.21.15.79:6835/1201787807] up:boot 2024-04-25T18:54:45.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:44 smithi062 ceph-mon[32844]: fsmap 6 up:standby 2024-04-25T18:54:45.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:54:45.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:44 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3104025943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:45.931 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:45 smithi079 ceph-mon[33002]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:45.931 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 18:54:45 smithi079 systemd[1]: Starting Ceph mds.c for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:54:45.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:45 smithi045 ceph-mon[27776]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:46.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:45 smithi062 ceph-mon[32844]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:46.184 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 18:54:45 smithi079 podman[63981]: 2024-04-25 18:54:45.832293435 +0000 UTC m=+0.018404439 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:54:46.185 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 18:54:45 smithi079 podman[63981]: 2024-04-25 18:54:45.930924758 +0000 UTC m=+0.117035729 container create 3a4bf803c7fb359cb854cfca5e2741e8bd85843d5eead41e8048b655e083f91b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:54:46.684 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 18:54:46 smithi079 podman[63981]: 2024-04-25 18:54:46.241934844 +0000 UTC m=+0.428045824 container init 3a4bf803c7fb359cb854cfca5e2741e8bd85843d5eead41e8048b655e083f91b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T18:54:46.684 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 18:54:46 smithi079 podman[63981]: 2024-04-25 18:54:46.245209104 +0000 UTC m=+0.431320075 container start 3a4bf803c7fb359cb854cfca5e2741e8bd85843d5eead41e8048b655e083f91b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T18:54:46.684 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 18:54:46 smithi079 ceph-mds[64000]: -- 172.21.15.79:0/1578098633 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55db24743a40 con 0x55db24716c00 2024-04-25T18:54:46.684 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 18:54:46 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c[63996]: starting mds.c at 2024-04-25T18:54:46.684 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 18:54:46 smithi079 bash[63981]: 3a4bf803c7fb359cb854cfca5e2741e8bd85843d5eead41e8048b655e083f91b 2024-04-25T18:54:46.684 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 18:54:46 smithi079 systemd[1]: Started Ceph mds.c for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:54:46.746 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:54:47.223 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: Deploying daemon mds.h on smithi062 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.79:6836/3290580293,v1:172.21.15.79:6837/3290580293] up:boot 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: fsmap 7 up:standby 2024-04-25T18:54:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:54:47.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:47.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:47.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:47.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:47.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:47.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:47.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:47.699 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: Deploying daemon mds.h on smithi062 2024-04-25T18:54:47.700 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.79:6836/3290580293,v1:172.21.15.79:6837/3290580293] up:boot 2024-04-25T18:54:47.700 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: fsmap 7 up:standby 2024-04-25T18:54:47.700 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:54:47.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: Deploying daemon mds.h on smithi062 2024-04-25T18:54:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.79:6836/3290580293,v1:172.21.15.79:6837/3290580293] up:boot 2024-04-25T18:54:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: fsmap 7 up:standby 2024-04-25T18:54:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:54:48.864 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 18:54:48 smithi062 systemd[1]: Starting Ceph mds.h for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:54:49.224 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 18:54:48 smithi062 podman[63579]: 2024-04-25 18:54:48.853800707 +0000 UTC m=+0.018378087 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:54:49.225 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 18:54:48 smithi062 podman[63579]: 2024-04-25 18:54:48.969662351 +0000 UTC m=+0.134239730 container create fdce5165caa71e416d7a306da4c187319d60a6e3b5b50e71b07f944a362ea2b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T18:54:49.225 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 18:54:49 smithi062 podman[63579]: 2024-04-25 18:54:49.197208385 +0000 UTC m=+0.361785766 container init fdce5165caa71e416d7a306da4c187319d60a6e3b5b50e71b07f944a362ea2b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:54:49.225 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 18:54:49 smithi062 podman[63579]: 2024-04-25 18:54:49.200070564 +0000 UTC m=+0.364647953 container start fdce5165caa71e416d7a306da4c187319d60a6e3b5b50e71b07f944a362ea2b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T18:54:49.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.362+0000 7f764a800700 1 -- 172.21.15.45:0/4223443779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7644074e70 msgr2=0x7f7644075250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.362+0000 7f764a800700 1 --2- 172.21.15.45:0/4223443779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7644074e70 0x7f7644075250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7638009990 tx=0x7f7638009ca0 comp rx=0 tx=0).stop 2024-04-25T18:54:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.363+0000 7f764a800700 1 -- 172.21.15.45:0/4223443779 shutdown_connections 2024-04-25T18:54:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.363+0000 7f764a800700 1 --2- 172.21.15.45:0/4223443779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f764410f190 0x7f7644111600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.363+0000 7f764a800700 1 --2- 172.21.15.45:0/4223443779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7644074e70 0x7f7644075250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.363+0000 7f764a800700 1 --2- 172.21.15.45:0/4223443779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644075790 0x7f764410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.363+0000 7f764a800700 1 -- 172.21.15.45:0/4223443779 >> 172.21.15.45:0/4223443779 conn(0x7f76440fe460 msgr2=0x7f7644100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:49.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.363+0000 7f764a800700 1 -- 172.21.15.45:0/4223443779 shutdown_connections 2024-04-25T18:54:49.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.363+0000 7f764a800700 1 -- 172.21.15.45:0/4223443779 wait complete. 2024-04-25T18:54:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.364+0000 7f764a800700 1 Processor -- start 2024-04-25T18:54:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.364+0000 7f764a800700 1 -- start start 2024-04-25T18:54:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.364+0000 7f764a800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644074e70 0x7f764419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.364+0000 7f764a800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7644075790 0x7f764419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f764a800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f764410f190 0x7f764419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:49.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f764a800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7644114d90 con 0x7f764410f190 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f764a800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7644114c10 con 0x7f7644074e70 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f764a800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7644114f10 con 0x7f7644075790 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f7649400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644074e70 0x7f764419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f7649e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f764410f190 0x7f764419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f7649400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644074e70 0x7f764419b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:60082/0 (socket says 172.21.15.45:60082) 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f7649400700 1 -- 172.21.15.45:0/2441877341 learned_addr learned my addr 172.21.15.45:0/2441877341 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f7648a00700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7644075790 0x7f764419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f7649400700 1 -- 172.21.15.45:0/2441877341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7644075790 msgr2=0x7f764419b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f7649400700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7644075790 0x7f764419b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f7649400700 1 -- 172.21.15.45:0/2441877341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f764410f190 msgr2=0x7f764419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f7649400700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f764410f190 0x7f764419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.365+0000 7f7649400700 1 -- 172.21.15.45:0/2441877341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76441a02b0 con 0x7f7644074e70 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.366+0000 7f7649400700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644074e70 0x7f764419b2a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f76380047b0 tx=0x7f7638009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:49.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.366+0000 7f7642000700 1 -- 172.21.15.45:0/2441877341 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638027070 con 0x7f7644074e70 2024-04-25T18:54:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.366+0000 7f7642000700 1 -- 172.21.15.45:0/2441877341 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7638022470 con 0x7f7644074e70 2024-04-25T18:54:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.366+0000 7f7642000700 1 -- 172.21.15.45:0/2441877341 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638017650 con 0x7f7644074e70 2024-04-25T18:54:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.366+0000 7f764a800700 1 -- 172.21.15.45:0/2441877341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76441a0540 con 0x7f7644074e70 2024-04-25T18:54:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.366+0000 7f764a800700 1 -- 172.21.15.45:0/2441877341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76441a7e20 con 0x7f7644074e70 2024-04-25T18:54:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.367+0000 7f764a800700 1 -- 172.21.15.45:0/2441877341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7644069730 con 0x7f7644074e70 2024-04-25T18:54:49.370 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.369+0000 7f7642000700 1 -- 172.21.15.45:0/2441877341 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f763802a070 con 0x7f7644074e70 2024-04-25T18:54:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.371+0000 7f7642000700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f762c0a3cc0 0x7f762c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.371+0000 7f7642000700 1 -- 172.21.15.45:0/2441877341 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f76380d0e30 con 0x7f7644074e70 2024-04-25T18:54:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.371+0000 7f7648a00700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f762c0a3cc0 0x7f762c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:49.372 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.372+0000 7f7648a00700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f762c0a3cc0 0x7f762c0a6180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76441105c0 tx=0x7f7634009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:49.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.373+0000 7f7642000700 1 -- 172.21.15.45:0/2441877341 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7638013070 con 0x7f7644074e70 2024-04-25T18:54:49.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:49 smithi045 ceph-mon[27776]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:49.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:49.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:49.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:49.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:49.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:49 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:49.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:49 smithi062 ceph-mon[32844]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:49.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:49.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:49.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:49.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:49.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:49 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:49.614 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 18:54:49 smithi062 ceph-mds[63598]: -- 172.21.15.62:0/3426257231 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562fac1b7a40 con 0x562fac18a800 2024-04-25T18:54:49.614 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 18:54:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h[63594]: starting mds.h at 2024-04-25T18:54:49.614 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 18:54:49 smithi062 bash[63579]: fdce5165caa71e416d7a306da4c187319d60a6e3b5b50e71b07f944a362ea2b1 2024-04-25T18:54:49.614 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 18:54:49 smithi062 systemd[1]: Started Ceph mds.h for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:54:49.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.672+0000 7f764a800700 1 -- 172.21.15.45:0/2441877341 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f764419c600 con 0x7f7644074e70 2024-04-25T18:54:49.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.673+0000 7f7642000700 1 -- 172.21.15.45:0/2441877341 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7162 (secure 0 0 0) 0x7f763809dca0 con 0x7f7644074e70 2024-04-25T18:54:49.674 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:54:49.674 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T18:54:49.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.676+0000 7f764a800700 1 -- 172.21.15.45:0/2441877341 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f762c0a3cc0 msgr2=0x7f762c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.676+0000 7f764a800700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f762c0a3cc0 0x7f762c0a6180 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f76441105c0 tx=0x7f7634009260 comp rx=0 tx=0).stop 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.676+0000 7f764a800700 1 -- 172.21.15.45:0/2441877341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644074e70 msgr2=0x7f764419b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.676+0000 7f764a800700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644074e70 0x7f764419b2a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f76380047b0 tx=0x7f7638009550 comp rx=0 tx=0).stop 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.676+0000 7f764a800700 1 -- 172.21.15.45:0/2441877341 shutdown_connections 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.676+0000 7f764a800700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f762c0a3cc0 0x7f762c0a6180 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.676+0000 7f764a800700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7644074e70 0x7f764419b2a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.676+0000 7f764a800700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f764410f190 0x7f764419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.676+0000 7f764a800700 1 --2- 172.21.15.45:0/2441877341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7644075790 0x7f764419b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.676+0000 7f764a800700 1 -- 172.21.15.45:0/2441877341 >> 172.21.15.45:0/2441877341 conn(0x7f76440fe460 msgr2=0x7f76440ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.677+0000 7f764a800700 1 -- 172.21.15.45:0/2441877341 shutdown_connections 2024-04-25T18:54:49.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:49.677+0000 7f764a800700 1 -- 172.21.15.45:0/2441877341 wait complete. 2024-04-25T18:54:49.680 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 9 2024-04-25T18:54:49.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:49 smithi079 ceph-mon[33002]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:49.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:49.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:49.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:49.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:49.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:49 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:50 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:50 smithi079 ceph-mon[33002]: Deploying daemon mds.i on smithi079 2024-04-25T18:54:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:50 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.62:6836/2985890701,v1:172.21.15.62:6837/2985890701] up:boot 2024-04-25T18:54:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:50 smithi079 ceph-mon[33002]: fsmap 8 up:standby 2024-04-25T18:54:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:50 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:54:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:50 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2441877341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:50 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:50 smithi045 ceph-mon[27776]: Deploying daemon mds.i on smithi079 2024-04-25T18:54:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:50 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.62:6836/2985890701,v1:172.21.15.62:6837/2985890701] up:boot 2024-04-25T18:54:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:50 smithi045 ceph-mon[27776]: fsmap 8 up:standby 2024-04-25T18:54:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:50 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:54:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:50 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2441877341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:50.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:50 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:50.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:50 smithi062 ceph-mon[32844]: Deploying daemon mds.i on smithi079 2024-04-25T18:54:50.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:50 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.62:6836/2985890701,v1:172.21.15.62:6837/2985890701] up:boot 2024-04-25T18:54:50.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:50 smithi062 ceph-mon[32844]: fsmap 8 up:standby 2024-04-25T18:54:50.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:50 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:54:50.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:50 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2441877341' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:51 smithi079 ceph-mon[33002]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:51.684 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 18:54:51 smithi079 systemd[1]: Starting Ceph mds.i for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:54:51.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:51 smithi045 ceph-mon[27776]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:51.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:51 smithi062 ceph-mon[32844]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:52.096 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 18:54:51 smithi079 podman[64291]: 2024-04-25 18:54:51.74011281 +0000 UTC m=+0.017942774 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:54:52.096 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 18:54:51 smithi079 podman[64291]: 2024-04-25 18:54:51.87635198 +0000 UTC m=+0.154181976 container create d9faa66d057d660b0bd5af454191bd62b67a7b954277e85605116e849e944624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-04-25T18:54:52.096 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 18:54:52 smithi079 podman[64291]: 2024-04-25 18:54:52.09530089 +0000 UTC m=+0.373130840 container init d9faa66d057d660b0bd5af454191bd62b67a7b954277e85605116e849e944624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T18:54:52.425 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:54:52.434 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 18:54:52 smithi079 podman[64291]: 2024-04-25 18:54:52.09818369 +0000 UTC m=+0.376013652 container start d9faa66d057d660b0bd5af454191bd62b67a7b954277e85605116e849e944624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T18:54:52.434 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 18:54:52 smithi079 ceph-mds[64310]: -- 172.21.15.79:0/611725253 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55755ff1ba40 con 0x55755feee800 2024-04-25T18:54:52.434 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 18:54:52 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i[64306]: starting mds.i at 2024-04-25T18:54:52.434 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 18:54:52 smithi079 bash[64291]: d9faa66d057d660b0bd5af454191bd62b67a7b954277e85605116e849e944624 2024-04-25T18:54:52.434 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 18:54:52 smithi079 systemd[1]: Started Ceph mds.i for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:54:52.874 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:54:53.362 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:53.362 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:53.362 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:53.362 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:53.362 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:53.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:53.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:53.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: Deploying daemon mds.g on smithi045 2024-04-25T18:54:53.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] up:boot 2024-04-25T18:54:53.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: fsmap 9 up:standby 2024-04-25T18:54:53.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:53 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:54:53.515 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:53.515 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:53.515 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:53.515 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:53.515 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:53.515 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:53.515 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:53.516 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: Deploying daemon mds.g on smithi045 2024-04-25T18:54:53.516 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] up:boot 2024-04-25T18:54:53.516 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: fsmap 9 up:standby 2024-04-25T18:54:53.516 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:53 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:54:53.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:53.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:53.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:53.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:53.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:53.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:53.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:53.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: Deploying daemon mds.g on smithi045 2024-04-25T18:54:53.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] up:boot 2024-04-25T18:54:53.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: fsmap 9 up:standby 2024-04-25T18:54:53.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:53 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:54:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.553+0000 7f7696400700 1 -- 172.21.15.45:0/4294143257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690075590 msgr2=0x7f769010ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.553+0000 7f7696400700 1 --2- 172.21.15.45:0/4294143257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690075590 0x7f769010ef00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76780098f0 tx=0x7f7678009c00 comp rx=0 tx=0).stop 2024-04-25T18:54:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.553+0000 7f7696400700 1 -- 172.21.15.45:0/4294143257 shutdown_connections 2024-04-25T18:54:55.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.553+0000 7f7696400700 1 --2- 172.21.15.45:0/4294143257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f769010f440 0x7f7690111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.553+0000 7f7696400700 1 --2- 172.21.15.45:0/4294143257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7690074c70 0x7f7690075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.553+0000 7f7696400700 1 --2- 172.21.15.45:0/4294143257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690075590 0x7f769010ef00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:55.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.553+0000 7f7696400700 1 -- 172.21.15.45:0/4294143257 >> 172.21.15.45:0/4294143257 conn(0x7f7690079ba0 msgr2=0x7f7690079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:55.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:55 smithi045 ceph-mon[27776]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.553+0000 7f7696400700 1 -- 172.21.15.45:0/4294143257 shutdown_connections 2024-04-25T18:54:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.553+0000 7f7696400700 1 -- 172.21.15.45:0/4294143257 wait complete. 2024-04-25T18:54:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.554+0000 7f7696400700 1 Processor -- start 2024-04-25T18:54:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.554+0000 7f7696400700 1 -- start start 2024-04-25T18:54:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f7696400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7690074c70 0x7f769019b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:55.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f7696400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690075590 0x7f769019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f7696400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f769010f440 0x7f769019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f7696400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7690114e70 con 0x7f7690074c70 2024-04-25T18:54:55.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f7696400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7690114cf0 con 0x7f769010f440 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f7696400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7690114ff0 con 0x7f7690075590 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f7686a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690075590 0x7f769019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f7686a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690075590 0x7f769019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:37212/0 (socket says 172.21.15.45:37212) 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f7686a00700 1 -- 172.21.15.45:0/4274133289 learned_addr learned my addr 172.21.15.45:0/4274133289 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f768fe00700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f769010f440 0x7f769019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.555+0000 7f768f400700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7690074c70 0x7f769019b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.556+0000 7f7686a00700 1 -- 172.21.15.45:0/4274133289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f769010f440 msgr2=0x7f769019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.556+0000 7f7686a00700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f769010f440 0x7f769019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.556+0000 7f7686a00700 1 -- 172.21.15.45:0/4274133289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7690074c70 msgr2=0x7f769019b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.556+0000 7f7686a00700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7690074c70 0x7f769019b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.556+0000 7f7686a00700 1 -- 172.21.15.45:0/4274133289 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76901a0350 con 0x7f7690075590 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.556+0000 7f7686a00700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690075590 0x7f769019b880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7678009230 tx=0x7f767800f990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.556+0000 7f768cc00700 1 -- 172.21.15.45:0/4274133289 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678027070 con 0x7f7690075590 2024-04-25T18:54:55.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.556+0000 7f768cc00700 1 -- 172.21.15.45:0/4274133289 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76780055f0 con 0x7f7690075590 2024-04-25T18:54:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.556+0000 7f768cc00700 1 -- 172.21.15.45:0/4274133289 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7678004930 con 0x7f7690075590 2024-04-25T18:54:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.556+0000 7f7696400700 1 -- 172.21.15.45:0/4274133289 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76901a05e0 con 0x7f7690075590 2024-04-25T18:54:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.557+0000 7f7696400700 1 -- 172.21.15.45:0/4274133289 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76901a7ec0 con 0x7f7690075590 2024-04-25T18:54:55.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.558+0000 7f7696400700 1 -- 172.21.15.45:0/4274133289 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7690076360 con 0x7f7690075590 2024-04-25T18:54:55.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.560+0000 7f768cc00700 1 -- 172.21.15.45:0/4274133289 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f767802a070 con 0x7f7690075590 2024-04-25T18:54:55.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.561+0000 7f768cc00700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f76740a3cc0 0x7f76740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:54:55.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.561+0000 7f768cc00700 1 -- 172.21.15.45:0/4274133289 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f76780d0940 con 0x7f7690075590 2024-04-25T18:54:55.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.561+0000 7f768f400700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f76740a3cc0 0x7f76740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:54:55.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.562+0000 7f768f400700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f76740a3cc0 0x7f76740a6180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7680009fd0 tx=0x7f768000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:54:55.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.565+0000 7f768cc00700 1 -- 172.21.15.45:0/4274133289 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f767805eff0 con 0x7f7690075590 2024-04-25T18:54:55.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:55 smithi062 ceph-mon[32844]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:55.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:55 smithi079 ceph-mon[33002]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:55.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.843+0000 7f7696400700 1 -- 172.21.15.45:0/4274133289 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f769019c6a0 con 0x7f7690075590 2024-04-25T18:54:55.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.844+0000 7f768cc00700 1 -- 172.21.15.45:0/4274133289 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+7986 (secure 0 0 0) 0x7f7678013030 con 0x7f7690075590 2024-04-25T18:54:55.845 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:54:55.845 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T18:54:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.847+0000 7f7696400700 1 -- 172.21.15.45:0/4274133289 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f76740a3cc0 msgr2=0x7f76740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.847+0000 7f7696400700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f76740a3cc0 0x7f76740a6180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7680009fd0 tx=0x7f768000b040 comp rx=0 tx=0).stop 2024-04-25T18:54:55.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.847+0000 7f7696400700 1 -- 172.21.15.45:0/4274133289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690075590 msgr2=0x7f769019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:54:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.847+0000 7f7696400700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690075590 0x7f769019b880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7678009230 tx=0x7f767800f990 comp rx=0 tx=0).stop 2024-04-25T18:54:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.847+0000 7f7696400700 1 -- 172.21.15.45:0/4274133289 shutdown_connections 2024-04-25T18:54:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.847+0000 7f7696400700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f76740a3cc0 0x7f76740a6180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.847+0000 7f7696400700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f769010f440 0x7f769019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.847+0000 7f7696400700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7690074c70 0x7f769019b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.847+0000 7f7696400700 1 --2- 172.21.15.45:0/4274133289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7690075590 0x7f769019b880 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:54:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.847+0000 7f7696400700 1 -- 172.21.15.45:0/4274133289 >> 172.21.15.45:0/4274133289 conn(0x7f7690079ba0 msgr2=0x7f76900ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:54:55.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.848+0000 7f7696400700 1 -- 172.21.15.45:0/4274133289 shutdown_connections 2024-04-25T18:54:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:54:55.848+0000 7f7696400700 1 -- 172.21.15.45:0/4274133289 wait complete. 2024-04-25T18:54:55.850 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 10 2024-04-25T18:54:56.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:56 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/4274133289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:56.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:56 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/4274133289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:56 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/4274133289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:54:57.606 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:57 smithi045 ceph-mon[27776]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:57.608 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 18:54:57 smithi045 podman[66647]: 2024-04-25 18:54:57.334824848 +0000 UTC m=+0.176139320 container create 753e25814128eb144bc50ba9d3baaa9827e325e93f52252f5d4577e684449e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T18:54:57.608 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 18:54:57 smithi045 podman[66647]: 2024-04-25 18:54:57.605218577 +0000 UTC m=+0.446533069 container init 753e25814128eb144bc50ba9d3baaa9827e325e93f52252f5d4577e684449e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-g, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T18:54:57.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:57 smithi062 ceph-mon[32844]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:57.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:57 smithi079 ceph-mon[33002]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:54:57.878 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 18:54:57 smithi045 podman[66647]: 2024-04-25 18:54:57.617015896 +0000 UTC m=+0.458330338 container start 753e25814128eb144bc50ba9d3baaa9827e325e93f52252f5d4577e684449e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:54:57.878 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 18:54:57 smithi045 ceph-mds[66666]: -- 172.21.15.45:0/2528922831 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e40b009a40 con 0x55e40afdc800 2024-04-25T18:54:57.878 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 18:54:57 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-g[66662]: starting mds.g at 2024-04-25T18:54:57.878 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 18:54:57 smithi045 bash[66647]: 753e25814128eb144bc50ba9d3baaa9827e325e93f52252f5d4577e684449e12 2024-04-25T18:54:57.878 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 18:54:57 smithi045 systemd[1]: Started Ceph mds.g for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:54:59.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:54:59.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:59.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:59.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:59.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:59.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:59.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:59.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:58 smithi045 ceph-mon[27776]: Deploying daemon mds.a on smithi045 2024-04-25T18:54:59.102 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:58 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.45:6838/3080448067,v1:172.21.15.45:6839/3080448067] up:boot 2024-04-25T18:54:59.103 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:58 smithi045 ceph-mon[27776]: fsmap 10 up:standby 2024-04-25T18:54:59.103 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:58 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:54:59.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:59.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:59.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:59.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:59.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:59.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:59.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:58 smithi062 ceph-mon[32844]: Deploying daemon mds.a on smithi045 2024-04-25T18:54:59.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:58 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.45:6838/3080448067,v1:172.21.15.45:6839/3080448067] up:boot 2024-04-25T18:54:59.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:58 smithi062 ceph-mon[32844]: fsmap 10 up:standby 2024-04-25T18:54:59.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:58 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:54:59.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:59.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:59.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:54:59.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:54:59.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:54:59.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:54:59.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:58 smithi079 ceph-mon[33002]: Deploying daemon mds.a on smithi045 2024-04-25T18:54:59.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:58 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.45:6838/3080448067,v1:172.21.15.45:6839/3080448067] up:boot 2024-04-25T18:54:59.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:58 smithi079 ceph-mon[33002]: fsmap 10 up:standby 2024-04-25T18:54:59.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:58 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:55:00.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:54:59 smithi079 ceph-mon[33002]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:00.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:54:59 smithi045 ceph-mon[27776]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:00.345 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:00.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:54:59 smithi062 ceph-mon[32844]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:01.215 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 18:55:01 smithi045 systemd[1]: Starting Ceph mds.a for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:55:01.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:01 smithi045 ceph-mon[27776]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:02.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:01 smithi079 ceph-mon[33002]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:02.246 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 18:55:01 smithi045 podman[67045]: 2024-04-25 18:55:01.887053828 +0000 UTC m=+0.038945839 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:55:02.247 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 18:55:02 smithi045 podman[67045]: 2024-04-25 18:55:02.015350985 +0000 UTC m=+0.167243016 container create 6f834433d81b2861bef19178f38eaecca216c2af72abd168813163ce5671c5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T18:55:02.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:01 smithi062 ceph-mon[32844]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:02.529 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 18:55:02 smithi045 podman[67045]: 2024-04-25 18:55:02.421255133 +0000 UTC m=+0.573147196 container init 6f834433d81b2861bef19178f38eaecca216c2af72abd168813163ce5671c5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:55:02.529 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 18:55:02 smithi045 podman[67045]: 2024-04-25 18:55:02.433812215 +0000 UTC m=+0.585704222 container start 6f834433d81b2861bef19178f38eaecca216c2af72abd168813163ce5671c5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T18:55:02.529 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 18:55:02 smithi045 ceph-mds[67081]: -- 172.21.15.45:0/1465483809 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c86af63a40 con 0x55c86af36c00 2024-04-25T18:55:02.529 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 18:55:02 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a[67075]: starting mds.a at 2024-04-25T18:55:02.859 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 18:55:02 smithi045 bash[67045]: 6f834433d81b2861bef19178f38eaecca216c2af72abd168813163ce5671c5da 2024-04-25T18:55:02.859 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 18:55:02 smithi045 systemd[1]: Started Ceph mds.a for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:55:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.038+0000 7f4d67e00700 1 -- 172.21.15.45:0/3977386792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d60105870 msgr2=0x7f4d601098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.038+0000 7f4d67e00700 1 --2- 172.21.15.45:0/3977386792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d60105870 0x7f4d601098c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c0098f0 tx=0x7f4d4c009c00 comp rx=0 tx=0).stop 2024-04-25T18:55:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.039+0000 7f4d67e00700 1 -- 172.21.15.45:0/3977386792 shutdown_connections 2024-04-25T18:55:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.039+0000 7f4d67e00700 1 --2- 172.21.15.45:0/3977386792 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d60109fa0 0x7f4d60111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.039+0000 7f4d67e00700 1 --2- 172.21.15.45:0/3977386792 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d60104ec0 0x7f4d601052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:03.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.039+0000 7f4d67e00700 1 --2- 172.21.15.45:0/3977386792 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d60105870 0x7f4d601098c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.039+0000 7f4d67e00700 1 -- 172.21.15.45:0/3977386792 >> 172.21.15.45:0/3977386792 conn(0x7f4d60100750 msgr2=0x7f4d60102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.039+0000 7f4d67e00700 1 -- 172.21.15.45:0/3977386792 shutdown_connections 2024-04-25T18:55:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.039+0000 7f4d67e00700 1 -- 172.21.15.45:0/3977386792 wait complete. 2024-04-25T18:55:03.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.040+0000 7f4d67e00700 1 Processor -- start 2024-04-25T18:55:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.040+0000 7f4d67e00700 1 -- start start 2024-04-25T18:55:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d67e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d60104ec0 0x7f4d6019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d67e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d60105870 0x7f4d6019b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d67e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d60109fa0 0x7f4d6019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d67e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60114e80 con 0x7f4d60104ec0 2024-04-25T18:55:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d67e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60114d00 con 0x7f4d60109fa0 2024-04-25T18:55:03.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d67e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4d60115000 con 0x7f4d60105870 2024-04-25T18:55:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d65400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d60104ec0 0x7f4d6019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:03.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d65e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d60109fa0 0x7f4d6019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d65e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d60109fa0 0x7f4d6019fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48426/0 (socket says 172.21.15.45:48426) 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d65e00700 1 -- 172.21.15.45:0/3799358825 learned_addr learned my addr 172.21.15.45:0/3799358825 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.041+0000 7f4d64a00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d60105870 0x7f4d6019b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d65e00700 1 -- 172.21.15.45:0/3799358825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d60105870 msgr2=0x7f4d6019b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d65e00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d60105870 0x7f4d6019b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d65e00700 1 -- 172.21.15.45:0/3799358825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d60104ec0 msgr2=0x7f4d6019b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d65e00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d60104ec0 0x7f4d6019b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d65e00700 1 -- 172.21.15.45:0/3799358825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d601a0450 con 0x7f4d60109fa0 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d64a00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d60105870 0x7f4d6019b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d65e00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d60109fa0 0x7f4d6019fd10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800c110 tx=0x7f4d5800c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d5e000700 1 -- 172.21.15.45:0/3799358825 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d580092e0 con 0x7f4d60109fa0 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d5e000700 1 -- 172.21.15.45:0/3799358825 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d58004bb0 con 0x7f4d60109fa0 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d5e000700 1 -- 172.21.15.45:0/3799358825 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58007be0 con 0x7f4d60109fa0 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d67e00700 1 -- 172.21.15.45:0/3799358825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d601a06e0 con 0x7f4d60109fa0 2024-04-25T18:55:03.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.042+0000 7f4d67e00700 1 -- 172.21.15.45:0/3799358825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d601a7fc0 con 0x7f4d60109fa0 2024-04-25T18:55:03.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.044+0000 7f4d67e00700 1 -- 172.21.15.45:0/3799358825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d60069730 con 0x7f4d60109fa0 2024-04-25T18:55:03.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.045+0000 7f4d5e000700 1 -- 172.21.15.45:0/3799358825 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4d5801a020 con 0x7f4d60109fa0 2024-04-25T18:55:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.047+0000 7f4d5e000700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4d480a3cc0 0x7f4d480a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.047+0000 7f4d5e000700 1 -- 172.21.15.45:0/3799358825 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f4d580cf9b0 con 0x7f4d60109fa0 2024-04-25T18:55:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.047+0000 7f4d65400700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4d480a3cc0 0x7f4d480a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:03.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.048+0000 7f4d65400700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4d480a3cc0 0x7f4d480a6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009c60 tx=0x7f4d54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:03.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.051+0000 7f4d5e000700 1 -- 172.21.15.45:0/3799358825 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d58010040 con 0x7f4d60109fa0 2024-04-25T18:55:03.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.338+0000 7f4d67e00700 1 -- 172.21.15.45:0/3799358825 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d6002df80 con 0x7f4d60109fa0 2024-04-25T18:55:03.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.339+0000 7f4d5e000700 1 -- 172.21.15.45:0/3799358825 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9638 (secure 0 0 0) 0x7f4d5809c790 con 0x7f4d60109fa0 2024-04-25T18:55:03.340 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:55:03.340 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-25T18:55:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 -- 172.21.15.45:0/3799358825 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4d480a3cc0 msgr2=0x7f4d480a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4d480a3cc0 0x7f4d480a6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009c60 tx=0x7f4d54009260 comp rx=0 tx=0).stop 2024-04-25T18:55:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 -- 172.21.15.45:0/3799358825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d60109fa0 msgr2=0x7f4d6019fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d60109fa0 0x7f4d6019fd10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800c110 tx=0x7f4d5800c420 comp rx=0 tx=0).stop 2024-04-25T18:55:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 -- 172.21.15.45:0/3799358825 shutdown_connections 2024-04-25T18:55:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4d480a3cc0 0x7f4d480a6180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4d60109fa0 0x7f4d6019fd10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d60104ec0 0x7f4d6019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:03.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 --2- 172.21.15.45:0/3799358825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d60105870 0x7f4d6019b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 -- 172.21.15.45:0/3799358825 >> 172.21.15.45:0/3799358825 conn(0x7f4d60100750 msgr2=0x7f4d60101b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 -- 172.21.15.45:0/3799358825 shutdown_connections 2024-04-25T18:55:03.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:03.342+0000 7f4d67e00700 1 -- 172.21.15.45:0/3799358825 wait complete. 2024-04-25T18:55:03.345 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 12 2024-04-25T18:55:03.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: Deploying daemon mds.b on smithi062 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] up:boot 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: fsmap 11 up:standby 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:55:03.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:03 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3799358825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:55:03.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:03.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:03.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:03.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:03.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:55:03.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:55:03.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:03.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: Deploying daemon mds.b on smithi062 2024-04-25T18:55:03.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] up:boot 2024-04-25T18:55:03.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: fsmap 11 up:standby 2024-04-25T18:55:03.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:55:03.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:03 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3799358825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:55:04.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:04.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:04.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:04.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:04.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T18:55:04.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T18:55:04.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:04.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: Deploying daemon mds.b on smithi062 2024-04-25T18:55:04.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] up:boot 2024-04-25T18:55:04.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: fsmap 11 up:standby 2024-04-25T18:55:04.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:55:04.116 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:03 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3799358825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:55:05.114 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:04 smithi062 systemd[1]: Starting Ceph mds.b for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T18:55:05.608 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:05 smithi062 podman[63889]: 2024-04-25 18:55:05.065551752 +0000 UTC m=+0.018663180 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T18:55:05.609 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:05 smithi062 podman[63889]: 2024-04-25 18:55:05.171852964 +0000 UTC m=+0.124964433 container create 5d57a1af6b5fdd50cc391ee5486e43b6393708869656f877ed4599dcdd8aed65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T18:55:05.609 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:05 smithi062 podman[63889]: 2024-04-25 18:55:05.382371476 +0000 UTC m=+0.335482912 container init 5d57a1af6b5fdd50cc391ee5486e43b6393708869656f877ed4599dcdd8aed65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T18:55:05.609 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:05 smithi062 podman[63889]: 2024-04-25 18:55:05.384962763 +0000 UTC m=+0.338074190 container start 5d57a1af6b5fdd50cc391ee5486e43b6393708869656f877ed4599dcdd8aed65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T18:55:05.609 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:05 smithi062 ceph-mds[63908]: -- 172.21.15.62:0/657314898 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563bc0a07a40 con 0x563bc09da800 2024-04-25T18:55:05.609 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:05 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b[63904]: starting mds.b at 2024-04-25T18:55:05.609 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:05 smithi062 bash[63889]: 5d57a1af6b5fdd50cc391ee5486e43b6393708869656f877ed4599dcdd8aed65 2024-04-25T18:55:05.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:05 smithi062 ceph-mon[32844]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:05.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:05 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:05.864 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:05 smithi062 systemd[1]: Started Ceph mds.b for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T18:55:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:05 smithi045 ceph-mon[27776]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:05 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:05 smithi079 ceph-mon[33002]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:06.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:05 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.190 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:55:06.643 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:06.688 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.688 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:06 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] up:boot 2024-04-25T18:55:06.689 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:06 smithi045 ceph-mon[27776]: fsmap 12 up:standby 2024-04-25T18:55:06.689 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:55:06.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:06.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:06 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:06 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] up:boot 2024-04-25T18:55:06.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:06 smithi062 ceph-mon[32844]: fsmap 12 up:standby 2024-04-25T18:55:06.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:55:06.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:06.955 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:06 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.959 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.959 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:06 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] up:boot 2024-04-25T18:55:06.959 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:06 smithi079 ceph-mon[33002]: fsmap 12 up:standby 2024-04-25T18:55:06.959 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:55:06.959 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.959 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.959 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:06.959 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:06.960 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:06 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:07.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:07 smithi045 ceph-mon[27776]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:07.817 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:07 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:08.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:07 smithi062 ceph-mon[32844]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:08.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:07 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:08.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:07 smithi079 ceph-mon[33002]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:08.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:07 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:08.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:08 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:08.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:08 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:08.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:08 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:09.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.046+0000 7f2aa8200700 1 -- 172.21.15.45:0/168810805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa0075590 msgr2=0x7f2aa010ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:09.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.046+0000 7f2aa8200700 1 --2- 172.21.15.45:0/168810805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa0075590 0x7f2aa010ef00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c009990 tx=0x7f2a8c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.046+0000 7f2aa8200700 1 -- 172.21.15.45:0/168810805 shutdown_connections 2024-04-25T18:55:09.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.046+0000 7f2aa8200700 1 --2- 172.21.15.45:0/168810805 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa0074c70 0x7f2aa0075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.046+0000 7f2aa8200700 1 --2- 172.21.15.45:0/168810805 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa010f440 0x7f2aa0111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.046+0000 7f2aa8200700 1 --2- 172.21.15.45:0/168810805 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa0075590 0x7f2aa010ef00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.046+0000 7f2aa8200700 1 -- 172.21.15.45:0/168810805 >> 172.21.15.45:0/168810805 conn(0x7f2aa0079ba0 msgr2=0x7f2aa0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:09.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.046+0000 7f2aa8200700 1 -- 172.21.15.45:0/168810805 shutdown_connections 2024-04-25T18:55:09.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.046+0000 7f2aa8200700 1 -- 172.21.15.45:0/168810805 wait complete. 2024-04-25T18:55:09.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.047+0000 7f2aa8200700 1 Processor -- start 2024-04-25T18:55:09.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.048+0000 7f2aa8200700 1 -- start start 2024-04-25T18:55:09.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.048+0000 7f2aa8200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa0074c70 0x7f2aa019b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:09.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.048+0000 7f2aa8200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa0075590 0x7f2aa019b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:09.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.048+0000 7f2aa8200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa010f440 0x7f2aa019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:09.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.048+0000 7f2aa8200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0114e20 con 0x7f2aa0074c70 2024-04-25T18:55:09.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.048+0000 7f2aa8200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0114ca0 con 0x7f2aa0075590 2024-04-25T18:55:09.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.048+0000 7f2aa8200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0114fa0 con 0x7f2aa010f440 2024-04-25T18:55:09.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.048+0000 7f2aa5800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa0074c70 0x7f2aa019b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:09.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa4e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa0075590 0x7f2aa019b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:09.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa4e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa0075590 0x7f2aa019b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:43586/0 (socket says 172.21.15.45:43586) 2024-04-25T18:55:09.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa5800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa0074c70 0x7f2aa019b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:35978/0 (socket says 172.21.15.45:35978) 2024-04-25T18:55:09.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa4e00700 1 -- 172.21.15.45:0/2509111303 learned_addr learned my addr 172.21.15.45:0/2509111303 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:09.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa6200700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa010f440 0x7f2aa019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:09.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa4e00700 1 -- 172.21.15.45:0/2509111303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa010f440 msgr2=0x7f2aa019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:09.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa4e00700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa010f440 0x7f2aa019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa4e00700 1 -- 172.21.15.45:0/2509111303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa0074c70 msgr2=0x7f2aa019b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:09.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa4e00700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa0074c70 0x7f2aa019b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa4e00700 1 -- 172.21.15.45:0/2509111303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa01a02b0 con 0x7f2aa0075590 2024-04-25T18:55:09.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa5800700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa0074c70 0x7f2aa019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:09.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.049+0000 7f2aa4e00700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa0075590 0x7f2aa019b830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0057c0 tx=0x7f2a8c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:09.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.050+0000 7f2a9e000700 1 -- 172.21.15.45:0/2509111303 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c027070 con 0x7f2aa0075590 2024-04-25T18:55:09.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.050+0000 7f2a9e000700 1 -- 172.21.15.45:0/2509111303 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a8c022470 con 0x7f2aa0075590 2024-04-25T18:55:09.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.050+0000 7f2aa8200700 1 -- 172.21.15.45:0/2509111303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa01a0540 con 0x7f2aa0075590 2024-04-25T18:55:09.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.050+0000 7f2a9e000700 1 -- 172.21.15.45:0/2509111303 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c017650 con 0x7f2aa0075590 2024-04-25T18:55:09.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.050+0000 7f2aa8200700 1 -- 172.21.15.45:0/2509111303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa01a7e20 con 0x7f2aa0075590 2024-04-25T18:55:09.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.051+0000 7f2aa8200700 1 -- 172.21.15.45:0/2509111303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa0069730 con 0x7f2aa0075590 2024-04-25T18:55:09.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.053+0000 7f2a9e000700 1 -- 172.21.15.45:0/2509111303 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2a8c02a070 con 0x7f2aa0075590 2024-04-25T18:55:09.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.054+0000 7f2a9e000700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2a880a3cc0 0x7f2a880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:09.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.054+0000 7f2a9e000700 1 -- 172.21.15.45:0/2509111303 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f2a8c0d0bc0 con 0x7f2aa0075590 2024-04-25T18:55:09.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.055+0000 7f2aa5800700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2a880a3cc0 0x7f2a880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:09.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.055+0000 7f2aa5800700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2a880a3cc0 0x7f2a880a6180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2a94011490 tx=0x7f2a94009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:09.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.059+0000 7f2a9e000700 1 -- 172.21.15.45:0/2509111303 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a8c013070 con 0x7f2aa0075590 2024-04-25T18:55:09.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:08 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:09.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:08 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:09.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:08 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:09.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:08 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:09.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:08 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:09.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:08 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:09.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.342+0000 7f2aa8200700 1 -- 172.21.15.45:0/2509111303 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2aa019c600 con 0x7f2aa0075590 2024-04-25T18:55:09.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.343+0000 7f2a9e000700 1 -- 172.21.15.45:0/2509111303 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10464 (secure 0 0 0) 0x7f2a8c09da30 con 0x7f2aa0075590 2024-04-25T18:55:09.345 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:55:09.345 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24485,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24493,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24505,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24512,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T18:55:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 -- 172.21.15.45:0/2509111303 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2a880a3cc0 msgr2=0x7f2a880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2a880a3cc0 0x7f2a880a6180 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2a94011490 tx=0x7f2a94009260 comp rx=0 tx=0).stop 2024-04-25T18:55:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 -- 172.21.15.45:0/2509111303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa0075590 msgr2=0x7f2aa019b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa0075590 0x7f2aa019b830 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0057c0 tx=0x7f2a8c00bfd0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 -- 172.21.15.45:0/2509111303 shutdown_connections 2024-04-25T18:55:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2a880a3cc0 0x7f2a880a6180 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2aa010f440 0x7f2aa019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2aa0074c70 0x7f2aa019b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 --2- 172.21.15.45:0/2509111303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2aa0075590 0x7f2aa019b830 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 -- 172.21.15.45:0/2509111303 >> 172.21.15.45:0/2509111303 conn(0x7f2aa0079ba0 msgr2=0x7f2aa00ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 -- 172.21.15.45:0/2509111303 shutdown_connections 2024-04-25T18:55:09.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:09.347+0000 7f2aa8200700 1 -- 172.21.15.45:0/2509111303 wait complete. 2024-04-25T18:55:09.350 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 13 2024-04-25T18:55:10.075 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:09 smithi045 ceph-mon[27776]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:10.075 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:09 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2509111303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:55:10.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:09 smithi062 ceph-mon[32844]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:10.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:09 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2509111303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:55:10.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-25T18:55:10.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:09 smithi079 ceph-mon[33002]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:10.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:09 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2509111303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:55:10.588 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:11.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:11 smithi045 ceph-mon[27776]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:11.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:11 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:11.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:11 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:11.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:11 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:11.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:11 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:11.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:11 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:12.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:11 smithi062 ceph-mon[32844]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:12.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:11 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:12.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:11 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:12.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:11 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:12.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:11 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:12.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:11 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:11 smithi079 ceph-mon[33002]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:11 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:11 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:11 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:11 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:11 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.830+0000 7f609c800700 1 -- 172.21.15.45:0/3612951994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f609406b0f0 msgr2=0x7f609410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.830+0000 7f609c800700 1 --2- 172.21.15.45:0/3612951994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f609406b0f0 0x7f609410f900 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6090009f90 tx=0x7f609000b330 comp rx=0 tx=0).stop 2024-04-25T18:55:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.830+0000 7f609c800700 1 -- 172.21.15.45:0/3612951994 shutdown_connections 2024-04-25T18:55:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.830+0000 7f609c800700 1 --2- 172.21.15.45:0/3612951994 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6094108a30 0x7f6094108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.830+0000 7f609c800700 1 --2- 172.21.15.45:0/3612951994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f609406b0f0 0x7f609410f900 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.830+0000 7f609c800700 1 --2- 172.21.15.45:0/3612951994 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f609406a750 0x7f609406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.830+0000 7f609c800700 1 -- 172.21.15.45:0/3612951994 >> 172.21.15.45:0/3612951994 conn(0x7f60940788f0 msgr2=0x7f6094078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.830+0000 7f609c800700 1 -- 172.21.15.45:0/3612951994 shutdown_connections 2024-04-25T18:55:12.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.831+0000 7f609c800700 1 -- 172.21.15.45:0/3612951994 wait complete. 2024-04-25T18:55:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.832+0000 7f609c800700 1 Processor -- start 2024-04-25T18:55:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.832+0000 7f609c800700 1 -- start start 2024-04-25T18:55:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.832+0000 7f609c800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f609406a750 0x7f609419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.832+0000 7f609c800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f609406b0f0 0x7f609419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.832+0000 7f609c800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6094108a30 0x7f609419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.832+0000 7f609c800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6094112c20 con 0x7f609406b0f0 2024-04-25T18:55:12.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.832+0000 7f609c800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6094112aa0 con 0x7f6094108a30 2024-04-25T18:55:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.832+0000 7f609c800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6094112da0 con 0x7f609406a750 2024-04-25T18:55:12.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f6099e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f609406a750 0x7f609419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f6099e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f609406a750 0x7f609419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:44436/0 (socket says 172.21.15.45:44436) 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f6099e00700 1 -- 172.21.15.45:0/2507139753 learned_addr learned my addr 172.21.15.45:0/2507139753 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f609a800700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6094108a30 0x7f609419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f6099400700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f609406b0f0 0x7f609419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f6099e00700 1 -- 172.21.15.45:0/2507139753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6094108a30 msgr2=0x7f609419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f6099e00700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6094108a30 0x7f609419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f6099e00700 1 -- 172.21.15.45:0/2507139753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f609406b0f0 msgr2=0x7f609419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f6099e00700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f609406b0f0 0x7f609419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f6099e00700 1 -- 172.21.15.45:0/2507139753 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60941a02f0 con 0x7f609406a750 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.833+0000 7f6099e00700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f609406a750 0x7f609419b2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f608800e9c0 tx=0x7f608800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.834+0000 7f608ea00700 1 -- 172.21.15.45:0/2507139753 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608800cbd0 con 0x7f609406a750 2024-04-25T18:55:12.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.834+0000 7f608ea00700 1 -- 172.21.15.45:0/2507139753 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608800cd70 con 0x7f609406a750 2024-04-25T18:55:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.834+0000 7f608ea00700 1 -- 172.21.15.45:0/2507139753 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088005580 con 0x7f609406a750 2024-04-25T18:55:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.834+0000 7f609c800700 1 -- 172.21.15.45:0/2507139753 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60941a05e0 con 0x7f609406a750 2024-04-25T18:55:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.834+0000 7f609c800700 1 -- 172.21.15.45:0/2507139753 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60941a7e60 con 0x7f609406a750 2024-04-25T18:55:12.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.835+0000 7f609c800700 1 -- 172.21.15.45:0/2507139753 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6094069730 con 0x7f609406a750 2024-04-25T18:55:12.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.837+0000 7f608ea00700 1 -- 172.21.15.45:0/2507139753 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6088004750 con 0x7f609406a750 2024-04-25T18:55:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.838+0000 7f608ea00700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f607c0a3cc0 0x7f607c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.838+0000 7f608ea00700 1 -- 172.21.15.45:0/2507139753 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7f60880d0120 con 0x7f609406a750 2024-04-25T18:55:12.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.838+0000 7f6099400700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f607c0a3cc0 0x7f607c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:12.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.839+0000 7f6099400700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f607c0a3cc0 0x7f607c0a6180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f609410fdb0 tx=0x7f6084009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:12.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:12.842+0000 7f608ea00700 1 -- 172.21.15.45:0/2507139753 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f608809cf00 con 0x7f609406a750 2024-04-25T18:55:12.941 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:12 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:12.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:12 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:13.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.097+0000 7f609c800700 1 -- 172.21.15.45:0/2507139753 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f60941a80d0 con 0x7f609406a750 2024-04-25T18:55:13.098 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.097+0000 7f608ea00700 1 -- 172.21.15.45:0/2507139753 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) ==== 84+0+65 (secure 0 0 0) 0x7f608809c800 con 0x7f609406a750 2024-04-25T18:55:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:55:13.099 INFO:teuthology.orchestra.run.smithi045.stdout:[ 2024-04-25T18:55:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T18:55:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "poolnum": 1, 2024-04-25T18:55:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: "poolname": ".mgr" 2024-04-25T18:55:13.099 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T18:55:13.099 INFO:teuthology.orchestra.run.smithi045.stdout:] 2024-04-25T18:55:13.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 -- 172.21.15.45:0/2507139753 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f607c0a3cc0 msgr2=0x7f607c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:13.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f607c0a3cc0 0x7f607c0a6180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f609410fdb0 tx=0x7f6084009260 comp rx=0 tx=0).stop 2024-04-25T18:55:13.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 -- 172.21.15.45:0/2507139753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f609406a750 msgr2=0x7f609419b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:13.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f609406a750 0x7f609419b2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f608800e9c0 tx=0x7f608800ecd0 comp rx=0 tx=0).stop 2024-04-25T18:55:13.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 -- 172.21.15.45:0/2507139753 shutdown_connections 2024-04-25T18:55:13.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f607c0a3cc0 0x7f607c0a6180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:13.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6094108a30 0x7f609419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:13.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f609406a750 0x7f609419b2e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:13.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 --2- 172.21.15.45:0/2507139753 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f609406b0f0 0x7f609419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:13.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 -- 172.21.15.45:0/2507139753 >> 172.21.15.45:0/2507139753 conn(0x7f60940788f0 msgr2=0x7f60940ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:13.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 -- 172.21.15.45:0/2507139753 shutdown_connections 2024-04-25T18:55:13.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:13.101+0000 7f609c800700 1 -- 172.21.15.45:0/2507139753 wait complete. 2024-04-25T18:55:13.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:12 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:55:14.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:13 smithi062 ceph-mon[32844]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:14.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:13 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2507139753' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T18:55:14.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.115 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:13 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:13 smithi079 ceph-mon[33002]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:14.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:13 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2507139753' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T18:55:14.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:13 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:13 smithi045 ceph-mon[27776]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:13 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2507139753' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T18:55:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:13 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:14.702 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T18:55:14.703 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T18:55:15.165 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:15.899 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:15 smithi045 ceph-mon[27776]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:16.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:15 smithi062 ceph-mon[32844]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:16.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:15 smithi079 ceph-mon[33002]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.321+0000 7fd8a4a00700 1 -- 172.21.15.45:0/1668023998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a0109fa0 msgr2=0x7fd8a0111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.321+0000 7fd8a4a00700 1 --2- 172.21.15.45:0/1668023998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a0109fa0 0x7fd8a0111b70 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd898009fc0 tx=0x7fd89800b330 comp rx=0 tx=0).stop 2024-04-25T18:55:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.321+0000 7fd8a4a00700 1 -- 172.21.15.45:0/1668023998 shutdown_connections 2024-04-25T18:55:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.321+0000 7fd8a4a00700 1 --2- 172.21.15.45:0/1668023998 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a0109fa0 0x7fd8a0111b70 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.321+0000 7fd8a4a00700 1 --2- 172.21.15.45:0/1668023998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0104ec0 0x7fd8a01052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.321+0000 7fd8a4a00700 1 --2- 172.21.15.45:0/1668023998 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8a0105870 0x7fd8a01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:17.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.321+0000 7fd8a4a00700 1 -- 172.21.15.45:0/1668023998 >> 172.21.15.45:0/1668023998 conn(0x7fd8a0100750 msgr2=0x7fd8a0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:17.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.321+0000 7fd8a4a00700 1 -- 172.21.15.45:0/1668023998 shutdown_connections 2024-04-25T18:55:17.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.322+0000 7fd8a4a00700 1 -- 172.21.15.45:0/1668023998 wait complete. 2024-04-25T18:55:17.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.323+0000 7fd8a4a00700 1 Processor -- start 2024-04-25T18:55:17.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.323+0000 7fd8a4a00700 1 -- start start 2024-04-25T18:55:17.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.323+0000 7fd8a4a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0104ec0 0x7fd8a019b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:17.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.323+0000 7fd8a4a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8a0105870 0x7fd8a019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:17.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.323+0000 7fd8a4a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a0109fa0 0x7fd8a019fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:17.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.323+0000 7fd8a4a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0114e30 con 0x7fd8a0104ec0 2024-04-25T18:55:17.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.323+0000 7fd8a4a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0114cb0 con 0x7fd8a0105870 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.323+0000 7fd8a4a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a0114fb0 con 0x7fd8a0109fa0 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd89de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0104ec0 0x7fd8a019b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd89de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0104ec0 0x7fd8a019b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49034/0 (socket says 172.21.15.45:49034) 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd89de00700 1 -- 172.21.15.45:0/2370301464 learned_addr learned my addr 172.21.15.45:0/2370301464 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd89e800700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a0109fa0 0x7fd8a019fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd897e00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8a0105870 0x7fd8a019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd89de00700 1 -- 172.21.15.45:0/2370301464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a0109fa0 msgr2=0x7fd8a019fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd89de00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a0109fa0 0x7fd8a019fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd89de00700 1 -- 172.21.15.45:0/2370301464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8a0105870 msgr2=0x7fd8a019b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd89de00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8a0105870 0x7fd8a019b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd89de00700 1 -- 172.21.15.45:0/2370301464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a01a0270 con 0x7fd8a0104ec0 2024-04-25T18:55:17.327 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.324+0000 7fd89e800700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a0109fa0 0x7fd8a019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:17.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.325+0000 7fd89de00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0104ec0 0x7fd8a019b260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd89000e9a0 tx=0x7fd89000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:17.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.325+0000 7fd896a00700 1 -- 172.21.15.45:0/2370301464 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89000c9c0 con 0x7fd8a0104ec0 2024-04-25T18:55:17.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.325+0000 7fd8a4a00700 1 -- 172.21.15.45:0/2370301464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8a01a0560 con 0x7fd8a0104ec0 2024-04-25T18:55:17.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.325+0000 7fd896a00700 1 -- 172.21.15.45:0/2370301464 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd89000cb60 con 0x7fd8a0104ec0 2024-04-25T18:55:17.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.325+0000 7fd896a00700 1 -- 172.21.15.45:0/2370301464 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd890018680 con 0x7fd8a0104ec0 2024-04-25T18:55:17.328 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.325+0000 7fd8a4a00700 1 -- 172.21.15.45:0/2370301464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8a01a7de0 con 0x7fd8a0104ec0 2024-04-25T18:55:17.329 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.327+0000 7fd8a4a00700 1 -- 172.21.15.45:0/2370301464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8a0069730 con 0x7fd8a0104ec0 2024-04-25T18:55:17.332 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.330+0000 7fd896a00700 1 -- 172.21.15.45:0/2370301464 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd890010430 con 0x7fd8a0104ec0 2024-04-25T18:55:17.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.331+0000 7fd896a00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd8840a3cc0 0x7fd8840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:17.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.332+0000 7fd896a00700 1 -- 172.21.15.45:0/2370301464 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7127+0+0 (secure 0 0 0) 0x7fd8900d03c0 con 0x7fd8a0104ec0 2024-04-25T18:55:17.333 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.332+0000 7fd897e00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd8840a3cc0 0x7fd8840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:17.334 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.332+0000 7fd897e00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd8840a3cc0 0x7fd8840a6180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0103e50 tx=0x7fd888009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:17.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.336+0000 7fd896a00700 1 -- 172.21.15.45:0/2370301464 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd89009d1a0 con 0x7fd8a0104ec0 2024-04-25T18:55:17.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:17.573+0000 7fd8a4a00700 1 -- 172.21.15.45:0/2370301464 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd8a002df80 con 0x7fd8a0104ec0 2024-04-25T18:55:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:17 smithi062 ceph-mon[32844]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:17 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2370301464' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T18:55:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:17 smithi079 ceph-mon[33002]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:17 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2370301464' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T18:55:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:17 smithi045 ceph-mon[27776]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:55:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:55:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:55:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:17 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2370301464' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T18:55:18.140 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.137+0000 7fd896a00700 1 -- 172.21.15.45:0/2370301464 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) ==== 138+0+0 (secure 0 0 0) 0x7fd89009caa0 con 0x7fd8a0104ec0 2024-04-25T18:55:18.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 -- 172.21.15.45:0/2370301464 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd8840a3cc0 msgr2=0x7fd8840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd8840a3cc0 0x7fd8840a6180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0103e50 tx=0x7fd888009310 comp rx=0 tx=0).stop 2024-04-25T18:55:18.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 -- 172.21.15.45:0/2370301464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0104ec0 msgr2=0x7fd8a019b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:18.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0104ec0 0x7fd8a019b260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd89000e9a0 tx=0x7fd89000ecb0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 -- 172.21.15.45:0/2370301464 shutdown_connections 2024-04-25T18:55:18.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd8840a3cc0 0x7fd8840a6180 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8a0109fa0 0x7fd8a019fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8a0104ec0 0x7fd8a019b260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 --2- 172.21.15.45:0/2370301464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd8a0105870 0x7fd8a019b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 -- 172.21.15.45:0/2370301464 >> 172.21.15.45:0/2370301464 conn(0x7fd8a0100750 msgr2=0x7fd8a0101280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.145+0000 7fd8a4a00700 1 -- 172.21.15.45:0/2370301464 shutdown_connections 2024-04-25T18:55:18.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:18.146+0000 7fd8a4a00700 1 -- 172.21.15.45:0/2370301464 wait complete. 2024-04-25T18:55:18.150 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' created 2024-04-25T18:55:19.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T18:55:19.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:19 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2370301464' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T18:55:19.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:19 smithi079 ceph-mon[33002]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T18:55:19.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:19 smithi079 ceph-mon[33002]: pgmap v317: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:19.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:19 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2370301464' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T18:55:19.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:19 smithi045 ceph-mon[27776]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T18:55:19.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:19 smithi045 ceph-mon[27776]: pgmap v317: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:19.545 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:19 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2370301464' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T18:55:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:19 smithi062 ceph-mon[32844]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T18:55:19.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:19 smithi062 ceph-mon[32844]: pgmap v317: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:20.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:20 smithi079 ceph-mon[33002]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:55:20.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:20 smithi079 ceph-mon[33002]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T18:55:20.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:20 smithi045 ceph-mon[27776]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:55:20.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:20 smithi045 ceph-mon[27776]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T18:55:20.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:20 smithi062 ceph-mon[32844]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:55:20.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:20 smithi062 ceph-mon[32844]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T18:55:21.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:21 smithi045 ceph-mon[27776]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T18:55:21.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:21 smithi045 ceph-mon[27776]: pgmap v320: 33 pgs: 1 creating+activating, 9 creating+peering, 12 active+clean, 11 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:21.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:21 smithi079 ceph-mon[33002]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T18:55:21.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:21 smithi079 ceph-mon[33002]: pgmap v320: 33 pgs: 1 creating+activating, 9 creating+peering, 12 active+clean, 11 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:21.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:21 smithi062 ceph-mon[32844]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T18:55:21.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:21 smithi062 ceph-mon[32844]: pgmap v320: 33 pgs: 1 creating+activating, 9 creating+peering, 12 active+clean, 11 unknown; 577 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.812+0000 7f5675c00700 1 -- 172.21.15.45:0/3624946830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5670103450 msgr2=0x7f567010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.812+0000 7f5675c00700 1 --2- 172.21.15.45:0/3624946830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5670103450 0x7f567010f8b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5668009990 tx=0x7f5668009ca0 comp rx=0 tx=0).stop 2024-04-25T18:55:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.812+0000 7f5675c00700 1 -- 172.21.15.45:0/3624946830 shutdown_connections 2024-04-25T18:55:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.812+0000 7f5675c00700 1 --2- 172.21.15.45:0/3624946830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5670108a50 0x7f5670108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:21.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.812+0000 7f5675c00700 1 --2- 172.21.15.45:0/3624946830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5670103450 0x7f567010f8b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.812+0000 7f5675c00700 1 --2- 172.21.15.45:0/3624946830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5670102a90 0x7f5670102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.812+0000 7f5675c00700 1 -- 172.21.15.45:0/3624946830 >> 172.21.15.45:0/3624946830 conn(0x7f5670079ba0 msgr2=0x7f5670079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.812+0000 7f5675c00700 1 -- 172.21.15.45:0/3624946830 shutdown_connections 2024-04-25T18:55:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.812+0000 7f5675c00700 1 -- 172.21.15.45:0/3624946830 wait complete. 2024-04-25T18:55:21.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.813+0000 7f5675c00700 1 Processor -- start 2024-04-25T18:55:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f5675c00700 1 -- start start 2024-04-25T18:55:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f5675c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5670102a90 0x7f56701991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:21.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f5675c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5670103450 0x7f5670199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f5675c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5670108a50 0x7f567019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f5675c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5670112c50 con 0x7f5670103450 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f5675c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5670112ad0 con 0x7f5670102a90 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f5675c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5670112dd0 con 0x7f5670108a50 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f566ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5670102a90 0x7f56701991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f566ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5670102a90 0x7f56701991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:53068/0 (socket says 172.21.15.45:53068) 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f5666000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5670103450 0x7f5670199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.814+0000 7f566ea00700 1 -- 172.21.15.45:0/2432620277 learned_addr learned my addr 172.21.15.45:0/2432620277 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.815+0000 7f566f400700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5670108a50 0x7f567019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.815+0000 7f566ea00700 1 -- 172.21.15.45:0/2432620277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5670108a50 msgr2=0x7f567019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.815+0000 7f566ea00700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5670108a50 0x7f567019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.815+0000 7f566ea00700 1 -- 172.21.15.45:0/2432620277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5670103450 msgr2=0x7f5670199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:21.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.815+0000 7f566ea00700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5670103450 0x7f5670199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.815+0000 7f566ea00700 1 -- 172.21.15.45:0/2432620277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567019e1f0 con 0x7f5670102a90 2024-04-25T18:55:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.815+0000 7f566f400700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5670108a50 0x7f567019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:55:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.815+0000 7f566ea00700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5670102a90 0x7f56701991e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f566000e9a0 tx=0x7f566000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.815+0000 7f5667e00700 1 -- 172.21.15.45:0/2432620277 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566000cbd0 con 0x7f5670102a90 2024-04-25T18:55:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.816+0000 7f5667e00700 1 -- 172.21.15.45:0/2432620277 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566000cd70 con 0x7f5670102a90 2024-04-25T18:55:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.816+0000 7f5667e00700 1 -- 172.21.15.45:0/2432620277 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660018680 con 0x7f5670102a90 2024-04-25T18:55:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.816+0000 7f5675c00700 1 -- 172.21.15.45:0/2432620277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567019e4e0 con 0x7f5670102a90 2024-04-25T18:55:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.816+0000 7f5675c00700 1 -- 172.21.15.45:0/2432620277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56701a5d60 con 0x7f5670102a90 2024-04-25T18:55:21.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.817+0000 7f5675c00700 1 -- 172.21.15.45:0/2432620277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5670069730 con 0x7f5670102a90 2024-04-25T18:55:21.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.819+0000 7f5667e00700 1 -- 172.21.15.45:0/2432620277 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5660010430 con 0x7f5670102a90 2024-04-25T18:55:21.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.820+0000 7f5667e00700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56540a3c40 0x7f56540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.821+0000 7f5667e00700 1 -- 172.21.15.45:0/2432620277 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7497+0+0 (secure 0 0 0) 0x7f56600d0380 con 0x7f5670102a90 2024-04-25T18:55:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.821+0000 7f5666000700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56540a3c40 0x7f56540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:21.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.821+0000 7f5666000700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56540a3c40 0x7f56540a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f567007a2c0 tx=0x7f565800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:21.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:21.823+0000 7f5667e00700 1 -- 172.21.15.45:0/2432620277 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f566009cff0 con 0x7f5670102a90 2024-04-25T18:55:22.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.055+0000 7f5675c00700 1 -- 172.21.15.45:0/2432620277 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f567002df80 con 0x7f5670102a90 2024-04-25T18:55:22.196 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.193+0000 7f5667e00700 1 -- 172.21.15.45:0/2432620277 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v89) ==== 170+0+0 (secure 0 0 0) 0x7f566009c8f0 con 0x7f5670102a90 2024-04-25T18:55:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.203+0000 7f5675c00700 1 -- 172.21.15.45:0/2432620277 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56540a3c40 msgr2=0x7f56540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.203+0000 7f5675c00700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56540a3c40 0x7f56540a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f567007a2c0 tx=0x7f565800b040 comp rx=0 tx=0).stop 2024-04-25T18:55:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.203+0000 7f5675c00700 1 -- 172.21.15.45:0/2432620277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5670102a90 msgr2=0x7f56701991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.203+0000 7f5675c00700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5670102a90 0x7f56701991e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f566000e9a0 tx=0x7f566000ecb0 comp rx=0 tx=0).stop 2024-04-25T18:55:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.204+0000 7f5675c00700 1 -- 172.21.15.45:0/2432620277 shutdown_connections 2024-04-25T18:55:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.204+0000 7f5675c00700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56540a3c40 0x7f56540a6100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.204+0000 7f5675c00700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5670102a90 0x7f56701991e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.204+0000 7f5675c00700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5670108a50 0x7f567019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.204+0000 7f5675c00700 1 --2- 172.21.15.45:0/2432620277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5670103450 0x7f5670199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:22.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.204+0000 7f5675c00700 1 -- 172.21.15.45:0/2432620277 >> 172.21.15.45:0/2432620277 conn(0x7f5670079ba0 msgr2=0x7f5670101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.205+0000 7f5675c00700 1 -- 172.21.15.45:0/2432620277 shutdown_connections 2024-04-25T18:55:22.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:22.205+0000 7f5675c00700 1 -- 172.21.15.45:0/2432620277 wait complete. 2024-04-25T18:55:22.212 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' created 2024-04-25T18:55:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:22 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2432620277' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:55:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:22 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:55:22.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:22 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2432620277' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:55:22.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:22 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:55:22.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:22 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2432620277' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:55:22.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:22 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T18:55:23.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T18:55:23.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:23 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T18:55:23.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:23 smithi045 ceph-mon[27776]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T18:55:23.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:23 smithi045 ceph-mon[27776]: pgmap v322: 97 pgs: 1 creating+activating, 9 creating+peering, 21 active+clean, 66 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:23.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:23 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T18:55:23.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:23 smithi062 ceph-mon[32844]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T18:55:23.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:23 smithi062 ceph-mon[32844]: pgmap v322: 97 pgs: 1 creating+activating, 9 creating+peering, 21 active+clean, 66 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:23.632 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:23.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:23 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T18:55:23.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:23 smithi079 ceph-mon[33002]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T18:55:23.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:23 smithi079 ceph-mon[33002]: pgmap v322: 97 pgs: 1 creating+activating, 9 creating+peering, 21 active+clean, 66 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:24.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:24 smithi045 ceph-mon[27776]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T18:55:24.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:24 smithi062 ceph-mon[32844]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T18:55:24.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:24 smithi079 ceph-mon[33002]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T18:55:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:25 smithi045 ceph-mon[27776]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T18:55:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:25 smithi045 ceph-mon[27776]: pgmap v325: 97 pgs: 1 creating+activating, 8 creating+peering, 38 active+clean, 50 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:25.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:25 smithi062 ceph-mon[32844]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T18:55:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:25 smithi062 ceph-mon[32844]: pgmap v325: 97 pgs: 1 creating+activating, 8 creating+peering, 38 active+clean, 50 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:25.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:25 smithi079 ceph-mon[33002]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T18:55:25.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:25 smithi079 ceph-mon[33002]: pgmap v325: 97 pgs: 1 creating+activating, 8 creating+peering, 38 active+clean, 50 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:25.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.726+0000 7f974fc00700 1 -- 172.21.15.45:0/979971483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9748103460 msgr2=0x7f974810f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:25.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.726+0000 7f974fc00700 1 --2- 172.21.15.45:0/979971483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9748103460 0x7f974810f940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f974000b0a0 tx=0x7f974000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:55:25.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.726+0000 7f974fc00700 1 -- 172.21.15.45:0/979971483 shutdown_connections 2024-04-25T18:55:25.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.726+0000 7f974fc00700 1 --2- 172.21.15.45:0/979971483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9748108a70 0x7f9748108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:25.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.726+0000 7f974fc00700 1 --2- 172.21.15.45:0/979971483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9748103460 0x7f974810f940 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:25.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.726+0000 7f974fc00700 1 --2- 172.21.15.45:0/979971483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9748102ac0 0x7f9748102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:25.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.726+0000 7f974fc00700 1 -- 172.21.15.45:0/979971483 >> 172.21.15.45:0/979971483 conn(0x7f9748079b10 msgr2=0x7f9748079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:25.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.727+0000 7f974fc00700 1 -- 172.21.15.45:0/979971483 shutdown_connections 2024-04-25T18:55:25.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.727+0000 7f974fc00700 1 -- 172.21.15.45:0/979971483 wait complete. 2024-04-25T18:55:25.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.728+0000 7f974fc00700 1 Processor -- start 2024-04-25T18:55:25.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.728+0000 7f974fc00700 1 -- start start 2024-04-25T18:55:25.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.728+0000 7f974fc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9748102ac0 0x7f974819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:25.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.728+0000 7f974fc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9748103460 0x7f974819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:25.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.728+0000 7f974fc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9748108a70 0x7f974819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:25.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.728+0000 7f974fc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9748112c50 con 0x7f9748108a70 2024-04-25T18:55:25.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.728+0000 7f974fc00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9748112ad0 con 0x7f9748102ac0 2024-04-25T18:55:25.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.728+0000 7f974fc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9748112dd0 con 0x7f9748103460 2024-04-25T18:55:25.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f974dc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9748108a70 0x7f974819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:25.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f974d200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9748102ac0 0x7f974819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:25.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f9747e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9748103460 0x7f974819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:25.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f9747e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9748103460 0x7f974819b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:35530/0 (socket says 172.21.15.45:35530) 2024-04-25T18:55:25.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f9747e00700 1 -- 172.21.15.45:0/807739301 learned_addr learned my addr 172.21.15.45:0/807739301 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:25.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f9747e00700 1 -- 172.21.15.45:0/807739301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9748102ac0 msgr2=0x7f974819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f9747e00700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9748102ac0 0x7f974819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f9747e00700 1 -- 172.21.15.45:0/807739301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9748108a70 msgr2=0x7f974819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f9747e00700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9748108a70 0x7f974819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f9747e00700 1 -- 172.21.15.45:0/807739301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97481a0310 con 0x7f9748103460 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f974d200700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9748102ac0 0x7f974819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.729+0000 7f974dc00700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9748108a70 0x7f974819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.730+0000 7f9747e00700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9748103460 0x7f974819b840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f973400d950 tx=0x7f973400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.730+0000 7f9745600700 1 -- 172.21.15.45:0/807739301 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734016070 con 0x7f9748103460 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.730+0000 7f974fc00700 1 -- 172.21.15.45:0/807739301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97481a0600 con 0x7f9748103460 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.730+0000 7f9745600700 1 -- 172.21.15.45:0/807739301 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f973400fd40 con 0x7f9748103460 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.730+0000 7f9745600700 1 -- 172.21.15.45:0/807739301 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973400e3b0 con 0x7f9748103460 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.730+0000 7f974fc00700 1 -- 172.21.15.45:0/807739301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97481a7e80 con 0x7f9748103460 2024-04-25T18:55:25.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.731+0000 7f974fc00700 1 -- 172.21.15.45:0/807739301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9748069730 con 0x7f9748103460 2024-04-25T18:55:25.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.733+0000 7f9745600700 1 -- 172.21.15.45:0/807739301 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9734019070 con 0x7f9748103460 2024-04-25T18:55:25.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.735+0000 7f9745600700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f97300a3cc0 0x7f97300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:25.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.735+0000 7f9745600700 1 -- 172.21.15.45:0/807739301 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7879+0+0 (secure 0 0 0) 0x7f97340cfce0 con 0x7f9748103460 2024-04-25T18:55:25.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.735+0000 7f974d200700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f97300a3cc0 0x7f97300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:25.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.736+0000 7f974d200700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f97300a3cc0 0x7f97300a6180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f973c009fd0 tx=0x7f973c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:25.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:25.737+0000 7f9745600700 1 -- 172.21.15.45:0/807739301 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f973409c7d0 con 0x7f9748103460 2024-04-25T18:55:26.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.022+0000 7f974fc00700 1 -- 172.21.15.45:0/807739301 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f97481a80b0 con 0x7f9748103460 2024-04-25T18:55:26.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.239+0000 7f9745600700 1 -- 172.21.15.45:0/807739301 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f973409c0d0 con 0x7f9748103460 2024-04-25T18:55:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.243+0000 7f974fc00700 1 -- 172.21.15.45:0/807739301 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f97300a3cc0 msgr2=0x7f97300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.243+0000 7f974fc00700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f97300a3cc0 0x7f97300a6180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f973c009fd0 tx=0x7f973c009260 comp rx=0 tx=0).stop 2024-04-25T18:55:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.243+0000 7f974fc00700 1 -- 172.21.15.45:0/807739301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9748103460 msgr2=0x7f974819b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.243+0000 7f974fc00700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9748103460 0x7f974819b840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f973400d950 tx=0x7f973400dc60 comp rx=0 tx=0).stop 2024-04-25T18:55:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.244+0000 7f974fc00700 1 -- 172.21.15.45:0/807739301 shutdown_connections 2024-04-25T18:55:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.244+0000 7f974fc00700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f97300a3cc0 0x7f97300a6180 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.244+0000 7f974fc00700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9748102ac0 0x7f974819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:26.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.244+0000 7f974fc00700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9748108a70 0x7f974819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:26.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.244+0000 7f974fc00700 1 --2- 172.21.15.45:0/807739301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9748103460 0x7f974819b840 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:26.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.244+0000 7f974fc00700 1 -- 172.21.15.45:0/807739301 >> 172.21.15.45:0/807739301 conn(0x7f9748079b10 msgr2=0x7f9748100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:26.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.245+0000 7f974fc00700 1 -- 172.21.15.45:0/807739301 shutdown_connections 2024-04-25T18:55:26.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:26.245+0000 7f974fc00700 1 -- 172.21.15.45:0/807739301 wait complete. 2024-04-25T18:55:26.253 INFO:teuthology.orchestra.run.smithi045.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T18:55:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:26 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/807739301' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:55:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:26 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:55:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:26 smithi045 ceph-mon[27776]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T18:55:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:26 smithi045 ceph-mon[27776]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:55:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:26 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/807739301' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:55:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:26 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:55:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:26 smithi062 ceph-mon[32844]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T18:55:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:26 smithi062 ceph-mon[32844]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:55:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:26 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/807739301' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:55:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:26 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T18:55:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:26 smithi079 ceph-mon[33002]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T18:55:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:26 smithi079 ceph-mon[33002]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T18:55:27.090 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:55:27.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:27 smithi045 ceph-mon[27776]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T18:55:27.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:27 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T18:55:27.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:27 smithi045 ceph-mon[27776]: pgmap v327: 97 pgs: 8 creating+peering, 85 active+clean, 4 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:27.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:27 smithi045 ceph-mon[27776]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T18:55:27.303 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:27 smithi045 ceph-mon[27776]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T18:55:27.610 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:27 smithi062 ceph-mon[32844]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T18:55:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:27 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T18:55:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:27 smithi062 ceph-mon[32844]: pgmap v327: 97 pgs: 8 creating+peering, 85 active+clean, 4 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:27 smithi062 ceph-mon[32844]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T18:55:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:27 smithi062 ceph-mon[32844]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T18:55:27.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:27 smithi079 ceph-mon[33002]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T18:55:27.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:27 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T18:55:27.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:27 smithi079 ceph-mon[33002]: pgmap v327: 97 pgs: 8 creating+peering, 85 active+clean, 4 unknown; 577 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:55:27.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:27 smithi079 ceph-mon[33002]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T18:55:27.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:27 smithi079 ceph-mon[33002]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T18:55:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:28 smithi045 ceph-mon[27776]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T18:55:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:28 smithi045 ceph-mon[27776]: Cluster is now healthy 2024-04-25T18:55:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:28 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] up:active 2024-04-25T18:55:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:28 smithi045 ceph-mon[27776]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:28 smithi045 ceph-mon[27776]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:28 smithi062 ceph-mon[32844]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T18:55:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:28 smithi062 ceph-mon[32844]: Cluster is now healthy 2024-04-25T18:55:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:28 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] up:active 2024-04-25T18:55:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:28 smithi062 ceph-mon[32844]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:28 smithi062 ceph-mon[32844]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:28 smithi079 ceph-mon[33002]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T18:55:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:28 smithi079 ceph-mon[33002]: Cluster is now healthy 2024-04-25T18:55:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:28 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] up:active 2024-04-25T18:55:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:28 smithi079 ceph-mon[33002]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:28 smithi079 ceph-mon[33002]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:29.372 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:29 smithi045 ceph-mon[27776]: pgmap v328: 97 pgs: 97 active+clean; 578 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-25T18:55:29.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:29 smithi062 ceph-mon[32844]: pgmap v328: 97 pgs: 97 active+clean; 578 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-25T18:55:29.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:29 smithi079 ceph-mon[33002]: pgmap v328: 97 pgs: 97 active+clean; 578 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-04-25T18:55:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.895+0000 7f2e45600700 1 -- 172.21.15.45:0/2820033059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40074c70 msgr2=0x7f2e40075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.895+0000 7f2e45600700 1 --2- 172.21.15.45:0/2820033059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40074c70 0x7f2e40075050 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000b0a0 tx=0x7f2e3000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:55:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.895+0000 7f2e45600700 1 -- 172.21.15.45:0/2820033059 shutdown_connections 2024-04-25T18:55:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.895+0000 7f2e45600700 1 --2- 172.21.15.45:0/2820033059 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e40113c40 0x7f2e40116030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.895+0000 7f2e45600700 1 --2- 172.21.15.45:0/2820033059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40074c70 0x7f2e40075050 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.895+0000 7f2e45600700 1 --2- 172.21.15.45:0/2820033059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e40075590 0x7f2e40113700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.895+0000 7f2e45600700 1 -- 172.21.15.45:0/2820033059 >> 172.21.15.45:0/2820033059 conn(0x7f2e40079bf0 msgr2=0x7f2e4007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.896+0000 7f2e45600700 1 -- 172.21.15.45:0/2820033059 shutdown_connections 2024-04-25T18:55:29.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.896+0000 7f2e45600700 1 -- 172.21.15.45:0/2820033059 wait complete. 2024-04-25T18:55:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.897+0000 7f2e45600700 1 Processor -- start 2024-04-25T18:55:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.897+0000 7f2e45600700 1 -- start start 2024-04-25T18:55:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.897+0000 7f2e45600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e40074c70 0x7f2e401a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:29.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.897+0000 7f2e45600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e40075590 0x7f2e401a5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.897+0000 7f2e45600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40113c40 0x7f2e4019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.897+0000 7f2e45600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40119310 con 0x7f2e40075590 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.897+0000 7f2e45600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40119190 con 0x7f2e40074c70 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.897+0000 7f2e45600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2e40119490 con 0x7f2e40113c40 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e36000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e40075590 0x7f2e401a5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e3ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e40074c70 0x7f2e401a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e36000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e40075590 0x7f2e401a5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51366/0 (socket says 172.21.15.45:51366) 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e36000700 1 -- 172.21.15.45:0/1840614457 learned_addr learned my addr 172.21.15.45:0/1840614457 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e3f400700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40113c40 0x7f2e4019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e3f400700 1 -- 172.21.15.45:0/1840614457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e40074c70 msgr2=0x7f2e401a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e3f400700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e40074c70 0x7f2e401a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e3f400700 1 -- 172.21.15.45:0/1840614457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e40075590 msgr2=0x7f2e401a5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e3f400700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e40075590 0x7f2e401a5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:29.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e3f400700 1 -- 172.21.15.45:0/1840614457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4019ff70 con 0x7f2e40113c40 2024-04-25T18:55:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e3ea00700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e40074c70 0x7f2e401a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.898+0000 7f2e36000700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e40075590 0x7f2e401a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:29.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.899+0000 7f2e3f400700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40113c40 0x7f2e4019f690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800e940 tx=0x7f2e3800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.899+0000 7f2e37e00700 1 -- 172.21.15.45:0/1840614457 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004220 con 0x7f2e40113c40 2024-04-25T18:55:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.899+0000 7f2e37e00700 1 -- 172.21.15.45:0/1840614457 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e380043c0 con 0x7f2e40113c40 2024-04-25T18:55:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.899+0000 7f2e37e00700 1 -- 172.21.15.45:0/1840614457 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38010700 con 0x7f2e40113c40 2024-04-25T18:55:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.899+0000 7f2e45600700 1 -- 172.21.15.45:0/1840614457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e401a0260 con 0x7f2e40113c40 2024-04-25T18:55:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.899+0000 7f2e45600700 1 -- 172.21.15.45:0/1840614457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e401ac540 con 0x7f2e40113c40 2024-04-25T18:55:29.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.900+0000 7f2e45600700 1 -- 172.21.15.45:0/1840614457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e400763b0 con 0x7f2e40113c40 2024-04-25T18:55:29.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.902+0000 7f2e37e00700 1 -- 172.21.15.45:0/1840614457 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2e38004750 con 0x7f2e40113c40 2024-04-25T18:55:29.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.903+0000 7f2e37e00700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2e240a3c40 0x7f2e240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:29.905 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.904+0000 7f2e37e00700 1 -- 172.21.15.45:0/1840614457 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f2e380cff90 con 0x7f2e40113c40 2024-04-25T18:55:29.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.904+0000 7f2e3ea00700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2e240a3c40 0x7f2e240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:29.906 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.905+0000 7f2e3ea00700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2e240a3c40 0x7f2e240a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e300062a0 tx=0x7f2e30006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:29.910 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:29.908+0000 7f2e37e00700 1 -- 172.21.15.45:0/1840614457 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e3809ca20 con 0x7f2e40113c40 2024-04-25T18:55:30.139 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.137+0000 7f2e45600700 1 -- 172.21.15.45:0/1840614457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e401a0bf0 con 0x7f2e40113c40 2024-04-25T18:55:30.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.139+0000 7f2e37e00700 1 -- 172.21.15.45:0/1840614457 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22124 (secure 0 0 0) 0x7f2e401a0bf0 con 0x7f2e40113c40 2024-04-25T18:55:30.142 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:55:30.142 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T18:55:26.025588+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:55:17.591864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:55:22.070218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:55:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 -- 172.21.15.45:0/1840614457 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2e240a3c40 msgr2=0x7f2e240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2e240a3c40 0x7f2e240a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e300062a0 tx=0x7f2e30006210 comp rx=0 tx=0).stop 2024-04-25T18:55:30.144 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 -- 172.21.15.45:0/1840614457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40113c40 msgr2=0x7f2e4019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40113c40 0x7f2e4019f690 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800e940 tx=0x7f2e3800ec50 comp rx=0 tx=0).stop 2024-04-25T18:55:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 -- 172.21.15.45:0/1840614457 shutdown_connections 2024-04-25T18:55:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2e240a3c40 0x7f2e240a6100 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2e40074c70 0x7f2e401a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e40113c40 0x7f2e4019f690 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 --2- 172.21.15.45:0/1840614457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2e40075590 0x7f2e401a5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 -- 172.21.15.45:0/1840614457 >> 172.21.15.45:0/1840614457 conn(0x7f2e40079bf0 msgr2=0x7f2e4010f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 -- 172.21.15.45:0/1840614457 shutdown_connections 2024-04-25T18:55:30.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:30.142+0000 7f2e45600700 1 -- 172.21.15.45:0/1840614457 wait complete. 2024-04-25T18:55:30.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:30 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1840614457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:55:30.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:30 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1840614457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:55:30.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:30 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1840614457' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:55:31.070 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:55:31.113 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:30 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b[63904]: 2024-04-25T18:55:30.703+0000 7f159315b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T18:55:31.566 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:31.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:31 smithi062 ceph-mon[32844]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-04-25T18:55:31.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:31 smithi079 ceph-mon[33002]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-04-25T18:55:31.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:31 smithi045 ceph-mon[27776]: pgmap v329: 97 pgs: 97 active+clean; 579 KiB data, 378 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-04-25T18:55:33.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:33 smithi045 ceph-mon[27776]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T18:55:33.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:33 smithi062 ceph-mon[32844]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T18:55:33.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:33 smithi079 ceph-mon[33002]: pgmap v330: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-25T18:55:33.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.719+0000 7fbd44600700 1 -- 172.21.15.45:0/2166391544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c074c70 msgr2=0x7fbd3c075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.719+0000 7fbd44600700 1 --2- 172.21.15.45:0/2166391544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c074c70 0x7fbd3c075050 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbd30009990 tx=0x7fbd30009ca0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.719+0000 7fbd44600700 1 -- 172.21.15.45:0/2166391544 shutdown_connections 2024-04-25T18:55:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.719+0000 7fbd44600700 1 --2- 172.21.15.45:0/2166391544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c074c70 0x7fbd3c075050 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.719+0000 7fbd44600700 1 --2- 172.21.15.45:0/2166391544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c10f470 0x7fbd3c111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.719+0000 7fbd44600700 1 --2- 172.21.15.45:0/2166391544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c075590 0x7fbd3c10ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.719+0000 7fbd44600700 1 -- 172.21.15.45:0/2166391544 >> 172.21.15.45:0/2166391544 conn(0x7fbd3c079bf0 msgr2=0x7fbd3c07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.720+0000 7fbd44600700 1 -- 172.21.15.45:0/2166391544 shutdown_connections 2024-04-25T18:55:33.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.720+0000 7fbd44600700 1 -- 172.21.15.45:0/2166391544 wait complete. 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.721+0000 7fbd44600700 1 Processor -- start 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.721+0000 7fbd44600700 1 -- start start 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.721+0000 7fbd44600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c075590 0x7fbd3c19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.721+0000 7fbd44600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c10f470 0x7fbd3c19bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.721+0000 7fbd44600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c19ff00 0x7fbd3c1a0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.721+0000 7fbd44600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c114ed0 con 0x7fbd3c10f470 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.721+0000 7fbd44600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c114d50 con 0x7fbd3c075590 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.721+0000 7fbd44600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3c115050 con 0x7fbd3c19ff00 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd41c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c075590 0x7fbd3c19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd41200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c10f470 0x7fbd3c19bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:33.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd41c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c075590 0x7fbd3c19b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:60502/0 (socket says 172.21.15.45:60502) 2024-04-25T18:55:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd41c00700 1 -- 172.21.15.45:0/200800163 learned_addr learned my addr 172.21.15.45:0/200800163 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd42600700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c19ff00 0x7fbd3c1a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:33.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd41c00700 1 -- 172.21.15.45:0/200800163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c19ff00 msgr2=0x7fbd3c1a0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd41c00700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c19ff00 0x7fbd3c1a0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd41c00700 1 -- 172.21.15.45:0/200800163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c10f470 msgr2=0x7fbd3c19bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd41c00700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c10f470 0x7fbd3c19bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd41c00700 1 -- 172.21.15.45:0/200800163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3c1a0a20 con 0x7fbd3c075590 2024-04-25T18:55:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.722+0000 7fbd41c00700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c075590 0x7fbd3c19b630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd300098e0 tx=0x7fbd300057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.723+0000 7fbd3a000700 1 -- 172.21.15.45:0/200800163 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30026070 con 0x7fbd3c075590 2024-04-25T18:55:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.723+0000 7fbd3a000700 1 -- 172.21.15.45:0/200800163 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd30021410 con 0x7fbd3c075590 2024-04-25T18:55:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.723+0000 7fbd3a000700 1 -- 172.21.15.45:0/200800163 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30017650 con 0x7fbd3c075590 2024-04-25T18:55:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.723+0000 7fbd44600700 1 -- 172.21.15.45:0/200800163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3c1a0cb0 con 0x7fbd3c075590 2024-04-25T18:55:33.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.723+0000 7fbd44600700 1 -- 172.21.15.45:0/200800163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3c1a8430 con 0x7fbd3c075590 2024-04-25T18:55:33.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.724+0000 7fbd44600700 1 -- 172.21.15.45:0/200800163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd3c069730 con 0x7fbd3c075590 2024-04-25T18:55:33.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.726+0000 7fbd3a000700 1 -- 172.21.15.45:0/200800163 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbd30029070 con 0x7fbd3c075590 2024-04-25T18:55:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.727+0000 7fbd3a000700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbd240a3cc0 0x7fbd240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.728+0000 7fbd3a000700 1 -- 172.21.15.45:0/200800163 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7fbd300d0b60 con 0x7fbd3c075590 2024-04-25T18:55:33.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.728+0000 7fbd41200700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbd240a3cc0 0x7fbd240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:33.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.728+0000 7fbd41200700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbd240a3cc0 0x7fbd240a6180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c111530 tx=0x7fbd28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:33.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.732+0000 7fbd3a000700 1 -- 172.21.15.45:0/200800163 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd3005dff0 con 0x7fbd3c075590 2024-04-25T18:55:33.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.962+0000 7fbd44600700 1 -- 172.21.15.45:0/200800163 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd3c02df80 con 0x7fbd3c075590 2024-04-25T18:55:33.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.964+0000 7fbd3a000700 1 -- 172.21.15.45:0/200800163 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22124 (secure 0 0 0) 0x7fbd30013070 con 0x7fbd3c075590 2024-04-25T18:55:33.966 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:55:33.967 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":92,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T18:55:26.025588+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:55:17.591864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:55:22.070218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.968+0000 7fbd44600700 1 -- 172.21.15.45:0/200800163 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbd240a3cc0 msgr2=0x7fbd240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.968+0000 7fbd44600700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbd240a3cc0 0x7fbd240a6180 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c111530 tx=0x7fbd28009260 comp rx=0 tx=0).stop 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.968+0000 7fbd44600700 1 -- 172.21.15.45:0/200800163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c075590 msgr2=0x7fbd3c19b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.968+0000 7fbd44600700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c075590 0x7fbd3c19b630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd300098e0 tx=0x7fbd300057c0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.968+0000 7fbd44600700 1 -- 172.21.15.45:0/200800163 shutdown_connections 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.969+0000 7fbd44600700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbd240a3cc0 0x7fbd240a6180 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.969+0000 7fbd44600700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbd3c075590 0x7fbd3c19b630 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.969+0000 7fbd44600700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbd3c19ff00 0x7fbd3c1a0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.969+0000 7fbd44600700 1 --2- 172.21.15.45:0/200800163 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbd3c10f470 0x7fbd3c19bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.969+0000 7fbd44600700 1 -- 172.21.15.45:0/200800163 >> 172.21.15.45:0/200800163 conn(0x7fbd3c079bf0 msgr2=0x7fbd3c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.969+0000 7fbd44600700 1 -- 172.21.15.45:0/200800163 shutdown_connections 2024-04-25T18:55:33.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:33.969+0000 7fbd44600700 1 -- 172.21.15.45:0/200800163 wait complete. 2024-04-25T18:55:34.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:34 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/200800163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:55:34.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:34 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/200800163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:55:34.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:34 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/200800163' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:55:35.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-25T18:55:35.490 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:35.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:35 smithi062 ceph-mon[32844]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:55:35.637 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:35 smithi045 ceph-mon[27776]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:55:35.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:35 smithi079 ceph-mon[33002]: pgmap v331: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:55:37.594 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:37 smithi045 ceph-mon[27776]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:55:37.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:37 smithi062 ceph-mon[32844]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:55:37.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:37 smithi079 ceph-mon[33002]: pgmap v332: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T18:55:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.858+0000 7f6ff1000700 1 -- 172.21.15.45:0/122432089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fec1092d0 msgr2=0x7f6fec1096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.858+0000 7f6ff1000700 1 --2- 172.21.15.45:0/122432089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fec1092d0 0x7f6fec1096b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0098f0 tx=0x7f6fdc009c00 comp rx=0 tx=0).stop 2024-04-25T18:55:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.858+0000 7f6ff1000700 1 -- 172.21.15.45:0/122432089 shutdown_connections 2024-04-25T18:55:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.858+0000 7f6ff1000700 1 --2- 172.21.15.45:0/122432089 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fec1092d0 0x7f6fec1096b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.858+0000 7f6ff1000700 1 --2- 172.21.15.45:0/122432089 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fec10e3b0 0x7f6fec115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.858+0000 7f6ff1000700 1 --2- 172.21.15.45:0/122432089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec109c80 0x7f6fec10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.858+0000 7f6ff1000700 1 -- 172.21.15.45:0/122432089 >> 172.21.15.45:0/122432089 conn(0x7f6fec076210 msgr2=0x7f6fec107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.858+0000 7f6ff1000700 1 -- 172.21.15.45:0/122432089 shutdown_connections 2024-04-25T18:55:37.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.858+0000 7f6ff1000700 1 -- 172.21.15.45:0/122432089 wait complete. 2024-04-25T18:55:37.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.859+0000 7f6ff1000700 1 Processor -- start 2024-04-25T18:55:37.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.860+0000 7f6ff1000700 1 -- start start 2024-04-25T18:55:37.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.860+0000 7f6ff1000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fec1092d0 0x7f6fec1a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:37.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.860+0000 7f6ff1000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fec109c80 0x7f6fec1a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:37.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.860+0000 7f6ff1000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec10e3b0 0x7f6fec19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:37.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.860+0000 7f6ff1000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec119290 con 0x7f6fec10e3b0 2024-04-25T18:55:37.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.860+0000 7f6ff1000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec119110 con 0x7f6fec1092d0 2024-04-25T18:55:37.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.860+0000 7f6ff1000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6fec119410 con 0x7f6fec109c80 2024-04-25T18:55:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.860+0000 7f6fea800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec10e3b0 0x7f6fec19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.860+0000 7f6fe9400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fec109c80 0x7f6fec1a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fea800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec10e3b0 0x7f6fec19f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52726/0 (socket says 172.21.15.45:52726) 2024-04-25T18:55:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fe9400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fec109c80 0x7f6fec1a5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33262/0 (socket says 172.21.15.45:33262) 2024-04-25T18:55:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fea800700 1 -- 172.21.15.45:0/1121467623 learned_addr learned my addr 172.21.15.45:0/1121467623 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fe9e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fec1092d0 0x7f6fec1a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:37.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fe9400700 1 -- 172.21.15.45:0/1121467623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fec1092d0 msgr2=0x7f6fec1a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fe9400700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fec1092d0 0x7f6fec1a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fe9400700 1 -- 172.21.15.45:0/1121467623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec10e3b0 msgr2=0x7f6fec19f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fe9400700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec10e3b0 0x7f6fec19f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fe9400700 1 -- 172.21.15.45:0/1121467623 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fec19fd30 con 0x7f6fec109c80 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fea800700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec10e3b0 0x7f6fec19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.861+0000 7f6fe9400700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fec109c80 0x7f6fec1a5950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8005fd0 tx=0x7f6fd800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.862+0000 7f6fe2a00700 1 -- 172.21.15.45:0/1121467623 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8010900 con 0x7f6fec109c80 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.862+0000 7f6ff1000700 1 -- 172.21.15.45:0/1121467623 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fec1a0020 con 0x7f6fec109c80 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.862+0000 7f6fe2a00700 1 -- 172.21.15.45:0/1121467623 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd8018420 con 0x7f6fec109c80 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.862+0000 7f6fe2a00700 1 -- 172.21.15.45:0/1121467623 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd800f680 con 0x7f6fec109c80 2024-04-25T18:55:37.865 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.862+0000 7f6ff1000700 1 -- 172.21.15.45:0/1121467623 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fec1ac2e0 con 0x7f6fec109c80 2024-04-25T18:55:37.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.863+0000 7f6ff1000700 1 -- 172.21.15.45:0/1121467623 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fec069730 con 0x7f6fec109c80 2024-04-25T18:55:37.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.865+0000 7f6fe2a00700 1 -- 172.21.15.45:0/1121467623 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6fd8010aa0 con 0x7f6fec109c80 2024-04-25T18:55:37.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.866+0000 7f6fe2a00700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6fd00a3c40 0x7f6fd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:37.868 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.867+0000 7f6fe2a00700 1 -- 172.21.15.45:0/1121467623 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f6fd80d0550 con 0x7f6fec109c80 2024-04-25T18:55:37.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.867+0000 7f6fe9e00700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6fd00a3c40 0x7f6fd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:37.869 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.868+0000 7f6fe9e00700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6fd00a3c40 0x7f6fd00a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0098c0 tx=0x7f6fdc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:37.871 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:37.869+0000 7f6fe2a00700 1 -- 172.21.15.45:0/1121467623 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fd809cfe0 con 0x7f6fec109c80 2024-04-25T18:55:38.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.150+0000 7f6ff1000700 1 -- 172.21.15.45:0/1121467623 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6fec1a0b00 con 0x7f6fec109c80 2024-04-25T18:55:38.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.335+0000 7f6fe2a00700 1 -- 172.21.15.45:0/1121467623 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f6fd809c8e0 con 0x7f6fec109c80 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.338+0000 7f6ff1000700 1 -- 172.21.15.45:0/1121467623 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6fd00a3c40 msgr2=0x7f6fd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.338+0000 7f6ff1000700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6fd00a3c40 0x7f6fd00a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0098c0 tx=0x7f6fdc019040 comp rx=0 tx=0).stop 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.339+0000 7f6ff1000700 1 -- 172.21.15.45:0/1121467623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fec109c80 msgr2=0x7f6fec1a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.339+0000 7f6ff1000700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fec109c80 0x7f6fec1a5950 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8005fd0 tx=0x7f6fd800eca0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.339+0000 7f6ff1000700 1 -- 172.21.15.45:0/1121467623 shutdown_connections 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.339+0000 7f6ff1000700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6fd00a3c40 0x7f6fd00a6100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.339+0000 7f6ff1000700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fec1092d0 0x7f6fec1a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.339+0000 7f6ff1000700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fec10e3b0 0x7f6fec19f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.339+0000 7f6ff1000700 1 --2- 172.21.15.45:0/1121467623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fec109c80 0x7f6fec1a5950 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.339+0000 7f6ff1000700 1 -- 172.21.15.45:0/1121467623 >> 172.21.15.45:0/1121467623 conn(0x7f6fec076210 msgr2=0x7f6fec10d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.339+0000 7f6ff1000700 1 -- 172.21.15.45:0/1121467623 shutdown_connections 2024-04-25T18:55:38.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:38.339+0000 7f6ff1000700 1 -- 172.21.15.45:0/1121467623 wait complete. 2024-04-25T18:55:38.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:38 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1121467623' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:55:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:38 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:55:38.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:38 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1121467623' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:55:38.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:38 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:55:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:38 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1121467623' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:55:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:38 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T18:55:39.364 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-25T18:55:39.364 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-04-25T18:55:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:39 smithi062 ceph-mon[32844]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T18:55:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:39 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T18:55:39.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:39 smithi062 ceph-mon[32844]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:39.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:39 smithi079 ceph-mon[33002]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T18:55:39.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:39 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T18:55:39.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:39 smithi079 ceph-mon[33002]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:39.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:39 smithi045 ceph-mon[27776]: pgmap v333: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T18:55:39.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:39 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T18:55:39.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:39 smithi045 ceph-mon[27776]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:39.836 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:41.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:41 smithi062 ceph-mon[32844]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-04-25T18:55:41.627 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:41 smithi045 ceph-mon[27776]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-04-25T18:55:41.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:41 smithi079 ceph-mon[33002]: pgmap v334: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-04-25T18:55:41.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.929+0000 7f35f2000700 1 -- 172.21.15.45:0/839637391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35ec06a750 msgr2=0x7f35ec06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:41.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.929+0000 7f35f2000700 1 --2- 172.21.15.45:0/839637391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35ec06a750 0x7f35ec06abb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f35d4009990 tx=0x7f35d4009ca0 comp rx=0 tx=0).stop 2024-04-25T18:55:41.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.929+0000 7f35f2000700 1 -- 172.21.15.45:0/839637391 shutdown_connections 2024-04-25T18:55:41.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.929+0000 7f35f2000700 1 --2- 172.21.15.45:0/839637391 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec06b0f0 0x7f35ec10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:41.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.929+0000 7f35f2000700 1 --2- 172.21.15.45:0/839637391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35ec108a30 0x7f35ec108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:41.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.929+0000 7f35f2000700 1 --2- 172.21.15.45:0/839637391 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35ec06a750 0x7f35ec06abb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:41.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.929+0000 7f35f2000700 1 -- 172.21.15.45:0/839637391 >> 172.21.15.45:0/839637391 conn(0x7f35ec0788f0 msgr2=0x7f35ec078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:41.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.929+0000 7f35f2000700 1 -- 172.21.15.45:0/839637391 shutdown_connections 2024-04-25T18:55:41.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.929+0000 7f35f2000700 1 -- 172.21.15.45:0/839637391 wait complete. 2024-04-25T18:55:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.930+0000 7f35f2000700 1 Processor -- start 2024-04-25T18:55:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.930+0000 7f35f2000700 1 -- start start 2024-04-25T18:55:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35f2000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec06a750 0x7f35ec19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35f2000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35ec06b0f0 0x7f35ec19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:41.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35f2000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35ec108a30 0x7f35ec19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:41.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35f2000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec112c20 con 0x7f35ec06b0f0 2024-04-25T18:55:41.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35f2000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec112aa0 con 0x7f35ec06a750 2024-04-25T18:55:41.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35f2000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f35ec112da0 con 0x7f35ec108a30 2024-04-25T18:55:41.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35eb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec06a750 0x7f35ec19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:41.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35e2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35ec06b0f0 0x7f35ec19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:41.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35eb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec06a750 0x7f35ec19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36342/0 (socket says 172.21.15.45:36342) 2024-04-25T18:55:41.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35eb400700 1 -- 172.21.15.45:0/41816359 learned_addr learned my addr 172.21.15.45:0/41816359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:41.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.931+0000 7f35ebe00700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35ec108a30 0x7f35ec19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:41.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.932+0000 7f35eb400700 1 -- 172.21.15.45:0/41816359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35ec108a30 msgr2=0x7f35ec19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:41.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.932+0000 7f35eb400700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35ec108a30 0x7f35ec19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:41.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.932+0000 7f35eb400700 1 -- 172.21.15.45:0/41816359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35ec06b0f0 msgr2=0x7f35ec19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:41.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.932+0000 7f35eb400700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35ec06b0f0 0x7f35ec19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:41.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.932+0000 7f35eb400700 1 -- 172.21.15.45:0/41816359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ec1a03c0 con 0x7f35ec06a750 2024-04-25T18:55:41.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.932+0000 7f35ebe00700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35ec108a30 0x7f35ec19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:41.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.932+0000 7f35e2a00700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35ec06b0f0 0x7f35ec19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:41.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.932+0000 7f35eb400700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec06a750 0x7f35ec19b3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e9c0 tx=0x7f35dc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:41.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.933+0000 7f35e8c00700 1 -- 172.21.15.45:0/41816359 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00cbd0 con 0x7f35ec06a750 2024-04-25T18:55:41.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.933+0000 7f35f2000700 1 -- 172.21.15.45:0/41816359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ec1a06b0 con 0x7f35ec06a750 2024-04-25T18:55:41.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.933+0000 7f35e8c00700 1 -- 172.21.15.45:0/41816359 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35dc00ee60 con 0x7f35ec06a750 2024-04-25T18:55:41.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.933+0000 7f35e8c00700 1 -- 172.21.15.45:0/41816359 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0055c0 con 0x7f35ec06a750 2024-04-25T18:55:41.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.933+0000 7f35f2000700 1 -- 172.21.15.45:0/41816359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ec1a7f30 con 0x7f35ec06a750 2024-04-25T18:55:41.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.934+0000 7f35f2000700 1 -- 172.21.15.45:0/41816359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ec069730 con 0x7f35ec06a750 2024-04-25T18:55:41.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.936+0000 7f35e8c00700 1 -- 172.21.15.45:0/41816359 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f35dc004750 con 0x7f35ec06a750 2024-04-25T18:55:41.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.937+0000 7f35e8c00700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f35d00a3c40 0x7f35d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:41.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.937+0000 7f35e8c00700 1 -- 172.21.15.45:0/41816359 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f35dc0d0520 con 0x7f35ec06a750 2024-04-25T18:55:41.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.938+0000 7f35e2a00700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f35d00a3c40 0x7f35d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:41.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.938+0000 7f35e2a00700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f35d00a3c40 0x7f35d00a6100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35ec078f40 tx=0x7f35d400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:41.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:41.942+0000 7f35e8c00700 1 -- 172.21.15.45:0/41816359 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35dc09cfb0 con 0x7f35ec06a750 2024-04-25T18:55:42.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.222+0000 7f35f2000700 1 -- 172.21.15.45:0/41816359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f35ec02df80 con 0x7f35ec06a750 2024-04-25T18:55:42.357 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.354+0000 7f35e8c00700 1 -- 172.21.15.45:0/41816359 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f35dc09c8b0 con 0x7f35ec06a750 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.358+0000 7f35f2000700 1 -- 172.21.15.45:0/41816359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f35d00a3c40 msgr2=0x7f35d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.358+0000 7f35f2000700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f35d00a3c40 0x7f35d00a6100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f35ec078f40 tx=0x7f35d400b260 comp rx=0 tx=0).stop 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.358+0000 7f35f2000700 1 -- 172.21.15.45:0/41816359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec06a750 msgr2=0x7f35ec19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.358+0000 7f35f2000700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec06a750 0x7f35ec19b3b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35dc00e9c0 tx=0x7f35dc00ecd0 comp rx=0 tx=0).stop 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.358+0000 7f35f2000700 1 -- 172.21.15.45:0/41816359 shutdown_connections 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.358+0000 7f35f2000700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f35d00a3c40 0x7f35d00a6100 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.358+0000 7f35f2000700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f35ec06a750 0x7f35ec19b3b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.358+0000 7f35f2000700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35ec108a30 0x7f35ec19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.358+0000 7f35f2000700 1 --2- 172.21.15.45:0/41816359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f35ec06b0f0 0x7f35ec19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.358+0000 7f35f2000700 1 -- 172.21.15.45:0/41816359 >> 172.21.15.45:0/41816359 conn(0x7f35ec0788f0 msgr2=0x7f35ec10df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.359+0000 7f35f2000700 1 -- 172.21.15.45:0/41816359 shutdown_connections 2024-04-25T18:55:42.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:42.359+0000 7f35f2000700 1 -- 172.21.15.45:0/41816359 wait complete. 2024-04-25T18:55:42.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:42 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/41816359' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:55:42.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:42 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:55:42.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:42 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/41816359' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:55:42.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:42 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:55:42.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:42 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/41816359' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:55:42.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:42 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T18:55:43.211 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-25T18:55:43.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:43 smithi045 ceph-mon[27776]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T18:55:43.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:43 smithi045 ceph-mon[27776]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T18:55:43.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:43 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T18:55:43.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:43 smithi045 ceph-mon[27776]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T18:55:43.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:43 smithi045 ceph-mon[27776]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:43.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:43 smithi045 ceph-mon[27776]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-25T18:55:43.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:43 smithi045 ceph-mon[27776]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-25T18:55:43.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:43 smithi062 ceph-mon[32844]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T18:55:43.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:43 smithi062 ceph-mon[32844]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T18:55:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:43 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T18:55:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:43 smithi062 ceph-mon[32844]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T18:55:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:43 smithi062 ceph-mon[32844]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:43 smithi062 ceph-mon[32844]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-25T18:55:43.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:43 smithi062 ceph-mon[32844]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-25T18:55:43.680 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:43 smithi079 ceph-mon[33002]: pgmap v335: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-04-25T18:55:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:43 smithi079 ceph-mon[33002]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T18:55:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:43 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T18:55:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:43 smithi079 ceph-mon[33002]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T18:55:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:43 smithi079 ceph-mon[33002]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T18:55:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:43 smithi079 ceph-mon[33002]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-25T18:55:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:43 smithi079 ceph-mon[33002]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-25T18:55:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:44 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] up:active 2024-04-25T18:55:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:44 smithi045 ceph-mon[27776]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T18:55:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:44 smithi045 ceph-mon[27776]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T18:55:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:44 smithi045 ceph-mon[27776]: Cluster is now healthy 2024-04-25T18:55:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:44 smithi045 ceph-mon[27776]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-25T18:55:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:44 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-25T18:55:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:44 smithi045 ceph-mon[27776]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T18:55:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:44 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] up:active 2024-04-25T18:55:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:44 smithi079 ceph-mon[33002]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T18:55:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:44 smithi079 ceph-mon[33002]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T18:55:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:44 smithi079 ceph-mon[33002]: Cluster is now healthy 2024-04-25T18:55:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:44 smithi079 ceph-mon[33002]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-25T18:55:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:44 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-25T18:55:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:44 smithi079 ceph-mon[33002]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T18:55:44.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:44 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] up:active 2024-04-25T18:55:44.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:44 smithi062 ceph-mon[32844]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T18:55:44.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:44 smithi062 ceph-mon[32844]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T18:55:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:44 smithi062 ceph-mon[32844]: Cluster is now healthy 2024-04-25T18:55:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:44 smithi062 ceph-mon[32844]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 10 up:standby 2024-04-25T18:55:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:44 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:creating} 9 up:standby 2024-04-25T18:55:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:44 smithi062 ceph-mon[32844]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-04-25T18:55:45.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:45 smithi079 ceph-mon[33002]: pgmap v336: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 2 op/s 2024-04-25T18:55:45.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:45 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] up:active 2024-04-25T18:55:45.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:45 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:55:45.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:45 smithi045 ceph-mon[27776]: pgmap v336: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 2 op/s 2024-04-25T18:55:45.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:45 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] up:active 2024-04-25T18:55:45.661 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:45 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:55:45.705 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:45 smithi062 ceph-mon[32844]: pgmap v336: 97 pgs: 97 active+clean; 579 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s wr, 2 op/s 2024-04-25T18:55:45.706 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:45 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] up:active 2024-04-25T18:55:45.706 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:45 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:55:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.840+0000 7f013c800700 1 -- 172.21.15.45:0/887216087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0134113920 msgr2=0x7f0134115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.840+0000 7f013c800700 1 --2- 172.21.15.45:0/887216087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0134113920 0x7f0134115d10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0130009f90 tx=0x7f013000b330 comp rx=0 tx=0).stop 2024-04-25T18:55:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.840+0000 7f013c800700 1 -- 172.21.15.45:0/887216087 shutdown_connections 2024-04-25T18:55:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.840+0000 7f013c800700 1 --2- 172.21.15.45:0/887216087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0134113920 0x7f0134115d10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.840+0000 7f013c800700 1 --2- 172.21.15.45:0/887216087 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0134076f70 0x7f0134077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.840+0000 7f013c800700 1 --2- 172.21.15.45:0/887216087 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134077890 0x7f01341133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.840+0000 7f013c800700 1 -- 172.21.15.45:0/887216087 >> 172.21.15.45:0/887216087 conn(0x7f0134100770 msgr2=0x7f0134102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.840+0000 7f013c800700 1 -- 172.21.15.45:0/887216087 shutdown_connections 2024-04-25T18:55:45.843 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.840+0000 7f013c800700 1 -- 172.21.15.45:0/887216087 wait complete. 2024-04-25T18:55:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.842+0000 7f013c800700 1 Processor -- start 2024-04-25T18:55:45.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.842+0000 7f013c800700 1 -- start start 2024-04-25T18:55:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.842+0000 7f013c800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0134076f70 0x7f013419f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f013c800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0134077890 0x7f013419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f013c800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134113920 0x7f01341a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f013c800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0134119280 con 0x7f0134077890 2024-04-25T18:55:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f013c800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0134119100 con 0x7f0134113920 2024-04-25T18:55:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f013c800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0134119400 con 0x7f0134076f70 2024-04-25T18:55:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f0139400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0134077890 0x7f013419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f0139e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0134076f70 0x7f013419f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f0139400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0134077890 0x7f013419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39302/0 (socket says 172.21.15.45:39302) 2024-04-25T18:55:45.845 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f0139400700 1 -- 172.21.15.45:0/2067152927 learned_addr learned my addr 172.21.15.45:0/2067152927 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:45.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f013a800700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134113920 0x7f01341a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f0139400700 1 -- 172.21.15.45:0/2067152927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0134076f70 msgr2=0x7f013419f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f0139400700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0134076f70 0x7f013419f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f0139400700 1 -- 172.21.15.45:0/2067152927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134113920 msgr2=0x7f01341a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.843+0000 7f0139400700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134113920 0x7f01341a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.844+0000 7f0139400700 1 -- 172.21.15.45:0/2067152927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01341a4720 con 0x7f0134077890 2024-04-25T18:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.844+0000 7f013a800700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134113920 0x7f01341a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.844+0000 7f0139e00700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0134076f70 0x7f013419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.844+0000 7f0139400700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0134077890 0x7f013419fc50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f012400e9c0 tx=0x7f012400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.844+0000 7f012ea00700 1 -- 172.21.15.45:0/2067152927 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012400cbd0 con 0x7f0134077890 2024-04-25T18:55:45.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.844+0000 7f012ea00700 1 -- 172.21.15.45:0/2067152927 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f012400ee50 con 0x7f0134077890 2024-04-25T18:55:45.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.844+0000 7f013c800700 1 -- 172.21.15.45:0/2067152927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01341a4a10 con 0x7f0134077890 2024-04-25T18:55:45.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.844+0000 7f012ea00700 1 -- 172.21.15.45:0/2067152927 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124005550 con 0x7f0134077890 2024-04-25T18:55:45.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.844+0000 7f013c800700 1 -- 172.21.15.45:0/2067152927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01341ac290 con 0x7f0134077890 2024-04-25T18:55:45.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.845+0000 7f013c800700 1 -- 172.21.15.45:0/2067152927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0118005330 con 0x7f0134077890 2024-04-25T18:55:45.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.848+0000 7f012ea00700 1 -- 172.21.15.45:0/2067152927 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0124004750 con 0x7f0134077890 2024-04-25T18:55:45.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.850+0000 7f012ea00700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f011c0a3c40 0x7f011c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:45.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.850+0000 7f012ea00700 1 -- 172.21.15.45:0/2067152927 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f01240d0650 con 0x7f0134077890 2024-04-25T18:55:45.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.850+0000 7f0139e00700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f011c0a3c40 0x7f011c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:45.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.851+0000 7f0139e00700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f011c0a3c40 0x7f011c0a6100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0128005fd0 tx=0x7f0128009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:45.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:45.854+0000 7f012ea00700 1 -- 172.21.15.45:0/2067152927 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0124053ff0 con 0x7f0134077890 2024-04-25T18:55:46.113 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b[63904]: 2024-04-25T18:55:45.703+0000 7f159315b700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T18:55:46.114 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 18:55:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b[63904]: 2024-04-25T18:55:45.703+0000 7f159315b700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T18:55:46.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.147+0000 7f013c800700 1 -- 172.21.15.45:0/2067152927 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0118005620 con 0x7f0134077890 2024-04-25T18:55:46.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.399+0000 7f012ea00700 1 -- 172.21.15.45:0/2067152927 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f012409d0e0 con 0x7f0134077890 2024-04-25T18:55:46.402 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:46 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2067152927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:55:46.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.405+0000 7f013c800700 1 -- 172.21.15.45:0/2067152927 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f011c0a3c40 msgr2=0x7f011c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:46.408 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.405+0000 7f013c800700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f011c0a3c40 0x7f011c0a6100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0128005fd0 tx=0x7f0128009310 comp rx=0 tx=0).stop 2024-04-25T18:55:46.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.405+0000 7f013c800700 1 -- 172.21.15.45:0/2067152927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0134077890 msgr2=0x7f013419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:46.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.405+0000 7f013c800700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0134077890 0x7f013419fc50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f012400e9c0 tx=0x7f012400ecd0 comp rx=0 tx=0).stop 2024-04-25T18:55:46.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.406+0000 7f013c800700 1 -- 172.21.15.45:0/2067152927 shutdown_connections 2024-04-25T18:55:46.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.406+0000 7f013c800700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f011c0a3c40 0x7f011c0a6100 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:46.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.406+0000 7f013c800700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0134113920 0x7f01341a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:46.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.406+0000 7f013c800700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0134076f70 0x7f013419f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:46.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.406+0000 7f013c800700 1 --2- 172.21.15.45:0/2067152927 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0134077890 0x7f013419fc50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:46.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.406+0000 7f013c800700 1 -- 172.21.15.45:0/2067152927 >> 172.21.15.45:0/2067152927 conn(0x7f0134100770 msgr2=0x7f0134102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:46.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.406+0000 7f013c800700 1 -- 172.21.15.45:0/2067152927 shutdown_connections 2024-04-25T18:55:46.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:46.406+0000 7f013c800700 1 -- 172.21.15.45:0/2067152927 wait complete. 2024-04-25T18:55:46.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:46 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2067152927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:55:46.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:46 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2067152927' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T18:55:47.155 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-25T18:55:47.615 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:47 smithi079 ceph-mon[33002]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-04-25T18:55:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:47 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2067152927' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T18:55:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:47 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:55:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:47 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:55:47.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:47 smithi045 ceph-mon[27776]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-04-25T18:55:47.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:47 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2067152927' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T18:55:47.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:47 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:55:47.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:47 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:55:47.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:47 smithi062 ceph-mon[32844]: pgmap v337: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-04-25T18:55:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:47 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2067152927' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T18:55:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:47 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 9 up:standby 2024-04-25T18:55:47.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:47 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:55:49.586 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:49 smithi045 ceph-mon[27776]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-25T18:55:49.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:49 smithi062 ceph-mon[32844]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-25T18:55:49.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:49 smithi079 ceph-mon[33002]: pgmap v338: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-04-25T18:55:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.767+0000 7fbb0f600700 1 -- 172.21.15.45:0/1538690586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08104eb0 msgr2=0x7fbb08105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.767+0000 7fbb0f600700 1 --2- 172.21.15.45:0/1538690586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08104eb0 0x7fbb08105290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009920 tx=0x7fbafc009c30 comp rx=0 tx=0).stop 2024-04-25T18:55:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.767+0000 7fbb0f600700 1 -- 172.21.15.45:0/1538690586 shutdown_connections 2024-04-25T18:55:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.767+0000 7fbb0f600700 1 --2- 172.21.15.45:0/1538690586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb08109f90 0x7fbb08111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.767+0000 7fbb0f600700 1 --2- 172.21.15.45:0/1538690586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08104eb0 0x7fbb08105290 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.767+0000 7fbb0f600700 1 --2- 172.21.15.45:0/1538690586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb08105860 0x7fbb081098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.767+0000 7fbb0f600700 1 -- 172.21.15.45:0/1538690586 >> 172.21.15.45:0/1538690586 conn(0x7fbb08100740 msgr2=0x7fbb08102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:49.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.768+0000 7fbb0f600700 1 -- 172.21.15.45:0/1538690586 shutdown_connections 2024-04-25T18:55:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.768+0000 7fbb0f600700 1 -- 172.21.15.45:0/1538690586 wait complete. 2024-04-25T18:55:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.769+0000 7fbb0f600700 1 Processor -- start 2024-04-25T18:55:49.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.769+0000 7fbb0f600700 1 -- start start 2024-04-25T18:55:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb0f600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb08104eb0 0x7fbb0819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb0f600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08105860 0x7fbb0819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb0f600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb08109f90 0x7fbb0819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb0f600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbb08114ea0 con 0x7fbb08104eb0 2024-04-25T18:55:49.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb0f600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbb08114d20 con 0x7fbb08109f90 2024-04-25T18:55:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb0f600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbb08115020 con 0x7fbb08105860 2024-04-25T18:55:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb0cc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb08104eb0 0x7fbb0819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb0cc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb08104eb0 0x7fbb0819b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39320/0 (socket says 172.21.15.45:39320) 2024-04-25T18:55:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb0cc00700 1 -- 172.21.15.45:0/2019550856 learned_addr learned my addr 172.21.15.45:0/2019550856 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:49.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb0d600700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb08109f90 0x7fbb0819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.770+0000 7fbb07e00700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08105860 0x7fbb0819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.771+0000 7fbb07e00700 1 -- 172.21.15.45:0/2019550856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb08109f90 msgr2=0x7fbb0819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.771+0000 7fbb07e00700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb08109f90 0x7fbb0819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.771+0000 7fbb07e00700 1 -- 172.21.15.45:0/2019550856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb08104eb0 msgr2=0x7fbb0819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.771+0000 7fbb07e00700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb08104eb0 0x7fbb0819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.771+0000 7fbb07e00700 1 -- 172.21.15.45:0/2019550856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb081a0410 con 0x7fbb08105860 2024-04-25T18:55:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.771+0000 7fbb07e00700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08105860 0x7fbb0819b990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005fd0 tx=0x7fbaf400eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:49.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.772+0000 7fbb05600700 1 -- 172.21.15.45:0/2019550856 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4010900 con 0x7fbb08105860 2024-04-25T18:55:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.772+0000 7fbb05600700 1 -- 172.21.15.45:0/2019550856 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbaf4018420 con 0x7fbb08105860 2024-04-25T18:55:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.772+0000 7fbb05600700 1 -- 172.21.15.45:0/2019550856 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf400f680 con 0x7fbb08105860 2024-04-25T18:55:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.772+0000 7fbb0f600700 1 -- 172.21.15.45:0/2019550856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb081a0700 con 0x7fbb08105860 2024-04-25T18:55:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.772+0000 7fbb0f600700 1 -- 172.21.15.45:0/2019550856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb081a7f80 con 0x7fbb08105860 2024-04-25T18:55:49.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.773+0000 7fbb0f600700 1 -- 172.21.15.45:0/2019550856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb08069730 con 0x7fbb08105860 2024-04-25T18:55:49.777 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.775+0000 7fbb05600700 1 -- 172.21.15.45:0/2019550856 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbaf40185d0 con 0x7fbb08105860 2024-04-25T18:55:49.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.777+0000 7fbb05600700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbaf00a3cc0 0x7fbaf00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.777+0000 7fbb05600700 1 -- 172.21.15.45:0/2019550856 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7fbaf40d0510 con 0x7fbb08105860 2024-04-25T18:55:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.777+0000 7fbb0cc00700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbaf00a3cc0 0x7fbaf00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:49.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.778+0000 7fbb0cc00700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbaf00a3cc0 0x7fbaf00a6180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0098f0 tx=0x7fbafc019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:49.781 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:49.779+0000 7fbb05600700 1 -- 172.21.15.45:0/2019550856 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbaf409cfa0 con 0x7fbb08105860 2024-04-25T18:55:50.062 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.059+0000 7fbb0f600700 1 -- 172.21.15.45:0/2019550856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fbb0819c760 con 0x7fbb08105860 2024-04-25T18:55:50.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.423+0000 7fbb05600700 1 -- 172.21.15.45:0/2019550856 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7fbaf409c8a0 con 0x7fbb08105860 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.427+0000 7fbb0f600700 1 -- 172.21.15.45:0/2019550856 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbaf00a3cc0 msgr2=0x7fbaf00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.427+0000 7fbb0f600700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbaf00a3cc0 0x7fbaf00a6180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbafc0098f0 tx=0x7fbafc019040 comp rx=0 tx=0).stop 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.427+0000 7fbb0f600700 1 -- 172.21.15.45:0/2019550856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08105860 msgr2=0x7fbb0819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.427+0000 7fbb0f600700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08105860 0x7fbb0819b990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005fd0 tx=0x7fbaf400eca0 comp rx=0 tx=0).stop 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.427+0000 7fbb0f600700 1 -- 172.21.15.45:0/2019550856 shutdown_connections 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.428+0000 7fbb0f600700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbaf00a3cc0 0x7fbaf00a6180 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.428+0000 7fbb0f600700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbb08109f90 0x7fbb0819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.428+0000 7fbb0f600700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb08104eb0 0x7fbb0819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.428+0000 7fbb0f600700 1 --2- 172.21.15.45:0/2019550856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08105860 0x7fbb0819b990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.428+0000 7fbb0f600700 1 -- 172.21.15.45:0/2019550856 >> 172.21.15.45:0/2019550856 conn(0x7fbb08100740 msgr2=0x7fbb08101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:50.430 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.428+0000 7fbb0f600700 1 -- 172.21.15.45:0/2019550856 shutdown_connections 2024-04-25T18:55:50.431 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:50.428+0000 7fbb0f600700 1 -- 172.21.15.45:0/2019550856 wait complete. 2024-04-25T18:55:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:50 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2019550856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:55:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:50 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:55:50.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:50 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2019550856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:55:50.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:50 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:55:50.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:50 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2019550856' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:55:50.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:50 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T18:55:51.191 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T18:55:51.191 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-25T18:55:51.670 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:51 smithi079 ceph-mon[33002]: pgmap v339: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T18:55:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:51 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T18:55:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:51 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:55:51.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:51 smithi045 ceph-mon[27776]: pgmap v339: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T18:55:51.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:51 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T18:55:51.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:51 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:55:51.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:51 smithi062 ceph-mon[32844]: pgmap v339: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-04-25T18:55:51.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:51 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T18:55:51.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:51 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:55:53.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:53 smithi079 ceph-mon[33002]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T18:55:53.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:53 smithi045 ceph-mon[27776]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T18:55:53.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:53 smithi062 ceph-mon[32844]: pgmap v340: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T18:55:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.996+0000 7f5cad800700 1 -- 172.21.15.45:0/3827334061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8109fc0 msgr2=0x7f5ca8111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.996+0000 7f5cad800700 1 --2- 172.21.15.45:0/3827334061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8109fc0 0x7f5ca8111b90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000b0a0 tx=0x7f5ca000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:55:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.997+0000 7f5cad800700 1 -- 172.21.15.45:0/3827334061 shutdown_connections 2024-04-25T18:55:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.997+0000 7f5cad800700 1 --2- 172.21.15.45:0/3827334061 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8109fc0 0x7f5ca8111b90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.997+0000 7f5cad800700 1 --2- 172.21.15.45:0/3827334061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca8104ee0 0x7f5ca81052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:53.998 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.997+0000 7f5cad800700 1 --2- 172.21.15.45:0/3827334061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca8105890 0x7f5ca81098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.997+0000 7f5cad800700 1 -- 172.21.15.45:0/3827334061 >> 172.21.15.45:0/3827334061 conn(0x7f5ca8100770 msgr2=0x7f5ca8102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.997+0000 7f5cad800700 1 -- 172.21.15.45:0/3827334061 shutdown_connections 2024-04-25T18:55:53.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.997+0000 7f5cad800700 1 -- 172.21.15.45:0/3827334061 wait complete. 2024-04-25T18:55:54.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.999+0000 7f5cad800700 1 Processor -- start 2024-04-25T18:55:54.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:53.999+0000 7f5cad800700 1 -- start start 2024-04-25T18:55:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5cad800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca8104ee0 0x7f5ca819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5cad800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8105890 0x7f5ca819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5cad800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca8109fc0 0x7f5ca819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5cad800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8114ec0 con 0x7f5ca8109fc0 2024-04-25T18:55:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5cad800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8114d40 con 0x7f5ca8105890 2024-04-25T18:55:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5cad800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca8115040 con 0x7f5ca8104ee0 2024-04-25T18:55:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5ca7400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca8109fc0 0x7f5ca819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:54.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5ca6a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca8104ee0 0x7f5ca819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5ca7400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca8109fc0 0x7f5ca819fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39338/0 (socket says 172.21.15.45:39338) 2024-04-25T18:55:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5ca7400700 1 -- 172.21.15.45:0/2668605113 learned_addr learned my addr 172.21.15.45:0/2668605113 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:54.002 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.000+0000 7f5ca6000700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8105890 0x7f5ca819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5ca7400700 1 -- 172.21.15.45:0/2668605113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca8104ee0 msgr2=0x7f5ca819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5ca7400700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca8104ee0 0x7f5ca819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5ca7400700 1 -- 172.21.15.45:0/2668605113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8105890 msgr2=0x7f5ca819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5ca7400700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8105890 0x7f5ca819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5ca7400700 1 -- 172.21.15.45:0/2668605113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ca81a0340 con 0x7f5ca8109fc0 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5ca6a00700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca8104ee0 0x7f5ca819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5ca6000700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8105890 0x7f5ca819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5ca7400700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca8109fc0 0x7f5ca819fc00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000b070 tx=0x7f5ca0004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5c9f400700 1 -- 172.21.15.45:0/2668605113 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0021070 con 0x7f5ca8109fc0 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5c9f400700 1 -- 172.21.15.45:0/2668605113 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca0003c00 con 0x7f5ca8109fc0 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.001+0000 7f5cad800700 1 -- 172.21.15.45:0/2668605113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ca81a05d0 con 0x7f5ca8109fc0 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.002+0000 7f5cad800700 1 -- 172.21.15.45:0/2668605113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ca81a7eb0 con 0x7f5ca8109fc0 2024-04-25T18:55:54.003 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.002+0000 7f5c9f400700 1 -- 172.21.15.45:0/2668605113 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0004640 con 0x7f5ca8109fc0 2024-04-25T18:55:54.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.003+0000 7f5cad800700 1 -- 172.21.15.45:0/2668605113 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ca8069730 con 0x7f5ca8109fc0 2024-04-25T18:55:54.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.005+0000 7f5c9f400700 1 -- 172.21.15.45:0/2668605113 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5ca0007980 con 0x7f5ca8109fc0 2024-04-25T18:55:54.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.006+0000 7f5c9f400700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5c8c0a3cc0 0x7f5c8c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:54.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.007+0000 7f5c9f400700 1 -- 172.21.15.45:0/2668605113 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f5ca00d1430 con 0x7f5ca8109fc0 2024-04-25T18:55:54.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.007+0000 7f5ca6a00700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5c8c0a3cc0 0x7f5c8c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:54.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.007+0000 7f5ca6a00700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5c8c0a3cc0 0x7f5c8c0a6180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c98011490 tx=0x7f5c98009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:54.010 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.009+0000 7f5c9f400700 1 -- 172.21.15.45:0/2668605113 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ca009df50 con 0x7f5ca8109fc0 2024-04-25T18:55:54.278 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.277+0000 7f5cad800700 1 -- 172.21.15.45:0/2668605113 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f5ca819c690 con 0x7f5c8c0a3cc0 2024-04-25T18:55:54.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.321+0000 7f5c9f400700 1 -- 172.21.15.45:0/2668605113 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ca819c690 con 0x7f5c8c0a3cc0 2024-04-25T18:55:54.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 -- 172.21.15.45:0/2668605113 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5c8c0a3cc0 msgr2=0x7f5c8c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:54.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5c8c0a3cc0 0x7f5c8c0a6180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c98011490 tx=0x7f5c98009260 comp rx=0 tx=0).stop 2024-04-25T18:55:54.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 -- 172.21.15.45:0/2668605113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca8109fc0 msgr2=0x7f5ca819fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:54.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca8109fc0 0x7f5ca819fc00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5ca000b070 tx=0x7f5ca0004c60 comp rx=0 tx=0).stop 2024-04-25T18:55:54.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 -- 172.21.15.45:0/2668605113 shutdown_connections 2024-04-25T18:55:54.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5c8c0a3cc0 0x7f5c8c0a6180 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:54.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5ca8109fc0 0x7f5ca819fc00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:54.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5ca8104ee0 0x7f5ca819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:54.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 --2- 172.21.15.45:0/2668605113 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5ca8105890 0x7f5ca819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:54.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 -- 172.21.15.45:0/2668605113 >> 172.21.15.45:0/2668605113 conn(0x7f5ca8100770 msgr2=0x7f5ca8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:54.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 -- 172.21.15.45:0/2668605113 shutdown_connections 2024-04-25T18:55:54.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:54.324+0000 7f5cad800700 1 -- 172.21.15.45:0/2668605113 wait complete. 2024-04-25T18:55:54.714 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:55:54 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:55:54.288+0000 7f02c8000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:55:54.715 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:55:54 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:55:54.288+0000 7f02c8000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:55:54.715 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:55:54 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:55:54.288+0000 7f02c8000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:55:54.715 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:55:54 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:55:54.288+0000 7f02c8000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:55:54.715 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:55:54 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:55:54.288+0000 7f02c8000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:55:55.138 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T18:55:55.606 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:55.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:55 smithi045 ceph-mon[27776]: pgmap v341: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T18:55:55.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:55 smithi045 ceph-mon[27776]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:55.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:55 smithi045 ceph-mon[27776]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T18:55:55.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:55 smithi062 ceph-mon[32844]: pgmap v341: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T18:55:55.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:55 smithi062 ceph-mon[32844]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:55.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:55 smithi062 ceph-mon[32844]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T18:55:55.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:55 smithi079 ceph-mon[33002]: pgmap v341: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T18:55:55.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:55 smithi079 ceph-mon[33002]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:55.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:55 smithi079 ceph-mon[33002]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-25T18:55:57.552 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:57 smithi045 ceph-mon[27776]: pgmap v342: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T18:55:57.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.728+0000 7faaf7c00700 1 -- 172.21.15.45:0/2872113835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faaf0104ee0 msgr2=0x7faaf01052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:57.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.728+0000 7faaf7c00700 1 --2- 172.21.15.45:0/2872113835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faaf0104ee0 0x7faaf01052c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faae4009920 tx=0x7faae4009c30 comp rx=0 tx=0).stop 2024-04-25T18:55:57.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.728+0000 7faaf7c00700 1 -- 172.21.15.45:0/2872113835 shutdown_connections 2024-04-25T18:55:57.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.728+0000 7faaf7c00700 1 --2- 172.21.15.45:0/2872113835 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf0109fc0 0x7faaf0111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:57.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.728+0000 7faaf7c00700 1 --2- 172.21.15.45:0/2872113835 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faaf0104ee0 0x7faaf01052c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:57.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.728+0000 7faaf7c00700 1 --2- 172.21.15.45:0/2872113835 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0105890 0x7faaf01098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:57.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.728+0000 7faaf7c00700 1 -- 172.21.15.45:0/2872113835 >> 172.21.15.45:0/2872113835 conn(0x7faaf0100770 msgr2=0x7faaf0102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:57.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.728+0000 7faaf7c00700 1 -- 172.21.15.45:0/2872113835 shutdown_connections 2024-04-25T18:55:57.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.728+0000 7faaf7c00700 1 -- 172.21.15.45:0/2872113835 wait complete. 2024-04-25T18:55:57.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.729+0000 7faaf7c00700 1 Processor -- start 2024-04-25T18:55:57.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.730+0000 7faaf7c00700 1 -- start start 2024-04-25T18:55:57.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.730+0000 7faaf7c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf0104ee0 0x7faaf019b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:57.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.730+0000 7faaf7c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0105890 0x7faaf019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:57.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.730+0000 7faaf7c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faaf0109fc0 0x7faaf019fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:57.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.730+0000 7faaf7c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0114eb0 con 0x7faaf0105890 2024-04-25T18:55:57.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.730+0000 7faaf7c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0114d30 con 0x7faaf0104ee0 2024-04-25T18:55:57.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.730+0000 7faaf7c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7faaf0115030 con 0x7faaf0109fc0 2024-04-25T18:55:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.731+0000 7faaefe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0105890 0x7faaf019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.731+0000 7faaf5200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf0104ee0 0x7faaf019b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.731+0000 7faaf5200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf0104ee0 0x7faaf019b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:49082/0 (socket says 172.21.15.45:49082) 2024-04-25T18:55:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.731+0000 7faaf5200700 1 -- 172.21.15.45:0/3319539866 learned_addr learned my addr 172.21.15.45:0/3319539866 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:55:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.731+0000 7faaf5c00700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faaf0109fc0 0x7faaf019fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.731+0000 7faaf5200700 1 -- 172.21.15.45:0/3319539866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faaf0109fc0 msgr2=0x7faaf019fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:57.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.731+0000 7faaf5200700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faaf0109fc0 0x7faaf019fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.731+0000 7faaf5200700 1 -- 172.21.15.45:0/3319539866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0105890 msgr2=0x7faaf019ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.731+0000 7faaf5200700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0105890 0x7faaf019ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.731+0000 7faaf5200700 1 -- 172.21.15.45:0/3319539866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaf01a04f0 con 0x7faaf0104ee0 2024-04-25T18:55:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.732+0000 7faaf5200700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf0104ee0 0x7faaf019b4e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faae4000c00 tx=0x7faae400f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.732+0000 7faaed600700 1 -- 172.21.15.45:0/3319539866 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4027070 con 0x7faaf0104ee0 2024-04-25T18:55:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.732+0000 7faaed600700 1 -- 172.21.15.45:0/3319539866 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faae40054f0 con 0x7faaf0104ee0 2024-04-25T18:55:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.732+0000 7faaf7c00700 1 -- 172.21.15.45:0/3319539866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaf01a0780 con 0x7faaf0104ee0 2024-04-25T18:55:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.732+0000 7faaed600700 1 -- 172.21.15.45:0/3319539866 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faae4004930 con 0x7faaf0104ee0 2024-04-25T18:55:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.732+0000 7faaf7c00700 1 -- 172.21.15.45:0/3319539866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaf01a8060 con 0x7faaf0104ee0 2024-04-25T18:55:57.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.733+0000 7faaf7c00700 1 -- 172.21.15.45:0/3319539866 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaf0069730 con 0x7faaf0104ee0 2024-04-25T18:55:57.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.735+0000 7faaed600700 1 -- 172.21.15.45:0/3319539866 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7faae402a070 con 0x7faaf0104ee0 2024-04-25T18:55:57.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.737+0000 7faaed600700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7faad80a3cd0 0x7faad80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:55:57.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.737+0000 7faaed600700 1 -- 172.21.15.45:0/3319539866 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7faae40d0da0 con 0x7faaf0104ee0 2024-04-25T18:55:57.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.737+0000 7faaefe00700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7faad80a3cd0 0x7faad80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:55:57.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.738+0000 7faaefe00700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7faad80a3cd0 0x7faad80a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faadc009750 tx=0x7faadc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:55:57.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:57.741+0000 7faaed600700 1 -- 172.21.15.45:0/3319539866 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faae405eff0 con 0x7faaf0104ee0 2024-04-25T18:55:57.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:57 smithi062 ceph-mon[32844]: pgmap v342: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T18:55:57.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:57 smithi079 ceph-mon[33002]: pgmap v342: 97 pgs: 97 active+clean; 582 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-25T18:55:58.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.008+0000 7faaf7c00700 1 -- 172.21.15.45:0/3319539866 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7faaf019c840 con 0x7faad80a3cd0 2024-04-25T18:55:58.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.056+0000 7faaed600700 1 -- 172.21.15.45:0/3319539866 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faaf019c840 con 0x7faad80a3cd0 2024-04-25T18:55:58.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 -- 172.21.15.45:0/3319539866 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7faad80a3cd0 msgr2=0x7faad80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7faad80a3cd0 0x7faad80a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7faadc009750 tx=0x7faadc009310 comp rx=0 tx=0).stop 2024-04-25T18:55:58.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 -- 172.21.15.45:0/3319539866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf0104ee0 msgr2=0x7faaf019b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:55:58.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf0104ee0 0x7faaf019b4e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faae4000c00 tx=0x7faae400f990 comp rx=0 tx=0).stop 2024-04-25T18:55:58.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 -- 172.21.15.45:0/3319539866 shutdown_connections 2024-04-25T18:55:58.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7faad80a3cd0 0x7faad80a6190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7faaf0104ee0 0x7faaf019b4e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faaf0109fc0 0x7faaf019fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 --2- 172.21.15.45:0/3319539866 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faaf0105890 0x7faaf019ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:55:58.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 -- 172.21.15.45:0/3319539866 >> 172.21.15.45:0/3319539866 conn(0x7faaf0100770 msgr2=0x7faaf0101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:55:58.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.059+0000 7faaf7c00700 1 -- 172.21.15.45:0/3319539866 shutdown_connections 2024-04-25T18:55:58.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:55:58.060+0000 7faaf7c00700 1 -- 172.21.15.45:0/3319539866 wait complete. 2024-04-25T18:55:58.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:58 smithi045 ceph-mon[27776]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:58.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:58 smithi062 ceph-mon[32844]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:58.874 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T18:55:58.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:58 smithi079 ceph-mon[33002]: from='client.24608 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:55:59.345 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:55:59.800 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:55:59 smithi045 ceph-mon[27776]: pgmap v343: 97 pgs: 97 active+clean; 589 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-25T18:55:59.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:55:59 smithi062 ceph-mon[32844]: pgmap v343: 97 pgs: 97 active+clean; 589 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-25T18:55:59.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:55:59 smithi079 ceph-mon[33002]: pgmap v343: 97 pgs: 97 active+clean; 589 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-25T18:56:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.436+0000 7f4631c00700 1 -- 172.21.15.45:0/1832557223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f462c109c80 msgr2=0x7f462c10dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.436+0000 7f4631c00700 1 --2- 172.21.15.45:0/1832557223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f462c109c80 0x7f462c10dcd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4618009920 tx=0x7f4618009c30 comp rx=0 tx=0).stop 2024-04-25T18:56:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.437+0000 7f4631c00700 1 -- 172.21.15.45:0/1832557223 shutdown_connections 2024-04-25T18:56:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.437+0000 7f4631c00700 1 --2- 172.21.15.45:0/1832557223 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c1092d0 0x7f462c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.437+0000 7f4631c00700 1 --2- 172.21.15.45:0/1832557223 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f462c10e3b0 0x7f462c115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:01.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.437+0000 7f4631c00700 1 --2- 172.21.15.45:0/1832557223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f462c109c80 0x7f462c10dcd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.437+0000 7f4631c00700 1 -- 172.21.15.45:0/1832557223 >> 172.21.15.45:0/1832557223 conn(0x7f462c076210 msgr2=0x7f462c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.437+0000 7f4631c00700 1 -- 172.21.15.45:0/1832557223 shutdown_connections 2024-04-25T18:56:01.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.437+0000 7f4631c00700 1 -- 172.21.15.45:0/1832557223 wait complete. 2024-04-25T18:56:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.438+0000 7f4631c00700 1 Processor -- start 2024-04-25T18:56:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.438+0000 7f4631c00700 1 -- start start 2024-04-25T18:56:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.438+0000 7f4631c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f462c1092d0 0x7f462c1a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f4631c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c109c80 0x7f462c1a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f4631c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f462c10e3b0 0x7f462c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f4631c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f462c119240 con 0x7f462c1092d0 2024-04-25T18:56:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f4631c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1190c0 con 0x7f462c109c80 2024-04-25T18:56:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f4631c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f462c1193c0 con 0x7f462c10e3b0 2024-04-25T18:56:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f462c1092d0 0x7f462c1a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:01.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f462c1092d0 0x7f462c1a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:53242/0 (socket says 172.21.15.45:53242) 2024-04-25T18:56:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462aa00700 1 -- 172.21.15.45:0/2464717979 learned_addr learned my addr 172.21.15.45:0/2464717979 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462a000700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c109c80 0x7f462c1a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:01.441 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462b400700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f462c10e3b0 0x7f462c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462aa00700 1 -- 172.21.15.45:0/2464717979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f462c10e3b0 msgr2=0x7f462c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462aa00700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f462c10e3b0 0x7f462c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462aa00700 1 -- 172.21.15.45:0/2464717979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c109c80 msgr2=0x7f462c1a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462aa00700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c109c80 0x7f462c1a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462aa00700 1 -- 172.21.15.45:0/2464717979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462c19ff50 con 0x7f462c1092d0 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.439+0000 7f462a000700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c109c80 0x7f462c1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.440+0000 7f462aa00700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f462c1092d0 0x7f462c1a55a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f461c00e9c0 tx=0x7f461c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.440+0000 7f4623400700 1 -- 172.21.15.45:0/2464717979 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c00cbd0 con 0x7f462c1092d0 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.440+0000 7f4631c00700 1 -- 172.21.15.45:0/2464717979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462c1a0240 con 0x7f462c1092d0 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.440+0000 7f4623400700 1 -- 172.21.15.45:0/2464717979 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c00ee60 con 0x7f462c1092d0 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.440+0000 7f4623400700 1 -- 172.21.15.45:0/2464717979 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0055c0 con 0x7f462c1092d0 2024-04-25T18:56:01.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.440+0000 7f4631c00700 1 -- 172.21.15.45:0/2464717979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462c1ac520 con 0x7f462c1092d0 2024-04-25T18:56:01.443 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.441+0000 7f4631c00700 1 -- 172.21.15.45:0/2464717979 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462c10a7b0 con 0x7f462c1092d0 2024-04-25T18:56:01.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.444+0000 7f4623400700 1 -- 172.21.15.45:0/2464717979 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f461c004750 con 0x7f462c1092d0 2024-04-25T18:56:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.445+0000 7f4623400700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46100a3cd0 0x7f46100a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.445+0000 7f4623400700 1 -- 172.21.15.45:0/2464717979 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f461c0d0790 con 0x7f462c1092d0 2024-04-25T18:56:01.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.445+0000 7f462a000700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46100a3cd0 0x7f46100a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:01.447 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.446+0000 7f462a000700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46100a3cd0 0x7f46100a6190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f46180098f0 tx=0x7f4618019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:01.451 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.450+0000 7f4623400700 1 -- 172.21.15.45:0/2464717979 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f461c09d0c0 con 0x7f462c1092d0 2024-04-25T18:56:01.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:01 smithi062 ceph-mon[32844]: pgmap v344: 97 pgs: 97 active+clean; 589 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-25T18:56:01.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:01 smithi045 ceph-mon[27776]: pgmap v344: 97 pgs: 97 active+clean; 589 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-25T18:56:01.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.723+0000 7f4631c00700 1 -- 172.21.15.45:0/2464717979 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f462c069790 con 0x7f46100a3cd0 2024-04-25T18:56:01.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.732+0000 7f4623400700 1 -- 172.21.15.45:0/2464717979 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f462c069790 con 0x7f46100a3cd0 2024-04-25T18:56:01.734 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_0/62f0d381-412b-4b65-b519-719f8d121e7e 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.735+0000 7f4631c00700 1 -- 172.21.15.45:0/2464717979 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46100a3cd0 msgr2=0x7f46100a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.735+0000 7f4631c00700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46100a3cd0 0x7f46100a6190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f46180098f0 tx=0x7f4618019040 comp rx=0 tx=0).stop 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.735+0000 7f4631c00700 1 -- 172.21.15.45:0/2464717979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f462c1092d0 msgr2=0x7f462c1a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.735+0000 7f4631c00700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f462c1092d0 0x7f462c1a55a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f461c00e9c0 tx=0x7f461c00ecd0 comp rx=0 tx=0).stop 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.735+0000 7f4631c00700 1 -- 172.21.15.45:0/2464717979 shutdown_connections 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.735+0000 7f4631c00700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46100a3cd0 0x7f46100a6190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.735+0000 7f4631c00700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f462c10e3b0 0x7f462c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.735+0000 7f4631c00700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f462c1092d0 0x7f462c1a55a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.735+0000 7f4631c00700 1 --2- 172.21.15.45:0/2464717979 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f462c109c80 0x7f462c1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.735+0000 7f4631c00700 1 -- 172.21.15.45:0/2464717979 >> 172.21.15.45:0/2464717979 conn(0x7f462c076210 msgr2=0x7f462c077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.736+0000 7f4631c00700 1 -- 172.21.15.45:0/2464717979 shutdown_connections 2024-04-25T18:56:01.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:01.736+0000 7f4631c00700 1 -- 172.21.15.45:0/2464717979 wait complete. 2024-04-25T18:56:01.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:01 smithi079 ceph-mon[33002]: pgmap v344: 97 pgs: 97 active+clean; 589 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-25T18:56:02.569 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T18:56:02.866 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:02 smithi062 ceph-mon[32844]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:02.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:02 smithi079 ceph-mon[33002]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:02.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:02 smithi045 ceph-mon[27776]: from='client.14502 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:03.059 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:03.835 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:03 smithi079 ceph-mon[33002]: pgmap v345: 97 pgs: 97 active+clean; 608 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-25T18:56:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:03 smithi062 ceph-mon[32844]: pgmap v345: 97 pgs: 97 active+clean; 608 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-25T18:56:03.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:03 smithi045 ceph-mon[27776]: pgmap v345: 97 pgs: 97 active+clean; 608 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-04-25T18:56:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.164+0000 7fc018e00700 1 -- 172.21.15.45:0/703862819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc01406b0f0 msgr2=0x7fc01410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.164+0000 7fc018e00700 1 --2- 172.21.15.45:0/703862819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc01406b0f0 0x7fc01410f900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00b0a0 tx=0x7fc00c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.164+0000 7fc018e00700 1 -- 172.21.15.45:0/703862819 shutdown_connections 2024-04-25T18:56:05.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.164+0000 7fc018e00700 1 --2- 172.21.15.45:0/703862819 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc014108a30 0x7fc014108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.164+0000 7fc018e00700 1 --2- 172.21.15.45:0/703862819 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc01406b0f0 0x7fc01410f900 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.164+0000 7fc018e00700 1 --2- 172.21.15.45:0/703862819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc01406a750 0x7fc01406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.164+0000 7fc018e00700 1 -- 172.21.15.45:0/703862819 >> 172.21.15.45:0/703862819 conn(0x7fc0140788f0 msgr2=0x7fc014078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.164+0000 7fc018e00700 1 -- 172.21.15.45:0/703862819 shutdown_connections 2024-04-25T18:56:05.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.164+0000 7fc018e00700 1 -- 172.21.15.45:0/703862819 wait complete. 2024-04-25T18:56:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.165+0000 7fc018e00700 1 Processor -- start 2024-04-25T18:56:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.165+0000 7fc018e00700 1 -- start start 2024-04-25T18:56:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc018e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc01406a750 0x7fc01419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc018e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01406b0f0 0x7fc01419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:05.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc018e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014108a30 0x7fc01419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc018e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc014112c20 con 0x7fc014108a30 2024-04-25T18:56:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc018e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc014112aa0 con 0x7fc01406b0f0 2024-04-25T18:56:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc018e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc014112da0 con 0x7fc01406a750 2024-04-25T18:56:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc011e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc01406a750 0x7fc01419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc012800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014108a30 0x7fc01419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc011e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc01406a750 0x7fc01419b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:57372/0 (socket says 172.21.15.45:57372) 2024-04-25T18:56:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc011e00700 1 -- 172.21.15.45:0/3479758261 learned_addr learned my addr 172.21.15.45:0/3479758261 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:05.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.166+0000 7fc011400700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01406b0f0 0x7fc01419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc011400700 1 -- 172.21.15.45:0/3479758261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc01406a750 msgr2=0x7fc01419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc011400700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc01406a750 0x7fc01419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc011400700 1 -- 172.21.15.45:0/3479758261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014108a30 msgr2=0x7fc01419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc011400700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014108a30 0x7fc01419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc011400700 1 -- 172.21.15.45:0/3479758261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0141a0300 con 0x7fc01406b0f0 2024-04-25T18:56:05.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc011e00700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc01406a750 0x7fc01419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc012800700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014108a30 0x7fc01419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc011400700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01406b0f0 0x7fc01419b830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc00000d950 tx=0x7fc00000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc00aa00700 1 -- 172.21.15.45:0/3479758261 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000016070 con 0x7fc01406b0f0 2024-04-25T18:56:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc00aa00700 1 -- 172.21.15.45:0/3479758261 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc00000fd40 con 0x7fc01406b0f0 2024-04-25T18:56:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc018e00700 1 -- 172.21.15.45:0/3479758261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0141a05f0 con 0x7fc01406b0f0 2024-04-25T18:56:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.167+0000 7fc00aa00700 1 -- 172.21.15.45:0/3479758261 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00000e3b0 con 0x7fc01406b0f0 2024-04-25T18:56:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.168+0000 7fc018e00700 1 -- 172.21.15.45:0/3479758261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0141a7e70 con 0x7fc01406b0f0 2024-04-25T18:56:05.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.169+0000 7fc018e00700 1 -- 172.21.15.45:0/3479758261 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc014069730 con 0x7fc01406b0f0 2024-04-25T18:56:05.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.171+0000 7fc00aa00700 1 -- 172.21.15.45:0/3479758261 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc000019070 con 0x7fc01406b0f0 2024-04-25T18:56:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.172+0000 7fc00aa00700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3cd0 0x7fbff80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.172+0000 7fc00aa00700 1 -- 172.21.15.45:0/3479758261 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7fc0000cfeb0 con 0x7fc01406b0f0 2024-04-25T18:56:05.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.172+0000 7fc011e00700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3cd0 0x7fbff80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:05.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.173+0000 7fc011e00700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3cd0 0x7fbff80a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc004009680 tx=0x7fc004009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:05.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.177+0000 7fc00aa00700 1 -- 172.21.15.45:0/3479758261 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc00009c7e0 con 0x7fc01406b0f0 2024-04-25T18:56:05.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.443+0000 7fc018e00700 1 -- 172.21.15.45:0/3479758261 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fc014102790 con 0x7fbff80a3cd0 2024-04-25T18:56:05.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.479+0000 7fc00aa00700 1 -- 172.21.15.45:0/3479758261 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc014102790 con 0x7fbff80a3cd0 2024-04-25T18:56:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 -- 172.21.15.45:0/3479758261 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3cd0 msgr2=0x7fbff80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3cd0 0x7fbff80a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc004009680 tx=0x7fc004009260 comp rx=0 tx=0).stop 2024-04-25T18:56:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 -- 172.21.15.45:0/3479758261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01406b0f0 msgr2=0x7fc01419b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01406b0f0 0x7fc01419b830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc00000d950 tx=0x7fc00000dc60 comp rx=0 tx=0).stop 2024-04-25T18:56:05.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 -- 172.21.15.45:0/3479758261 shutdown_connections 2024-04-25T18:56:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3cd0 0x7fbff80a6190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc014108a30 0x7fc01419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc01406a750 0x7fc01419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 --2- 172.21.15.45:0/3479758261 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc01406b0f0 0x7fc01419b830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 -- 172.21.15.45:0/3479758261 >> 172.21.15.45:0/3479758261 conn(0x7fc0140788f0 msgr2=0x7fc0140ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:05.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 -- 172.21.15.45:0/3479758261 shutdown_connections 2024-04-25T18:56:05.486 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:05.482+0000 7fc018e00700 1 -- 172.21.15.45:0/3479758261 wait complete. 2024-04-25T18:56:05.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:05 smithi045 ceph-mon[27776]: pgmap v346: 97 pgs: 97 active+clean; 608 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T18:56:05.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:05 smithi062 ceph-mon[32844]: pgmap v346: 97 pgs: 97 active+clean; 608 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T18:56:05.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:05 smithi079 ceph-mon[33002]: pgmap v346: 97 pgs: 97 active+clean; 608 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-04-25T18:56:06.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T18:56:06.737 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:06.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:06 smithi062 ceph-mon[32844]: from='client.24620 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:06.882 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:06 smithi045 ceph-mon[27776]: from='client.24620 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:06.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:06 smithi079 ceph-mon[33002]: from='client.24620 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:07.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:07 smithi062 ceph-mon[32844]: pgmap v347: 97 pgs: 97 active+clean; 608 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-04-25T18:56:07.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:07 smithi045 ceph-mon[27776]: pgmap v347: 97 pgs: 97 active+clean; 608 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-04-25T18:56:07.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:07 smithi079 ceph-mon[33002]: pgmap v347: 97 pgs: 97 active+clean; 608 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2024-04-25T18:56:08.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.922+0000 7fe4e7000700 1 -- 172.21.15.45:0/4263396864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4e010e3b0 msgr2=0x7fe4e0115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:08.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.922+0000 7fe4e7000700 1 --2- 172.21.15.45:0/4263396864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4e010e3b0 0x7fe4e0115f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800b0d0 tx=0x7fe4d800b3e0 comp rx=0 tx=0).stop 2024-04-25T18:56:08.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.922+0000 7fe4e7000700 1 -- 172.21.15.45:0/4263396864 shutdown_connections 2024-04-25T18:56:08.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.922+0000 7fe4e7000700 1 --2- 172.21.15.45:0/4263396864 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4e010e3b0 0x7fe4e0115f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:08.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.922+0000 7fe4e7000700 1 --2- 172.21.15.45:0/4263396864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4e01092d0 0x7fe4e01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:08.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.922+0000 7fe4e7000700 1 --2- 172.21.15.45:0/4263396864 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e0109c80 0x7fe4e010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:08.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.922+0000 7fe4e7000700 1 -- 172.21.15.45:0/4263396864 >> 172.21.15.45:0/4263396864 conn(0x7fe4e0076210 msgr2=0x7fe4e0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:08.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.922+0000 7fe4e7000700 1 -- 172.21.15.45:0/4263396864 shutdown_connections 2024-04-25T18:56:08.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.922+0000 7fe4e7000700 1 -- 172.21.15.45:0/4263396864 wait complete. 2024-04-25T18:56:08.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.923+0000 7fe4e7000700 1 Processor -- start 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.924+0000 7fe4e7000700 1 -- start start 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.924+0000 7fe4e7000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4e01092d0 0x7fe4e019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.924+0000 7fe4e7000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e0109c80 0x7fe4e019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.924+0000 7fe4e7000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4e010e3b0 0x7fe4e01a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.924+0000 7fe4e7000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e01192c0 con 0x7fe4e01092d0 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.924+0000 7fe4e7000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e0119140 con 0x7fe4e0109c80 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.924+0000 7fe4e7000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e0119440 con 0x7fe4e010e3b0 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.924+0000 7fe4dfe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4e01092d0 0x7fe4e019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.924+0000 7fe4dfe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4e01092d0 0x7fe4e019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46106/0 (socket says 172.21.15.45:46106) 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.924+0000 7fe4dfe00700 1 -- 172.21.15.45:0/49760808 learned_addr learned my addr 172.21.15.45:0/49760808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.925+0000 7fe4e5000700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4e010e3b0 0x7fe4e01a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:08.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.925+0000 7fe4d7e00700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e0109c80 0x7fe4e019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.925+0000 7fe4dfe00700 1 -- 172.21.15.45:0/49760808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4e010e3b0 msgr2=0x7fe4e01a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.925+0000 7fe4dfe00700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4e010e3b0 0x7fe4e01a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.925+0000 7fe4dfe00700 1 -- 172.21.15.45:0/49760808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e0109c80 msgr2=0x7fe4e019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.925+0000 7fe4dfe00700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e0109c80 0x7fe4e019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.925+0000 7fe4dfe00700 1 -- 172.21.15.45:0/49760808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e01a4760 con 0x7fe4e01092d0 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.925+0000 7fe4dfe00700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4e01092d0 0x7fe4e019f750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000e9c0 tx=0x7fe4d000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.926+0000 7fe4dd600700 1 -- 172.21.15.45:0/49760808 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d000cbd0 con 0x7fe4e01092d0 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.926+0000 7fe4dd600700 1 -- 172.21.15.45:0/49760808 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4d000ee60 con 0x7fe4e01092d0 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.926+0000 7fe4dd600700 1 -- 172.21.15.45:0/49760808 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00055c0 con 0x7fe4e01092d0 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.926+0000 7fe4e7000700 1 -- 172.21.15.45:0/49760808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e01a4a50 con 0x7fe4e01092d0 2024-04-25T18:56:08.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.926+0000 7fe4e7000700 1 -- 172.21.15.45:0/49760808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e01ac2d0 con 0x7fe4e01092d0 2024-04-25T18:56:08.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.927+0000 7fe4e7000700 1 -- 172.21.15.45:0/49760808 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e0069730 con 0x7fe4e01092d0 2024-04-25T18:56:08.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.929+0000 7fe4dd600700 1 -- 172.21.15.45:0/49760808 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe4d0004750 con 0x7fe4e01092d0 2024-04-25T18:56:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.930+0000 7fe4dd600700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe4c40a3cd0 0x7fe4c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:08.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.931+0000 7fe4dd600700 1 -- 172.21.15.45:0/49760808 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7fe4d00d0790 con 0x7fe4e01092d0 2024-04-25T18:56:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.931+0000 7fe4d7e00700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe4c40a3cd0 0x7fe4c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:08.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.932+0000 7fe4d7e00700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe4c40a3cd0 0x7fe4c40a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009680 tx=0x7fe4c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:08.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:08.936+0000 7fe4dd600700 1 -- 172.21.15.45:0/49760808 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe4d009d0c0 con 0x7fe4e01092d0 2024-04-25T18:56:09.204 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.203+0000 7fe4e7000700 1 -- 172.21.15.45:0/49760808 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe4e0062e80 con 0x7fe4c40a3cd0 2024-04-25T18:56:09.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.212+0000 7fe4dd600700 1 -- 172.21.15.45:0/49760808 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe4e0062e80 con 0x7fe4c40a3cd0 2024-04-25T18:56:09.213 INFO:teuthology.orchestra.run.smithi045.stdout:/volumes/qa/sv_1/17f67f2f-4f96-4dd1-ba78-d9dd5e9b7635 2024-04-25T18:56:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.215+0000 7fe4e7000700 1 -- 172.21.15.45:0/49760808 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe4c40a3cd0 msgr2=0x7fe4c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.215+0000 7fe4e7000700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe4c40a3cd0 0x7fe4c40a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8009680 tx=0x7fe4c8009260 comp rx=0 tx=0).stop 2024-04-25T18:56:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.215+0000 7fe4e7000700 1 -- 172.21.15.45:0/49760808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4e01092d0 msgr2=0x7fe4e019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:09.216 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.215+0000 7fe4e7000700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4e01092d0 0x7fe4e019f750 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000e9c0 tx=0x7fe4d000ecd0 comp rx=0 tx=0).stop 2024-04-25T18:56:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.215+0000 7fe4e7000700 1 -- 172.21.15.45:0/49760808 shutdown_connections 2024-04-25T18:56:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.215+0000 7fe4e7000700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe4c40a3cd0 0x7fe4c40a6190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.215+0000 7fe4e7000700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4e010e3b0 0x7fe4e01a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.215+0000 7fe4e7000700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe4e01092d0 0x7fe4e019f750 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.215+0000 7fe4e7000700 1 --2- 172.21.15.45:0/49760808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe4e0109c80 0x7fe4e019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.215+0000 7fe4e7000700 1 -- 172.21.15.45:0/49760808 >> 172.21.15.45:0/49760808 conn(0x7fe4e0076210 msgr2=0x7fe4e0076790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.216+0000 7fe4e7000700 1 -- 172.21.15.45:0/49760808 shutdown_connections 2024-04-25T18:56:09.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:09.216+0000 7fe4e7000700 1 -- 172.21.15.45:0/49760808 wait complete. 2024-04-25T18:56:09.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:09 smithi062 ceph-mon[32844]: pgmap v348: 97 pgs: 97 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T18:56:09.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:09 smithi079 ceph-mon[33002]: pgmap v348: 97 pgs: 97 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T18:56:09.934 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:09 smithi045 ceph-mon[27776]: pgmap v348: 97 pgs: 97 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T18:56:10.006 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:56:10.473 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:10.798 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:10 smithi045 ceph-mon[27776]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:10.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:10 smithi062 ceph-mon[32844]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:10.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:10 smithi079 ceph-mon[33002]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:56:11.841 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:11 smithi045 ceph-mon[27776]: pgmap v349: 97 pgs: 97 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-04-25T18:56:11.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:11 smithi062 ceph-mon[32844]: pgmap v349: 97 pgs: 97 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-04-25T18:56:11.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:11 smithi079 ceph-mon[33002]: pgmap v349: 97 pgs: 97 active+clean; 625 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.8 KiB/s wr, 8 op/s 2024-04-25T18:56:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.635+0000 7f5962400700 1 -- 172.21.15.45:0/3777277618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c1137f0 msgr2=0x7f595c115be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.635+0000 7f5962400700 1 --2- 172.21.15.45:0/3777277618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c1137f0 0x7f595c115be0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f595400b0a0 tx=0x7f595400b3b0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.635+0000 7f5962400700 1 -- 172.21.15.45:0/3777277618 shutdown_connections 2024-04-25T18:56:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.635+0000 7f5962400700 1 --2- 172.21.15.45:0/3777277618 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c1137f0 0x7f595c115be0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.635+0000 7f5962400700 1 --2- 172.21.15.45:0/3777277618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c076f70 0x7f595c077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.635+0000 7f5962400700 1 --2- 172.21.15.45:0/3777277618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f595c077890 0x7f595c1132b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.635+0000 7f5962400700 1 -- 172.21.15.45:0/3777277618 >> 172.21.15.45:0/3777277618 conn(0x7f595c100620 msgr2=0x7f595c102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.635+0000 7f5962400700 1 -- 172.21.15.45:0/3777277618 shutdown_connections 2024-04-25T18:56:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.635+0000 7f5962400700 1 -- 172.21.15.45:0/3777277618 wait complete. 2024-04-25T18:56:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.636+0000 7f5962400700 1 Processor -- start 2024-04-25T18:56:12.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.636+0000 7f5962400700 1 -- start start 2024-04-25T18:56:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f5962400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c076f70 0x7f595c19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f5962400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c077890 0x7f595c19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f5962400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f595c1137f0 0x7f595c1a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f5962400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f595c1191a0 con 0x7f595c076f70 2024-04-25T18:56:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f5962400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f595c119020 con 0x7f595c077890 2024-04-25T18:56:12.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f5962400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f595c119320 con 0x7f595c1137f0 2024-04-25T18:56:12.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f5961000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c076f70 0x7f595c19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:12.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f5961000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c076f70 0x7f595c19f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46114/0 (socket says 172.21.15.45:46114) 2024-04-25T18:56:12.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f5961000700 1 -- 172.21.15.45:0/1894890892 learned_addr learned my addr 172.21.15.45:0/1894890892 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:12.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f595be00700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c077890 0x7f595c19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:12.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.637+0000 7f5961a00700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f595c1137f0 0x7f595c1a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5961a00700 1 -- 172.21.15.45:0/1894890892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c077890 msgr2=0x7f595c19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5961a00700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c077890 0x7f595c19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5961a00700 1 -- 172.21.15.45:0/1894890892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c076f70 msgr2=0x7f595c19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5961a00700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c076f70 0x7f595c19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5961a00700 1 -- 172.21.15.45:0/1894890892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595c1a46e0 con 0x7f595c1137f0 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f595be00700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c077890 0x7f595c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5961000700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c076f70 0x7f595c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5961a00700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f595c1137f0 0x7f595c1a3fa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f595400d010 tx=0x7f5954004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5959600700 1 -- 172.21.15.45:0/1894890892 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954021070 con 0x7f595c1137f0 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5959600700 1 -- 172.21.15.45:0/1894890892 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5954007e30 con 0x7f595c1137f0 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5962400700 1 -- 172.21.15.45:0/1894890892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595c1a4970 con 0x7f595c1137f0 2024-04-25T18:56:12.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.638+0000 7f5959600700 1 -- 172.21.15.45:0/1894890892 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59540045f0 con 0x7f595c1137f0 2024-04-25T18:56:12.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.639+0000 7f5962400700 1 -- 172.21.15.45:0/1894890892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595c1ac250 con 0x7f595c1137f0 2024-04-25T18:56:12.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.640+0000 7f5962400700 1 -- 172.21.15.45:0/1894890892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595c069730 con 0x7f595c1137f0 2024-04-25T18:56:12.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.642+0000 7f5959600700 1 -- 172.21.15.45:0/1894890892 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5954007980 con 0x7f595c1137f0 2024-04-25T18:56:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.643+0000 7f5959600700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f59440a3d50 0x7f59440a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.643+0000 7f5959600700 1 -- 172.21.15.45:0/1894890892 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 7979+0+0 (secure 0 0 0) 0x7f595400f020 con 0x7f595c1137f0 2024-04-25T18:56:12.644 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.643+0000 7f5961000700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f59440a3d50 0x7f59440a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:12.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.644+0000 7f5961000700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f59440a3d50 0x7f59440a6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5950011550 tx=0x7f5950009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:12.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.647+0000 7f5959600700 1 -- 172.21.15.45:0/1894890892 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f595402a020 con 0x7f595c1137f0 2024-04-25T18:56:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.928+0000 7f5962400700 1 -- 172.21.15.45:0/1894890892 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f595c02df80 con 0x7f595c1137f0 2024-04-25T18:56:12.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.929+0000 7f5959600700 1 -- 172.21.15.45:0/1894890892 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7f595409e100 con 0x7f595c1137f0 2024-04-25T18:56:12.931 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:56:12.931 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:55:50.062944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24512]},"id":1}]} 2024-04-25T18:56:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.933+0000 7f5962400700 1 -- 172.21.15.45:0/1894890892 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f59440a3d50 msgr2=0x7f59440a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.933+0000 7f5962400700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f59440a3d50 0x7f59440a6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5950011550 tx=0x7f5950009260 comp rx=0 tx=0).stop 2024-04-25T18:56:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.933+0000 7f5962400700 1 -- 172.21.15.45:0/1894890892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f595c1137f0 msgr2=0x7f595c1a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.933+0000 7f5962400700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f595c1137f0 0x7f595c1a3fa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f595400d010 tx=0x7f5954004c80 comp rx=0 tx=0).stop 2024-04-25T18:56:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.933+0000 7f5962400700 1 -- 172.21.15.45:0/1894890892 shutdown_connections 2024-04-25T18:56:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.933+0000 7f5962400700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f59440a3d50 0x7f59440a6210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.933+0000 7f5962400700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f595c1137f0 0x7f595c1a3fa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.933+0000 7f5962400700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f595c076f70 0x7f595c19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.933+0000 7f5962400700 1 --2- 172.21.15.45:0/1894890892 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f595c077890 0x7f595c19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.933+0000 7f5962400700 1 -- 172.21.15.45:0/1894890892 >> 172.21.15.45:0/1894890892 conn(0x7f595c100620 msgr2=0x7f595c102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.934+0000 7f5962400700 1 -- 172.21.15.45:0/1894890892 shutdown_connections 2024-04-25T18:56:12.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:12.934+0000 7f5962400700 1 -- 172.21.15.45:0/1894890892 wait complete. 2024-04-25T18:56:12.938 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-25T18:56:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:13 smithi045 ceph-mon[27776]: pgmap v350: 97 pgs: 97 active+clean; 634 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-04-25T18:56:13.802 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:13 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1894890892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:56:13.806 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:56:13.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:13 smithi062 ceph-mon[32844]: pgmap v350: 97 pgs: 97 active+clean; 634 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-04-25T18:56:13.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:13 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1894890892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:56:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:13 smithi079 ceph-mon[33002]: pgmap v350: 97 pgs: 97 active+clean; 634 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 9 op/s 2024-04-25T18:56:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:13 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1894890892' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:56:14.293 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:15.707 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:15 smithi045 ceph-mon[27776]: pgmap v351: 97 pgs: 97 active+clean; 634 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-25T18:56:15.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:15 smithi062 ceph-mon[32844]: pgmap v351: 97 pgs: 97 active+clean; 634 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-25T18:56:15.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:15 smithi079 ceph-mon[33002]: pgmap v351: 97 pgs: 97 active+clean; 634 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-04-25T18:56:16.537 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T18:56:16.537 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T18:56:16.537 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-25T18:56:16.537 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T18:56:16.537 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 6]}]: dispatch 2024-04-25T18:56:16.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T18:56:16.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-25T18:56:16.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-25T18:56:16.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T18:56:16.538 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:16 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T18:56:16.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.552+0000 7f22dc000700 1 -- 172.21.15.45:0/783731793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4102300 msgr2=0x7f22d410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:16.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.552+0000 7f22dc000700 1 --2- 172.21.15.45:0/783731793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4102300 0x7f22d410e7f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00b0a0 tx=0x7f22cc00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.553+0000 7f22dc000700 1 -- 172.21.15.45:0/783731793 shutdown_connections 2024-04-25T18:56:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.553+0000 7f22dc000700 1 --2- 172.21.15.45:0/783731793 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4102300 0x7f22d410e7f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.553+0000 7f22dc000700 1 --2- 172.21.15.45:0/783731793 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22d4107990 0x7f22d4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.553+0000 7f22dc000700 1 --2- 172.21.15.45:0/783731793 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d4101940 0x7f22d4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.553+0000 7f22dc000700 1 -- 172.21.15.45:0/783731793 >> 172.21.15.45:0/783731793 conn(0x7f22d40788f0 msgr2=0x7f22d4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.554+0000 7f22dc000700 1 -- 172.21.15.45:0/783731793 shutdown_connections 2024-04-25T18:56:16.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.554+0000 7f22dc000700 1 -- 172.21.15.45:0/783731793 wait complete. 2024-04-25T18:56:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.556+0000 7f22dc000700 1 Processor -- start 2024-04-25T18:56:16.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.556+0000 7f22dc000700 1 -- start start 2024-04-25T18:56:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.557+0000 7f22dc000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22d4101940 0x7f22d41034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.557+0000 7f22dc000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d4102300 0x7f22d4103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:16.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.557+0000 7f22dc000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4107990 0x7f22d4103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.557+0000 7f22dc000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f22d4111b50 con 0x7f22d4102300 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.557+0000 7f22dc000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f22d41119d0 con 0x7f22d4107990 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.557+0000 7f22dc000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22d4111cd0 con 0x7f22d4101940 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.557+0000 7f22d9600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22d4101940 0x7f22d41034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.557+0000 7f22da000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4107990 0x7f22d4103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.557+0000 7f22d9600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22d4101940 0x7f22d41034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39864/0 (socket says 172.21.15.45:39864) 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.557+0000 7f22d9600700 1 -- 172.21.15.45:0/2451109713 learned_addr learned my addr 172.21.15.45:0/2451109713 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.558+0000 7f22d9600700 1 -- 172.21.15.45:0/2451109713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4107990 msgr2=0x7f22d4103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.558+0000 7f22d9600700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4107990 0x7f22d4103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.558+0000 7f22d9600700 1 -- 172.21.15.45:0/2451109713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d4102300 msgr2=0x7f22d4103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.558+0000 7f22d9600700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d4102300 0x7f22d4103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.558+0000 7f22d9600700 1 -- 172.21.15.45:0/2451109713 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d41a7650 con 0x7f22d4101940 2024-04-25T18:56:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.558+0000 7f22d9600700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22d4101940 0x7f22d41034f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22c800e9c0 tx=0x7f22c800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.558+0000 7f22d2000700 1 -- 172.21.15.45:0/2451109713 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c800cbd0 con 0x7f22d4101940 2024-04-25T18:56:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.559+0000 7f22dc000700 1 -- 172.21.15.45:0/2451109713 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d41a7940 con 0x7f22d4101940 2024-04-25T18:56:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.559+0000 7f22d2000700 1 -- 172.21.15.45:0/2451109713 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c800cd70 con 0x7f22d4101940 2024-04-25T18:56:16.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.559+0000 7f22d2000700 1 -- 172.21.15.45:0/2451109713 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8005610 con 0x7f22d4101940 2024-04-25T18:56:16.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.559+0000 7f22dc000700 1 -- 172.21.15.45:0/2451109713 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d41a7e80 con 0x7f22d4101940 2024-04-25T18:56:16.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.560+0000 7f22dc000700 1 -- 172.21.15.45:0/2451109713 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d4069730 con 0x7f22d4101940 2024-04-25T18:56:16.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.562+0000 7f22d2000700 1 -- 172.21.15.45:0/2451109713 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f22c8004750 con 0x7f22d4101940 2024-04-25T18:56:16.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.565+0000 7f22d2000700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22bc0a3d50 0x7f22bc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.565+0000 7f22d2000700 1 -- 172.21.15.45:0/2451109713 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8269+0+0 (secure 0 0 0) 0x7f22c80d0990 con 0x7f22d4101940 2024-04-25T18:56:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.565+0000 7f22d8c00700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22bc0a3d50 0x7f22bc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:16.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.566+0000 7f22d8c00700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22bc0a3d50 0x7f22bc0a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005fd0 tx=0x7f22c0005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:16.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.566+0000 7f22d2000700 1 -- 172.21.15.45:0/2451109713 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f22c809d190 con 0x7f22d4101940 2024-04-25T18:56:16.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.803+0000 7f22dc000700 1 -- 172.21.15.45:0/2451109713 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22d402df80 con 0x7f22d4101940 2024-04-25T18:56:16.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.806+0000 7f22d2000700 1 -- 172.21.15.45:0/2451109713 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22654 (secure 0 0 0) 0x7f22c80bd0d0 con 0x7f22d4101940 2024-04-25T18:56:16.807 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:56:16.808 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":93,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T18:56:16.524852+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:55:17.591864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:55:22.070218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:56:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 -- 172.21.15.45:0/2451109713 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22bc0a3d50 msgr2=0x7f22bc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22bc0a3d50 0x7f22bc0a6210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005fd0 tx=0x7f22c0005f60 comp rx=0 tx=0).stop 2024-04-25T18:56:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 -- 172.21.15.45:0/2451109713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22d4101940 msgr2=0x7f22d41034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22d4101940 0x7f22d41034f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22c800e9c0 tx=0x7f22c800ecd0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 -- 172.21.15.45:0/2451109713 shutdown_connections 2024-04-25T18:56:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f22bc0a3d50 0x7f22bc0a6210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f22d4107990 0x7f22d4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22d4101940 0x7f22d41034f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 --2- 172.21.15.45:0/2451109713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f22d4102300 0x7f22d4103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:16.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 -- 172.21.15.45:0/2451109713 >> 172.21.15.45:0/2451109713 conn(0x7f22d40788f0 msgr2=0x7f22d406aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:16.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 -- 172.21.15.45:0/2451109713 shutdown_connections 2024-04-25T18:56:16.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:16.810+0000 7f22dc000700 1 -- 172.21.15.45:0/2451109713 wait complete. 2024-04-25T18:56:16.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T18:56:16.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T18:56:16.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-25T18:56:16.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T18:56:16.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 6]}]: dispatch 2024-04-25T18:56:16.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T18:56:16.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-25T18:56:16.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-25T18:56:16.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T18:56:16.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:16 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T18:56:16.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-04-25T18:56:16.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-04-25T18:56:16.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]: dispatch 2024-04-25T18:56:16.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]: dispatch 2024-04-25T18:56:16.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 6]}]: dispatch 2024-04-25T18:56:16.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]: dispatch 2024-04-25T18:56:16.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-04-25T18:56:16.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2024-04-25T18:56:16.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-04-25T18:56:16.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:16 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T18:56:17.599 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T18:56:17.599 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:56:17.821 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: pgmap v352: 97 pgs: 97 active+clean; 634 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-25T18:56:17.821 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T18:56:17.821 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T18:56:17.821 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-25T18:56:17.822 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T18:56:17.822 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 6]}]': finished 2024-04-25T18:56:17.822 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T18:56:17.822 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-25T18:56:17.822 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-25T18:56:17.823 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T18:56:17.823 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T18:56:17.823 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T18:56:17.823 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2451109713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:56:17.823 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:17 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:17.908 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: pgmap v352: 97 pgs: 97 active+clean; 634 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-25T18:56:17.908 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T18:56:17.908 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T18:56:17.909 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-25T18:56:17.909 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T18:56:17.909 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 6]}]': finished 2024-04-25T18:56:17.909 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T18:56:17.909 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-25T18:56:17.909 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-25T18:56:17.910 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T18:56:17.910 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T18:56:17.910 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T18:56:17.910 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2451109713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:56:17.910 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:17 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:17.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: pgmap v352: 97 pgs: 97 active+clean; 634 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 8 op/s 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.19", "id": [3, 8]}]': finished 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [4, 8]}]': finished 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [5, 6]}]': finished 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.23", "id": [4, 7]}]': finished 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2451109713' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:56:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:17 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:56:18.341 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:18.682 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:18 smithi045 ceph-mon[27776]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T18:56:18.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:18 smithi062 ceph-mon[32844]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T18:56:18.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:18 smithi079 ceph-mon[33002]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T18:56:19.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:19 smithi079 ceph-mon[33002]: pgmap v355: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T18:56:19.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:19.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:19.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:19.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:19 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:19.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:19 smithi045 ceph-mon[27776]: pgmap v355: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T18:56:19.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:19.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:19.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:19.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:19 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:20.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:19 smithi062 ceph-mon[32844]: pgmap v355: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T18:56:20.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:20.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:20.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:20.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:19 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:20.966 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.965+0000 7fcf29e00700 1 -- 172.21.15.45:0/1017691825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24109c80 msgr2=0x7fcf2410dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.965+0000 7fcf29e00700 1 --2- 172.21.15.45:0/1017691825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24109c80 0x7fcf2410dcd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009920 tx=0x7fcf10009c30 comp rx=0 tx=0).stop 2024-04-25T18:56:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.965+0000 7fcf29e00700 1 -- 172.21.15.45:0/1017691825 shutdown_connections 2024-04-25T18:56:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.965+0000 7fcf29e00700 1 --2- 172.21.15.45:0/1017691825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf2410e3b0 0x7fcf24115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.965+0000 7fcf29e00700 1 --2- 172.21.15.45:0/1017691825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf241092d0 0x7fcf241096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.965+0000 7fcf29e00700 1 --2- 172.21.15.45:0/1017691825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24109c80 0x7fcf2410dcd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:20.967 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.965+0000 7fcf29e00700 1 -- 172.21.15.45:0/1017691825 >> 172.21.15.45:0/1017691825 conn(0x7fcf24076210 msgr2=0x7fcf24107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.965+0000 7fcf29e00700 1 -- 172.21.15.45:0/1017691825 shutdown_connections 2024-04-25T18:56:20.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.965+0000 7fcf29e00700 1 -- 172.21.15.45:0/1017691825 wait complete. 2024-04-25T18:56:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.966+0000 7fcf29e00700 1 Processor -- start 2024-04-25T18:56:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.967+0000 7fcf29e00700 1 -- start start 2024-04-25T18:56:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.967+0000 7fcf29e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf241092d0 0x7fcf241a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.967+0000 7fcf29e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24109c80 0x7fcf241a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.967+0000 7fcf29e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf2410e3b0 0x7fcf2419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:20.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.967+0000 7fcf29e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24119290 con 0x7fcf24109c80 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.967+0000 7fcf29e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24119110 con 0x7fcf241092d0 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.967+0000 7fcf29e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24119410 con 0x7fcf2410e3b0 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24109c80 0x7fcf241a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf241092d0 0x7fcf241a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24109c80 0x7fcf241a5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51516/0 (socket says 172.21.15.45:51516) 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22000700 1 -- 172.21.15.45:0/1698110751 learned_addr learned my addr 172.21.15.45:0/1698110751 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf23400700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf2410e3b0 0x7fcf2419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22a00700 1 -- 172.21.15.45:0/1698110751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf2410e3b0 msgr2=0x7fcf2419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22a00700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf2410e3b0 0x7fcf2419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22a00700 1 -- 172.21.15.45:0/1698110751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24109c80 msgr2=0x7fcf241a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22a00700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24109c80 0x7fcf241a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22a00700 1 -- 172.21.15.45:0/1698110751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2419ffd0 con 0x7fcf241092d0 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22000700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24109c80 0x7fcf241a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.968+0000 7fcf22a00700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf241092d0 0x7fcf241a55d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400e9f0 tx=0x7fcf1400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:20.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.969+0000 7fcf28a00700 1 -- 172.21.15.45:0/1698110751 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14004220 con 0x7fcf241092d0 2024-04-25T18:56:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.969+0000 7fcf28a00700 1 -- 172.21.15.45:0/1698110751 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf140043c0 con 0x7fcf241092d0 2024-04-25T18:56:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.969+0000 7fcf29e00700 1 -- 172.21.15.45:0/1698110751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf241a02c0 con 0x7fcf241092d0 2024-04-25T18:56:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.969+0000 7fcf28a00700 1 -- 172.21.15.45:0/1698110751 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14005020 con 0x7fcf241092d0 2024-04-25T18:56:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.969+0000 7fcf29e00700 1 -- 172.21.15.45:0/1698110751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf241ac550 con 0x7fcf241092d0 2024-04-25T18:56:20.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.970+0000 7fcf29e00700 1 -- 172.21.15.45:0/1698110751 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2410a7b0 con 0x7fcf241092d0 2024-04-25T18:56:20.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.972+0000 7fcf28a00700 1 -- 172.21.15.45:0/1698110751 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcf14004750 con 0x7fcf241092d0 2024-04-25T18:56:20.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.973+0000 7fcf28a00700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fcf080a3d50 0x7fcf080a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:20.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.974+0000 7fcf28a00700 1 -- 172.21.15.45:0/1698110751 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fcf140d04c0 con 0x7fcf241092d0 2024-04-25T18:56:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.974+0000 7fcf22000700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fcf080a3d50 0x7fcf080a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:20.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.975+0000 7fcf22000700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fcf080a3d50 0x7fcf080a6210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcf241a0e60 tx=0x7fcf10019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:20.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:20.978+0000 7fcf28a00700 1 -- 172.21.15.45:0/1698110751 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf1409ccc0 con 0x7fcf241092d0 2024-04-25T18:56:21.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.252+0000 7fcf29e00700 1 -- 172.21.15.45:0/1698110751 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcf241a0bd0 con 0x7fcf080a3d50 2024-04-25T18:56:21.290 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.289+0000 7fcf28a00700 1 -- 172.21.15.45:0/1698110751 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258741 (secure 0 0 0) 0x7fcf241a0bd0 con 0x7fcf080a3d50 2024-04-25T18:56:21.296 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:56:21.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.296+0000 7fcf29e00700 1 -- 172.21.15.45:0/1698110751 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fcf080a3d50 msgr2=0x7fcf080a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:21.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.296+0000 7fcf29e00700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fcf080a3d50 0x7fcf080a6210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcf241a0e60 tx=0x7fcf10019040 comp rx=0 tx=0).stop 2024-04-25T18:56:21.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.296+0000 7fcf29e00700 1 -- 172.21.15.45:0/1698110751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf241092d0 msgr2=0x7fcf241a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:21.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.296+0000 7fcf29e00700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf241092d0 0x7fcf241a55d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400e9f0 tx=0x7fcf1400ed00 comp rx=0 tx=0).stop 2024-04-25T18:56:21.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.297+0000 7fcf29e00700 1 -- 172.21.15.45:0/1698110751 shutdown_connections 2024-04-25T18:56:21.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.297+0000 7fcf29e00700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fcf080a3d50 0x7fcf080a6210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:21.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.297+0000 7fcf29e00700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcf241092d0 0x7fcf241a55d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:21.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.297+0000 7fcf29e00700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf2410e3b0 0x7fcf2419f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:21.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.297+0000 7fcf29e00700 1 --2- 172.21.15.45:0/1698110751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcf24109c80 0x7fcf241a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:21.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.297+0000 7fcf29e00700 1 -- 172.21.15.45:0/1698110751 >> 172.21.15.45:0/1698110751 conn(0x7fcf24076210 msgr2=0x7fcf241086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:21.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.297+0000 7fcf29e00700 1 -- 172.21.15.45:0/1698110751 shutdown_connections 2024-04-25T18:56:21.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:21.297+0000 7fcf29e00700 1 -- 172.21.15.45:0/1698110751 wait complete. 2024-04-25T18:56:21.299 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T18:56:21.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:21 smithi079 ceph-mon[33002]: pgmap v356: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T18:56:21.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:21 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:21.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:21 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:21.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:21 smithi045 ceph-mon[27776]: pgmap v356: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T18:56:21.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:21 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:21.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:21 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:22.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:21 smithi062 ceph-mon[32844]: pgmap v356: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-04-25T18:56:22.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:21 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:22.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:21 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:22.261 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-04-25T18:56:20.263008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658066,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":354,"num_read_kb":299,"num_write":350,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392296,"kb_used_data":13988,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549528,"statfs":{"total":1151940427776,"available":1151538716672,"internally_reserved":0,"allocated":14323712,"data_stored":7848608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8705,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":41,"num_read_kb":29,"num_write":10,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002927"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.248982+0000","last_change":"2024-04-25T18:55:23.241030+0000","last_active":"2024-04-25T18:55:26.248982+0000","last_peered":"2024-04-25T18:55:26.248982+0000","last_clean":"2024-04-25T18:55:26.248982+0000","last_became_active":"2024-04-25T18:55:23.240724+0000","last_became_peered":"2024-04-25T18:55:23.240724+0000","last_unstale":"2024-04-25T18:55:26.248982+0000","last_undegraded":"2024-04-25T18:55:26.248982+0000","last_fullsized":"2024-04-25T18:55:26.248982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:37.995475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.255936+0000","last_change":"2024-04-25T18:55:23.229875+0000","last_active":"2024-04-25T18:55:26.255936+0000","last_peered":"2024-04-25T18:55:26.255936+0000","last_clean":"2024-04-25T18:55:26.255936+0000","last_became_active":"2024-04-25T18:55:23.229640+0000","last_became_peered":"2024-04-25T18:55:23.229640+0000","last_unstale":"2024-04-25T18:55:26.255936+0000","last_undegraded":"2024-04-25T18:55:26.255936+0000","last_fullsized":"2024-04-25T18:55:26.255936+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:10.262376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.255943+0000","last_change":"2024-04-25T18:55:23.229821+0000","last_active":"2024-04-25T18:55:26.255943+0000","last_peered":"2024-04-25T18:55:26.255943+0000","last_clean":"2024-04-25T18:55:26.255943+0000","last_became_active":"2024-04-25T18:55:23.229414+0000","last_became_peered":"2024-04-25T18:55:23.229414+0000","last_unstale":"2024-04-25T18:55:26.255943+0000","last_undegraded":"2024-04-25T18:55:26.255943+0000","last_fullsized":"2024-04-25T18:55:26.255943+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:56.063786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.562827+0000","last_change":"2024-04-25T18:56:17.560167+0000","last_active":"2024-04-25T18:56:17.562827+0000","last_peered":"2024-04-25T18:56:17.562827+0000","last_clean":"2024-04-25T18:56:17.562827+0000","last_became_active":"2024-04-25T18:56:17.558466+0000","last_became_peered":"2024-04-25T18:56:17.558466+0000","last_unstale":"2024-04-25T18:56:17.562827+0000","last_undegraded":"2024-04-25T18:56:17.562827+0000","last_fullsized":"2024-04-25T18:56:17.562827+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:22.371076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540378+0000","last_change":"2024-04-25T18:55:23.233512+0000","last_active":"2024-04-25T18:56:17.540378+0000","last_peered":"2024-04-25T18:56:17.540378+0000","last_clean":"2024-04-25T18:56:17.540378+0000","last_became_active":"2024-04-25T18:55:23.233158+0000","last_became_peered":"2024-04-25T18:55:23.233158+0000","last_unstale":"2024-04-25T18:56:17.540378+0000","last_undegraded":"2024-04-25T18:56:17.540378+0000","last_fullsized":"2024-04-25T18:56:17.540378+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:44.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.539214+0000","last_change":"2024-04-25T18:55:23.225921+0000","last_active":"2024-04-25T18:56:16.539214+0000","last_peered":"2024-04-25T18:56:16.539214+0000","last_clean":"2024-04-25T18:56:16.539214+0000","last_became_active":"2024-04-25T18:55:23.225547+0000","last_became_peered":"2024-04-25T18:55:23.225547+0000","last_unstale":"2024-04-25T18:56:16.539214+0000","last_undegraded":"2024-04-25T18:56:16.539214+0000","last_fullsized":"2024-04-25T18:56:16.539214+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:29.828394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.248455+0000","last_change":"2024-04-25T18:55:23.217111+0000","last_active":"2024-04-25T18:55:26.248455+0000","last_peered":"2024-04-25T18:55:26.248455+0000","last_clean":"2024-04-25T18:55:26.248455+0000","last_became_active":"2024-04-25T18:55:23.216731+0000","last_became_peered":"2024-04-25T18:55:23.216731+0000","last_unstale":"2024-04-25T18:55:26.248455+0000","last_undegraded":"2024-04-25T18:55:26.248455+0000","last_fullsized":"2024-04-25T18:55:26.248455+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:17.931928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.558980+0000","last_change":"2024-04-25T18:56:17.555014+0000","last_active":"2024-04-25T18:56:17.558980+0000","last_peered":"2024-04-25T18:56:17.558980+0000","last_clean":"2024-04-25T18:56:17.558980+0000","last_became_active":"2024-04-25T18:56:17.553863+0000","last_became_peered":"2024-04-25T18:56:17.553863+0000","last_unstale":"2024-04-25T18:56:17.558980+0000","last_undegraded":"2024-04-25T18:56:17.558980+0000","last_fullsized":"2024-04-25T18:56:17.558980+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.240754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.541075+0000","last_change":"2024-04-25T18:55:23.218197+0000","last_active":"2024-04-25T18:56:17.541075+0000","last_peered":"2024-04-25T18:56:17.541075+0000","last_clean":"2024-04-25T18:56:17.541075+0000","last_became_active":"2024-04-25T18:55:23.217665+0000","last_became_peered":"2024-04-25T18:55:23.217665+0000","last_unstale":"2024-04-25T18:56:17.541075+0000","last_undegraded":"2024-04-25T18:56:17.541075+0000","last_fullsized":"2024-04-25T18:56:17.541075+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:11.829991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.539598+0000","last_change":"2024-04-25T18:55:23.221533+0000","last_active":"2024-04-25T18:56:16.539598+0000","last_peered":"2024-04-25T18:56:16.539598+0000","last_clean":"2024-04-25T18:56:16.539598+0000","last_became_active":"2024-04-25T18:55:23.220711+0000","last_became_peered":"2024-04-25T18:55:23.220711+0000","last_unstale":"2024-04-25T18:56:16.539598+0000","last_undegraded":"2024-04-25T18:56:16.539598+0000","last_fullsized":"2024-04-25T18:56:16.539598+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:45.185166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.249496+0000","last_change":"2024-04-25T18:55:23.221329+0000","last_active":"2024-04-25T18:55:26.249496+0000","last_peered":"2024-04-25T18:55:26.249496+0000","last_clean":"2024-04-25T18:55:26.249496+0000","last_became_active":"2024-04-25T18:55:23.220156+0000","last_became_peered":"2024-04-25T18:55:23.220156+0000","last_unstale":"2024-04-25T18:55:26.249496+0000","last_undegraded":"2024-04-25T18:55:26.249496+0000","last_fullsized":"2024-04-25T18:55:26.249496+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:46.981199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.562773+0000","last_change":"2024-04-25T18:56:17.558018+0000","last_active":"2024-04-25T18:56:17.562773+0000","last_peered":"2024-04-25T18:56:17.562773+0000","last_clean":"2024-04-25T18:56:17.562773+0000","last_became_active":"2024-04-25T18:56:17.556229+0000","last_became_peered":"2024-04-25T18:56:17.556229+0000","last_unstale":"2024-04-25T18:56:17.562773+0000","last_undegraded":"2024-04-25T18:56:17.562773+0000","last_fullsized":"2024-04-25T18:56:17.562773+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:50.949895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550463+0000","last_change":"2024-04-25T18:55:23.227474+0000","last_active":"2024-04-25T18:56:17.550463+0000","last_peered":"2024-04-25T18:56:17.550463+0000","last_clean":"2024-04-25T18:56:17.550463+0000","last_became_active":"2024-04-25T18:55:23.227068+0000","last_became_peered":"2024-04-25T18:55:23.227068+0000","last_unstale":"2024-04-25T18:56:17.550463+0000","last_undegraded":"2024-04-25T18:56:17.550463+0000","last_fullsized":"2024-04-25T18:56:17.550463+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:13.884833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.252396+0000","last_change":"2024-04-25T18:55:23.230572+0000","last_active":"2024-04-25T18:55:26.252396+0000","last_peered":"2024-04-25T18:55:26.252396+0000","last_clean":"2024-04-25T18:55:26.252396+0000","last_became_active":"2024-04-25T18:55:23.229756+0000","last_became_peered":"2024-04-25T18:55:23.229756+0000","last_unstale":"2024-04-25T18:55:26.252396+0000","last_undegraded":"2024-04-25T18:55:26.252396+0000","last_fullsized":"2024-04-25T18:55:26.252396+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:15.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.250201+0000","last_change":"2024-04-25T18:55:23.227913+0000","last_active":"2024-04-25T18:55:26.250201+0000","last_peered":"2024-04-25T18:55:26.250201+0000","last_clean":"2024-04-25T18:55:26.250201+0000","last_became_active":"2024-04-25T18:55:23.227598+0000","last_became_peered":"2024-04-25T18:55:23.227598+0000","last_unstale":"2024-04-25T18:55:26.250201+0000","last_undegraded":"2024-04-25T18:55:26.250201+0000","last_fullsized":"2024-04-25T18:55:26.250201+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:33.555741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540636+0000","last_change":"2024-04-25T18:55:23.230241+0000","last_active":"2024-04-25T18:56:17.540636+0000","last_peered":"2024-04-25T18:56:17.540636+0000","last_clean":"2024-04-25T18:56:17.540636+0000","last_became_active":"2024-04-25T18:55:23.229898+0000","last_became_peered":"2024-04-25T18:55:23.229898+0000","last_unstale":"2024-04-25T18:56:17.540636+0000","last_undegraded":"2024-04-25T18:56:17.540636+0000","last_fullsized":"2024-04-25T18:56:17.540636+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:27.278475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.249379+0000","last_change":"2024-04-25T18:55:23.240978+0000","last_active":"2024-04-25T18:55:26.249379+0000","last_peered":"2024-04-25T18:55:26.249379+0000","last_clean":"2024-04-25T18:55:26.249379+0000","last_became_active":"2024-04-25T18:55:23.240702+0000","last_became_peered":"2024-04-25T18:55:23.240702+0000","last_unstale":"2024-04-25T18:55:26.249379+0000","last_undegraded":"2024-04-25T18:55:26.249379+0000","last_fullsized":"2024-04-25T18:55:26.249379+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:06.053688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.547162+0000","last_change":"2024-04-25T18:55:23.232116+0000","last_active":"2024-04-25T18:56:17.547162+0000","last_peered":"2024-04-25T18:56:17.547162+0000","last_clean":"2024-04-25T18:56:17.547162+0000","last_became_active":"2024-04-25T18:55:23.231586+0000","last_became_peered":"2024-04-25T18:55:23.231586+0000","last_unstale":"2024-04-25T18:56:17.547162+0000","last_undegraded":"2024-04-25T18:56:17.547162+0000","last_fullsized":"2024-04-25T18:56:17.547162+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:32.640823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536773+0000","last_change":"2024-04-25T18:55:23.224519+0000","last_active":"2024-04-25T18:56:17.536773+0000","last_peered":"2024-04-25T18:56:17.536773+0000","last_clean":"2024-04-25T18:56:17.536773+0000","last_became_active":"2024-04-25T18:55:23.224145+0000","last_became_peered":"2024-04-25T18:55:23.224145+0000","last_unstale":"2024-04-25T18:56:17.536773+0000","last_undegraded":"2024-04-25T18:56:17.536773+0000","last_fullsized":"2024-04-25T18:56:17.536773+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:03.340162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.548353+0000","last_change":"2024-04-25T18:55:19.164977+0000","last_active":"2024-04-25T18:56:17.548353+0000","last_peered":"2024-04-25T18:56:17.548353+0000","last_clean":"2024-04-25T18:56:17.548353+0000","last_became_active":"2024-04-25T18:55:19.164316+0000","last_became_peered":"2024-04-25T18:55:19.164316+0000","last_unstale":"2024-04-25T18:56:17.548353+0000","last_undegraded":"2024-04-25T18:56:17.548353+0000","last_fullsized":"2024-04-25T18:56:17.548353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:08.031680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":92,"state":"peering","last_fresh":"2024-04-25T18:55:26.231390+0000","last_change":"2024-04-25T18:56:16.539024+0000","last_active":"2024-04-25T18:56:16.539024+0000","last_peered":"2024-04-25T18:55:26.231390+0000","last_clean":"2024-04-25T18:55:26.231390+0000","last_became_active":"2024-04-25T18:55:23.232154+0000","last_became_peered":"2024-04-25T18:55:23.232154+0000","last_unstale":"2024-04-25T18:55:26.231390+0000","last_undegraded":"2024-04-25T18:55:26.231390+0000","last_fullsized":"2024-04-25T18:55:26.231390+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,5,6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.545240+0000","last_change":"2024-04-25T18:55:19.179943+0000","last_active":"2024-04-25T18:56:16.545240+0000","last_peered":"2024-04-25T18:56:16.545240+0000","last_clean":"2024-04-25T18:56:16.545240+0000","last_became_active":"2024-04-25T18:55:19.179519+0000","last_became_peered":"2024-04-25T18:55:19.179519+0000","last_unstale":"2024-04-25T18:56:16.545240+0000","last_undegraded":"2024-04-25T18:56:16.545240+0000","last_fullsized":"2024-04-25T18:56:16.545240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:46.365187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.257397+0000","last_change":"2024-04-25T18:55:23.237965+0000","last_active":"2024-04-25T18:55:26.257397+0000","last_peered":"2024-04-25T18:55:26.257397+0000","last_clean":"2024-04-25T18:55:26.257397+0000","last_became_active":"2024-04-25T18:55:23.237520+0000","last_became_peered":"2024-04-25T18:55:23.237520+0000","last_unstale":"2024-04-25T18:55:26.257397+0000","last_undegraded":"2024-04-25T18:55:26.257397+0000","last_fullsized":"2024-04-25T18:55:26.257397+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:03.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540660+0000","last_change":"2024-04-25T18:55:19.163965+0000","last_active":"2024-04-25T18:56:17.540660+0000","last_peered":"2024-04-25T18:56:17.540660+0000","last_clean":"2024-04-25T18:56:17.540660+0000","last_became_active":"2024-04-25T18:55:19.163390+0000","last_became_peered":"2024-04-25T18:55:19.163390+0000","last_unstale":"2024-04-25T18:56:17.540660+0000","last_undegraded":"2024-04-25T18:56:17.540660+0000","last_fullsized":"2024-04-25T18:56:17.540660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:54.183730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.541392+0000","last_change":"2024-04-25T18:55:23.227124+0000","last_active":"2024-04-25T18:56:17.541392+0000","last_peered":"2024-04-25T18:56:17.541392+0000","last_clean":"2024-04-25T18:56:17.541392+0000","last_became_active":"2024-04-25T18:55:23.226383+0000","last_became_peered":"2024-04-25T18:55:23.226383+0000","last_unstale":"2024-04-25T18:56:17.541392+0000","last_undegraded":"2024-04-25T18:56:17.541392+0000","last_fullsized":"2024-04-25T18:56:17.541392+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:22.011212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.251166+0000","last_change":"2024-04-25T18:55:19.160572+0000","last_active":"2024-04-25T18:55:26.251166+0000","last_peered":"2024-04-25T18:55:26.251166+0000","last_clean":"2024-04-25T18:55:26.251166+0000","last_became_active":"2024-04-25T18:55:19.160001+0000","last_became_peered":"2024-04-25T18:55:19.160001+0000","last_unstale":"2024-04-25T18:55:26.251166+0000","last_undegraded":"2024-04-25T18:55:26.251166+0000","last_fullsized":"2024-04-25T18:55:26.251166+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:07:00.242850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540179+0000","last_change":"2024-04-25T18:55:23.217703+0000","last_active":"2024-04-25T18:56:17.540179+0000","last_peered":"2024-04-25T18:56:17.540179+0000","last_clean":"2024-04-25T18:56:17.540179+0000","last_became_active":"2024-04-25T18:55:23.216767+0000","last_became_peered":"2024-04-25T18:55:23.216767+0000","last_unstale":"2024-04-25T18:56:17.540179+0000","last_undegraded":"2024-04-25T18:56:17.540179+0000","last_fullsized":"2024-04-25T18:56:17.540179+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:17.779356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":69,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:18.467032+0000","last_change":"2024-04-25T18:55:19.164199+0000","last_active":"2024-04-25T18:56:18.467032+0000","last_peered":"2024-04-25T18:56:18.467032+0000","last_clean":"2024-04-25T18:56:18.467032+0000","last_became_active":"2024-04-25T18:55:19.163849+0000","last_became_peered":"2024-04-25T18:55:19.163849+0000","last_unstale":"2024-04-25T18:56:18.467032+0000","last_undegraded":"2024-04-25T18:56:18.467032+0000","last_fullsized":"2024-04-25T18:56:18.467032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:25.234229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540648+0000","last_change":"2024-04-25T18:55:23.216916+0000","last_active":"2024-04-25T18:56:17.540648+0000","last_peered":"2024-04-25T18:56:17.540648+0000","last_clean":"2024-04-25T18:56:17.540648+0000","last_became_active":"2024-04-25T18:55:23.216249+0000","last_became_peered":"2024-04-25T18:55:23.216249+0000","last_unstale":"2024-04-25T18:56:17.540648+0000","last_undegraded":"2024-04-25T18:56:17.540648+0000","last_fullsized":"2024-04-25T18:56:17.540648+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:21.522318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.539162+0000","last_change":"2024-04-25T18:55:19.171004+0000","last_active":"2024-04-25T18:56:16.539162+0000","last_peered":"2024-04-25T18:56:16.539162+0000","last_clean":"2024-04-25T18:56:16.539162+0000","last_became_active":"2024-04-25T18:55:19.170570+0000","last_became_peered":"2024-04-25T18:55:19.170570+0000","last_unstale":"2024-04-25T18:56:16.539162+0000","last_undegraded":"2024-04-25T18:56:16.539162+0000","last_fullsized":"2024-04-25T18:56:16.539162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:15:24.954033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.552615+0000","last_change":"2024-04-25T18:56:17.552615+0000","last_active":"2024-04-25T18:56:17.552615+0000","last_peered":"2024-04-25T18:56:17.552615+0000","last_clean":"2024-04-25T18:56:17.552615+0000","last_became_active":"2024-04-25T18:56:17.552154+0000","last_became_peered":"2024-04-25T18:56:17.552154+0000","last_unstale":"2024-04-25T18:56:17.552615+0000","last_undegraded":"2024-04-25T18:56:17.552615+0000","last_fullsized":"2024-04-25T18:56:17.552615+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:01.199859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.549445+0000","last_change":"2024-04-25T18:55:19.173398+0000","last_active":"2024-04-25T18:56:17.549445+0000","last_peered":"2024-04-25T18:56:17.549445+0000","last_clean":"2024-04-25T18:56:17.549445+0000","last_became_active":"2024-04-25T18:55:19.171477+0000","last_became_peered":"2024-04-25T18:55:19.171477+0000","last_unstale":"2024-04-25T18:56:17.549445+0000","last_undegraded":"2024-04-25T18:56:17.549445+0000","last_fullsized":"2024-04-25T18:56:17.549445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:35.230596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.540501+0000","last_change":"2024-04-25T18:55:23.235209+0000","last_active":"2024-04-25T18:56:16.540501+0000","last_peered":"2024-04-25T18:56:16.540501+0000","last_clean":"2024-04-25T18:56:16.540501+0000","last_became_active":"2024-04-25T18:55:23.231898+0000","last_became_peered":"2024-04-25T18:55:23.231898+0000","last_unstale":"2024-04-25T18:56:16.540501+0000","last_undegraded":"2024-04-25T18:56:16.540501+0000","last_fullsized":"2024-04-25T18:56:16.540501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.521282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539932+0000","last_change":"2024-04-25T18:55:19.163780+0000","last_active":"2024-04-25T18:56:17.539932+0000","last_peered":"2024-04-25T18:56:17.539932+0000","last_clean":"2024-04-25T18:56:17.539932+0000","last_became_active":"2024-04-25T18:55:19.163197+0000","last_became_peered":"2024-04-25T18:55:19.163197+0000","last_unstale":"2024-04-25T18:56:17.539932+0000","last_undegraded":"2024-04-25T18:56:17.539932+0000","last_fullsized":"2024-04-25T18:56:17.539932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:02:26.519160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.556804+0000","last_change":"2024-04-25T18:56:17.556804+0000","last_active":"2024-04-25T18:56:17.556804+0000","last_peered":"2024-04-25T18:56:17.556804+0000","last_clean":"2024-04-25T18:56:17.556804+0000","last_became_active":"2024-04-25T18:56:17.554885+0000","last_became_peered":"2024-04-25T18:56:17.554885+0000","last_unstale":"2024-04-25T18:56:17.556804+0000","last_undegraded":"2024-04-25T18:56:17.556804+0000","last_fullsized":"2024-04-25T18:56:17.556804+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:36.316156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.539378+0000","last_change":"2024-04-25T18:55:19.170565+0000","last_active":"2024-04-25T18:56:16.539378+0000","last_peered":"2024-04-25T18:56:16.539378+0000","last_clean":"2024-04-25T18:56:16.539378+0000","last_became_active":"2024-04-25T18:55:19.170162+0000","last_became_peered":"2024-04-25T18:55:19.170162+0000","last_unstale":"2024-04-25T18:56:16.539378+0000","last_undegraded":"2024-04-25T18:56:16.539378+0000","last_fullsized":"2024-04-25T18:56:16.539378+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:36.606661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.541196+0000","last_change":"2024-04-25T18:55:23.227436+0000","last_active":"2024-04-25T18:56:17.541196+0000","last_peered":"2024-04-25T18:56:17.541196+0000","last_clean":"2024-04-25T18:56:17.541196+0000","last_became_active":"2024-04-25T18:55:23.227044+0000","last_became_peered":"2024-04-25T18:55:23.227044+0000","last_unstale":"2024-04-25T18:56:17.541196+0000","last_undegraded":"2024-04-25T18:56:17.541196+0000","last_fullsized":"2024-04-25T18:56:17.541196+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:34.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":24,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:42.380185+0000","last_change":"2024-04-25T18:55:19.167514+0000","last_active":"2024-04-25T18:55:42.380185+0000","last_peered":"2024-04-25T18:55:42.380185+0000","last_clean":"2024-04-25T18:55:42.380185+0000","last_became_active":"2024-04-25T18:55:19.167102+0000","last_became_peered":"2024-04-25T18:55:19.167102+0000","last_unstale":"2024-04-25T18:55:42.380185+0000","last_undegraded":"2024-04-25T18:55:42.380185+0000","last_fullsized":"2024-04-25T18:55:42.380185+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:41.472274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.257409+0000","last_change":"2024-04-25T18:55:23.237012+0000","last_active":"2024-04-25T18:55:26.257409+0000","last_peered":"2024-04-25T18:55:26.257409+0000","last_clean":"2024-04-25T18:55:26.257409+0000","last_became_active":"2024-04-25T18:55:23.230263+0000","last_became_peered":"2024-04-25T18:55:23.230263+0000","last_unstale":"2024-04-25T18:55:26.257409+0000","last_undegraded":"2024-04-25T18:55:26.257409+0000","last_fullsized":"2024-04-25T18:55:26.257409+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:30.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540250+0000","last_change":"2024-04-25T18:55:19.179555+0000","last_active":"2024-04-25T18:56:17.540250+0000","last_peered":"2024-04-25T18:56:17.540250+0000","last_clean":"2024-04-25T18:56:17.540250+0000","last_became_active":"2024-04-25T18:55:19.179190+0000","last_became_peered":"2024-04-25T18:55:19.179190+0000","last_unstale":"2024-04-25T18:56:17.540250+0000","last_undegraded":"2024-04-25T18:56:17.540250+0000","last_fullsized":"2024-04-25T18:56:17.540250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:57.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.541440+0000","last_change":"2024-04-25T18:55:23.219978+0000","last_active":"2024-04-25T18:56:17.541440+0000","last_peered":"2024-04-25T18:56:17.541440+0000","last_clean":"2024-04-25T18:56:17.541440+0000","last_became_active":"2024-04-25T18:55:23.219630+0000","last_became_peered":"2024-04-25T18:55:23.219630+0000","last_unstale":"2024-04-25T18:56:17.541440+0000","last_undegraded":"2024-04-25T18:56:17.541440+0000","last_fullsized":"2024-04-25T18:56:17.541440+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:31.294500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'30","reported_seq":125,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.539441+0000","last_change":"2024-04-25T18:55:19.164042+0000","last_active":"2024-04-25T18:56:16.539441+0000","last_peered":"2024-04-25T18:56:16.539441+0000","last_clean":"2024-04-25T18:56:16.539441+0000","last_became_active":"2024-04-25T18:55:19.163604+0000","last_became_peered":"2024-04-25T18:55:19.163604+0000","last_unstale":"2024-04-25T18:56:16.539441+0000","last_undegraded":"2024-04-25T18:56:16.539441+0000","last_fullsized":"2024-04-25T18:56:16.539441+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:55.030715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":71,"num_read_kb":96,"num_write":31,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540306+0000","last_change":"2024-04-25T18:55:23.222702+0000","last_active":"2024-04-25T18:56:17.540306+0000","last_peered":"2024-04-25T18:56:17.540306+0000","last_clean":"2024-04-25T18:56:17.540306+0000","last_became_active":"2024-04-25T18:55:23.221849+0000","last_became_peered":"2024-04-25T18:55:23.221849+0000","last_unstale":"2024-04-25T18:56:17.540306+0000","last_undegraded":"2024-04-25T18:56:17.540306+0000","last_fullsized":"2024-04-25T18:56:17.540306+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.957027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:43.391179+0000","last_change":"2024-04-25T18:55:19.166429+0000","last_active":"2024-04-25T18:55:43.391179+0000","last_peered":"2024-04-25T18:55:43.391179+0000","last_clean":"2024-04-25T18:55:43.391179+0000","last_became_active":"2024-04-25T18:55:19.165947+0000","last_became_peered":"2024-04-25T18:55:19.165947+0000","last_unstale":"2024-04-25T18:55:43.391179+0000","last_undegraded":"2024-04-25T18:55:43.391179+0000","last_fullsized":"2024-04-25T18:55:43.391179+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:30.846480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550782+0000","last_change":"2024-04-25T18:55:23.233074+0000","last_active":"2024-04-25T18:56:17.550782+0000","last_peered":"2024-04-25T18:56:17.550782+0000","last_clean":"2024-04-25T18:56:17.550782+0000","last_became_active":"2024-04-25T18:55:23.232751+0000","last_became_peered":"2024-04-25T18:55:23.232751+0000","last_unstale":"2024-04-25T18:56:17.550782+0000","last_undegraded":"2024-04-25T18:56:17.550782+0000","last_fullsized":"2024-04-25T18:56:17.550782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.584929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539986+0000","last_change":"2024-04-25T18:55:19.166093+0000","last_active":"2024-04-25T18:56:17.539986+0000","last_peered":"2024-04-25T18:56:17.539986+0000","last_clean":"2024-04-25T18:56:17.539986+0000","last_became_active":"2024-04-25T18:55:19.165715+0000","last_became_peered":"2024-04-25T18:55:19.165715+0000","last_unstale":"2024-04-25T18:56:17.539986+0000","last_undegraded":"2024-04-25T18:56:17.539986+0000","last_fullsized":"2024-04-25T18:56:17.539986+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:32.177943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.248768+0000","last_change":"2024-04-25T18:55:23.224617+0000","last_active":"2024-04-25T18:55:26.248768+0000","last_peered":"2024-04-25T18:55:26.248768+0000","last_clean":"2024-04-25T18:55:26.248768+0000","last_became_active":"2024-04-25T18:55:23.224269+0000","last_became_peered":"2024-04-25T18:55:23.224269+0000","last_unstale":"2024-04-25T18:55:26.248768+0000","last_undegraded":"2024-04-25T18:55:26.248768+0000","last_fullsized":"2024-04-25T18:55:26.248768+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:19.079910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'4","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:56:13.479287+0000","last_change":"2024-04-25T18:55:19.164971+0000","last_active":"2024-04-25T18:56:13.479287+0000","last_peered":"2024-04-25T18:56:13.479287+0000","last_clean":"2024-04-25T18:56:13.479287+0000","last_became_active":"2024-04-25T18:55:19.164579+0000","last_became_peered":"2024-04-25T18:55:19.164579+0000","last_unstale":"2024-04-25T18:56:13.479287+0000","last_undegraded":"2024-04-25T18:56:13.479287+0000","last_fullsized":"2024-04-25T18:56:13.479287+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:37.419379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.247082+0000","last_change":"2024-04-25T18:55:23.222160+0000","last_active":"2024-04-25T18:55:26.247082+0000","last_peered":"2024-04-25T18:55:26.247082+0000","last_clean":"2024-04-25T18:55:26.247082+0000","last_became_active":"2024-04-25T18:55:23.221812+0000","last_became_peered":"2024-04-25T18:55:23.221812+0000","last_unstale":"2024-04-25T18:55:26.247082+0000","last_undegraded":"2024-04-25T18:55:26.247082+0000","last_fullsized":"2024-04-25T18:55:26.247082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:53:53.388764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540070+0000","last_change":"2024-04-25T18:55:19.179970+0000","last_active":"2024-04-25T18:56:17.540070+0000","last_peered":"2024-04-25T18:56:17.540070+0000","last_clean":"2024-04-25T18:56:17.540070+0000","last_became_active":"2024-04-25T18:55:19.179545+0000","last_became_peered":"2024-04-25T18:55:19.179545+0000","last_unstale":"2024-04-25T18:56:17.540070+0000","last_undegraded":"2024-04-25T18:56:17.540070+0000","last_fullsized":"2024-04-25T18:56:17.540070+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:42.841661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539929+0000","last_change":"2024-04-25T18:55:23.216134+0000","last_active":"2024-04-25T18:56:17.539929+0000","last_peered":"2024-04-25T18:56:17.539929+0000","last_clean":"2024-04-25T18:56:17.539929+0000","last_became_active":"2024-04-25T18:55:23.215535+0000","last_became_peered":"2024-04-25T18:55:23.215535+0000","last_unstale":"2024-04-25T18:56:17.539929+0000","last_undegraded":"2024-04-25T18:56:17.539929+0000","last_fullsized":"2024-04-25T18:56:17.539929+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:08.042095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540676+0000","last_change":"2024-04-25T18:55:19.162437+0000","last_active":"2024-04-25T18:56:17.540676+0000","last_peered":"2024-04-25T18:56:17.540676+0000","last_clean":"2024-04-25T18:56:17.540676+0000","last_became_active":"2024-04-25T18:55:19.162005+0000","last_became_peered":"2024-04-25T18:55:19.162005+0000","last_unstale":"2024-04-25T18:56:17.540676+0000","last_undegraded":"2024-04-25T18:56:17.540676+0000","last_fullsized":"2024-04-25T18:56:17.540676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:50:53.967081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.249696+0000","last_change":"2024-04-25T18:55:23.222345+0000","last_active":"2024-04-25T18:55:26.249696+0000","last_peered":"2024-04-25T18:55:26.249696+0000","last_clean":"2024-04-25T18:55:26.249696+0000","last_became_active":"2024-04-25T18:55:23.221142+0000","last_became_peered":"2024-04-25T18:55:23.221142+0000","last_unstale":"2024-04-25T18:55:26.249696+0000","last_undegraded":"2024-04-25T18:55:26.249696+0000","last_fullsized":"2024-04-25T18:55:26.249696+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:53.823585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550605+0000","last_change":"2024-04-25T18:55:19.181153+0000","last_active":"2024-04-25T18:56:17.550605+0000","last_peered":"2024-04-25T18:56:17.550605+0000","last_clean":"2024-04-25T18:56:17.550605+0000","last_became_active":"2024-04-25T18:55:19.180553+0000","last_became_peered":"2024-04-25T18:55:19.180553+0000","last_unstale":"2024-04-25T18:56:17.550605+0000","last_undegraded":"2024-04-25T18:56:17.550605+0000","last_fullsized":"2024-04-25T18:56:17.550605+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:07.829617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.539692+0000","last_change":"2024-04-25T18:55:23.238824+0000","last_active":"2024-04-25T18:56:16.539692+0000","last_peered":"2024-04-25T18:56:16.539692+0000","last_clean":"2024-04-25T18:56:16.539692+0000","last_became_active":"2024-04-25T18:55:23.238228+0000","last_became_peered":"2024-04-25T18:55:23.238228+0000","last_unstale":"2024-04-25T18:56:16.539692+0000","last_undegraded":"2024-04-25T18:56:16.539692+0000","last_fullsized":"2024-04-25T18:56:16.539692+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:16.279254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":23,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.248743+0000","last_change":"2024-04-25T18:55:19.171186+0000","last_active":"2024-04-25T18:55:26.248743+0000","last_peered":"2024-04-25T18:55:26.248743+0000","last_clean":"2024-04-25T18:55:26.248743+0000","last_became_active":"2024-04-25T18:55:19.170800+0000","last_became_peered":"2024-04-25T18:55:19.170800+0000","last_unstale":"2024-04-25T18:55:26.248743+0000","last_undegraded":"2024-04-25T18:55:26.248743+0000","last_fullsized":"2024-04-25T18:55:26.248743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.599119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.539732+0000","last_change":"2024-04-25T18:55:23.236951+0000","last_active":"2024-04-25T18:56:16.539732+0000","last_peered":"2024-04-25T18:56:16.539732+0000","last_clean":"2024-04-25T18:56:16.539732+0000","last_became_active":"2024-04-25T18:55:23.236596+0000","last_became_peered":"2024-04-25T18:55:23.236596+0000","last_unstale":"2024-04-25T18:56:16.539732+0000","last_undegraded":"2024-04-25T18:56:16.539732+0000","last_fullsized":"2024-04-25T18:56:16.539732+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:14.634653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540262+0000","last_change":"2024-04-25T18:55:23.211984+0000","last_active":"2024-04-25T18:56:17.540262+0000","last_peered":"2024-04-25T18:56:17.540262+0000","last_clean":"2024-04-25T18:56:17.540262+0000","last_became_active":"2024-04-25T18:55:23.211591+0000","last_became_peered":"2024-04-25T18:55:23.211591+0000","last_unstale":"2024-04-25T18:56:17.540262+0000","last_undegraded":"2024-04-25T18:56:17.540262+0000","last_fullsized":"2024-04-25T18:56:17.540262+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:09.024891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.549379+0000","last_change":"2024-04-25T18:55:19.180378+0000","last_active":"2024-04-25T18:56:17.549379+0000","last_peered":"2024-04-25T18:56:17.549379+0000","last_clean":"2024-04-25T18:56:17.549379+0000","last_became_active":"2024-04-25T18:55:19.179905+0000","last_became_peered":"2024-04-25T18:55:19.179905+0000","last_unstale":"2024-04-25T18:56:17.549379+0000","last_undegraded":"2024-04-25T18:56:17.549379+0000","last_fullsized":"2024-04-25T18:56:17.549379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:36.434477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.541915+0000","last_change":"2024-04-25T18:55:23.231489+0000","last_active":"2024-04-25T18:56:16.541915+0000","last_peered":"2024-04-25T18:56:16.541915+0000","last_clean":"2024-04-25T18:56:16.541915+0000","last_became_active":"2024-04-25T18:55:23.231049+0000","last_became_peered":"2024-04-25T18:55:23.231049+0000","last_unstale":"2024-04-25T18:56:16.541915+0000","last_undegraded":"2024-04-25T18:56:16.541915+0000","last_fullsized":"2024-04-25T18:56:16.541915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:25.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:46.413765+0000","last_change":"2024-04-25T18:55:19.171694+0000","last_active":"2024-04-25T18:55:46.413765+0000","last_peered":"2024-04-25T18:55:46.413765+0000","last_clean":"2024-04-25T18:55:46.413765+0000","last_became_active":"2024-04-25T18:55:19.171296+0000","last_became_peered":"2024-04-25T18:55:19.171296+0000","last_unstale":"2024-04-25T18:55:46.413765+0000","last_undegraded":"2024-04-25T18:55:46.413765+0000","last_fullsized":"2024-04-25T18:55:46.413765+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:33.216980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.247263+0000","last_change":"2024-04-25T18:53:49.127506+0000","last_active":"2024-04-25T18:55:26.247263+0000","last_peered":"2024-04-25T18:55:26.247263+0000","last_clean":"2024-04-25T18:55:26.247263+0000","last_became_active":"2024-04-25T18:53:49.126974+0000","last_became_peered":"2024-04-25T18:53:49.126974+0000","last_unstale":"2024-04-25T18:55:26.247263+0000","last_undegraded":"2024-04-25T18:55:26.247263+0000","last_fullsized":"2024-04-25T18:55:26.247263+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_clean_scrub_stamp":"2024-04-25T18:50:00.765468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.488525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.549219+0000","last_change":"2024-04-25T18:55:23.211697+0000","last_active":"2024-04-25T18:56:17.549219+0000","last_peered":"2024-04-25T18:56:17.549219+0000","last_clean":"2024-04-25T18:56:17.549219+0000","last_became_active":"2024-04-25T18:55:23.211160+0000","last_became_peered":"2024-04-25T18:55:23.211160+0000","last_unstale":"2024-04-25T18:56:17.549219+0000","last_undegraded":"2024-04-25T18:56:17.549219+0000","last_fullsized":"2024-04-25T18:56:17.549219+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:03.452578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:43.383480+0000","last_change":"2024-04-25T18:55:19.165955+0000","last_active":"2024-04-25T18:55:43.383480+0000","last_peered":"2024-04-25T18:55:43.383480+0000","last_clean":"2024-04-25T18:55:43.383480+0000","last_became_active":"2024-04-25T18:55:19.165594+0000","last_became_peered":"2024-04-25T18:55:19.165594+0000","last_unstale":"2024-04-25T18:55:43.383480+0000","last_undegraded":"2024-04-25T18:55:43.383480+0000","last_fullsized":"2024-04-25T18:55:43.383480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:11.692374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540424+0000","last_change":"2024-04-25T18:55:19.182698+0000","last_active":"2024-04-25T18:56:17.540424+0000","last_peered":"2024-04-25T18:56:17.540424+0000","last_clean":"2024-04-25T18:56:17.540424+0000","last_became_active":"2024-04-25T18:55:19.182393+0000","last_became_peered":"2024-04-25T18:55:19.182393+0000","last_unstale":"2024-04-25T18:56:17.540424+0000","last_undegraded":"2024-04-25T18:56:17.540424+0000","last_fullsized":"2024-04-25T18:56:17.540424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.513613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550144+0000","last_change":"2024-04-25T18:55:23.223240+0000","last_active":"2024-04-25T18:56:17.550144+0000","last_peered":"2024-04-25T18:56:17.550144+0000","last_clean":"2024-04-25T18:56:17.550144+0000","last_became_active":"2024-04-25T18:55:23.218253+0000","last_became_peered":"2024-04-25T18:55:23.218253+0000","last_unstale":"2024-04-25T18:56:17.550144+0000","last_undegraded":"2024-04-25T18:56:17.550144+0000","last_fullsized":"2024-04-25T18:56:17.550144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:40.435633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'12","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.547262+0000","last_change":"2024-04-25T18:55:19.168535+0000","last_active":"2024-04-25T18:56:17.547262+0000","last_peered":"2024-04-25T18:56:17.547262+0000","last_clean":"2024-04-25T18:56:17.547262+0000","last_became_active":"2024-04-25T18:55:19.166735+0000","last_became_peered":"2024-04-25T18:55:19.166735+0000","last_unstale":"2024-04-25T18:56:17.547262+0000","last_undegraded":"2024-04-25T18:56:17.547262+0000","last_fullsized":"2024-04-25T18:56:17.547262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:50:14.008290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.544950+0000","last_change":"2024-04-25T18:55:23.233989+0000","last_active":"2024-04-25T18:56:16.544950+0000","last_peered":"2024-04-25T18:56:16.544950+0000","last_clean":"2024-04-25T18:56:16.544950+0000","last_became_active":"2024-04-25T18:55:23.226867+0000","last_became_peered":"2024-04-25T18:55:23.226867+0000","last_unstale":"2024-04-25T18:56:16.544950+0000","last_undegraded":"2024-04-25T18:56:16.544950+0000","last_fullsized":"2024-04-25T18:56:16.544950+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:02.314368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:42.381134+0000","last_change":"2024-04-25T18:55:19.870412+0000","last_active":"2024-04-25T18:55:42.381134+0000","last_peered":"2024-04-25T18:55:42.381134+0000","last_clean":"2024-04-25T18:55:42.381134+0000","last_became_active":"2024-04-25T18:55:19.869970+0000","last_became_peered":"2024-04-25T18:55:19.869970+0000","last_unstale":"2024-04-25T18:55:42.381134+0000","last_undegraded":"2024-04-25T18:55:42.381134+0000","last_fullsized":"2024-04-25T18:55:42.381134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:22.236417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.257213+0000","last_change":"2024-04-25T18:55:23.239172+0000","last_active":"2024-04-25T18:55:26.257213+0000","last_peered":"2024-04-25T18:55:26.257213+0000","last_clean":"2024-04-25T18:55:26.257213+0000","last_became_active":"2024-04-25T18:55:23.238732+0000","last_became_peered":"2024-04-25T18:55:23.238732+0000","last_unstale":"2024-04-25T18:55:26.257213+0000","last_undegraded":"2024-04-25T18:55:26.257213+0000","last_fullsized":"2024-04-25T18:55:26.257213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:27.459659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540749+0000","last_change":"2024-04-25T18:55:19.179687+0000","last_active":"2024-04-25T18:56:17.540749+0000","last_peered":"2024-04-25T18:56:17.540749+0000","last_clean":"2024-04-25T18:56:17.540749+0000","last_became_active":"2024-04-25T18:55:19.179225+0000","last_became_peered":"2024-04-25T18:55:19.179225+0000","last_unstale":"2024-04-25T18:56:17.540749+0000","last_undegraded":"2024-04-25T18:56:17.540749+0000","last_fullsized":"2024-04-25T18:56:17.540749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:16:16.266021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540122+0000","last_change":"2024-04-25T18:55:23.221379+0000","last_active":"2024-04-25T18:56:17.540122+0000","last_peered":"2024-04-25T18:56:17.540122+0000","last_clean":"2024-04-25T18:56:17.540122+0000","last_became_active":"2024-04-25T18:55:23.221037+0000","last_became_peered":"2024-04-25T18:55:23.221037+0000","last_unstale":"2024-04-25T18:56:17.540122+0000","last_undegraded":"2024-04-25T18:56:17.540122+0000","last_fullsized":"2024-04-25T18:56:17.540122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:27.581084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.539810+0000","last_change":"2024-04-25T18:55:19.171298+0000","last_active":"2024-04-25T18:56:16.539810+0000","last_peered":"2024-04-25T18:56:16.539810+0000","last_clean":"2024-04-25T18:56:16.539810+0000","last_became_active":"2024-04-25T18:55:19.170738+0000","last_became_peered":"2024-04-25T18:55:19.170738+0000","last_unstale":"2024-04-25T18:56:16.539810+0000","last_undegraded":"2024-04-25T18:56:16.539810+0000","last_fullsized":"2024-04-25T18:56:16.539810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.236331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.249899+0000","last_change":"2024-04-25T18:55:23.226577+0000","last_active":"2024-04-25T18:55:26.249899+0000","last_peered":"2024-04-25T18:55:26.249899+0000","last_clean":"2024-04-25T18:55:26.249899+0000","last_became_active":"2024-04-25T18:55:23.226198+0000","last_became_peered":"2024-04-25T18:55:23.226198+0000","last_unstale":"2024-04-25T18:55:26.249899+0000","last_undegraded":"2024-04-25T18:55:26.249899+0000","last_fullsized":"2024-04-25T18:55:26.249899+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:34:59.124523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536353+0000","last_change":"2024-04-25T18:55:19.171917+0000","last_active":"2024-04-25T18:56:17.536353+0000","last_peered":"2024-04-25T18:56:17.536353+0000","last_clean":"2024-04-25T18:56:17.536353+0000","last_became_active":"2024-04-25T18:55:19.171506+0000","last_became_peered":"2024-04-25T18:55:19.171506+0000","last_unstale":"2024-04-25T18:56:17.536353+0000","last_undegraded":"2024-04-25T18:56:17.536353+0000","last_fullsized":"2024-04-25T18:56:17.536353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:53.446504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.549350+0000","last_change":"2024-04-25T18:55:23.237828+0000","last_active":"2024-04-25T18:56:16.549350+0000","last_peered":"2024-04-25T18:56:16.549350+0000","last_clean":"2024-04-25T18:56:16.549350+0000","last_became_active":"2024-04-25T18:55:23.237310+0000","last_became_peered":"2024-04-25T18:55:23.237310+0000","last_unstale":"2024-04-25T18:56:16.549350+0000","last_undegraded":"2024-04-25T18:56:16.549350+0000","last_fullsized":"2024-04-25T18:56:16.549350+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:10.987032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540249+0000","last_change":"2024-04-25T18:55:19.163824+0000","last_active":"2024-04-25T18:56:17.540249+0000","last_peered":"2024-04-25T18:56:17.540249+0000","last_clean":"2024-04-25T18:56:17.540249+0000","last_became_active":"2024-04-25T18:55:19.163465+0000","last_became_peered":"2024-04-25T18:55:19.163465+0000","last_unstale":"2024-04-25T18:56:17.540249+0000","last_undegraded":"2024-04-25T18:56:17.540249+0000","last_fullsized":"2024-04-25T18:56:17.540249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:38.347100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.248885+0000","last_change":"2024-04-25T18:55:23.224648+0000","last_active":"2024-04-25T18:55:26.248885+0000","last_peered":"2024-04-25T18:55:26.248885+0000","last_clean":"2024-04-25T18:55:26.248885+0000","last_became_active":"2024-04-25T18:55:23.224323+0000","last_became_peered":"2024-04-25T18:55:23.224323+0000","last_unstale":"2024-04-25T18:55:26.248885+0000","last_undegraded":"2024-04-25T18:55:26.248885+0000","last_fullsized":"2024-04-25T18:55:26.248885+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.117792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536684+0000","last_change":"2024-04-25T18:55:19.870213+0000","last_active":"2024-04-25T18:56:17.536684+0000","last_peered":"2024-04-25T18:56:17.536684+0000","last_clean":"2024-04-25T18:56:17.536684+0000","last_became_active":"2024-04-25T18:55:19.869756+0000","last_became_peered":"2024-04-25T18:55:19.869756+0000","last_unstale":"2024-04-25T18:56:17.536684+0000","last_undegraded":"2024-04-25T18:56:17.536684+0000","last_fullsized":"2024-04-25T18:56:17.536684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:44.695876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.249792+0000","last_change":"2024-04-25T18:55:23.217094+0000","last_active":"2024-04-25T18:55:26.249792+0000","last_peered":"2024-04-25T18:55:26.249792+0000","last_clean":"2024-04-25T18:55:26.249792+0000","last_became_active":"2024-04-25T18:55:23.216696+0000","last_became_peered":"2024-04-25T18:55:23.216696+0000","last_unstale":"2024-04-25T18:55:26.249792+0000","last_undegraded":"2024-04-25T18:55:26.249792+0000","last_fullsized":"2024-04-25T18:55:26.249792+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:07.049349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540084+0000","last_change":"2024-04-25T18:55:19.166377+0000","last_active":"2024-04-25T18:56:17.540084+0000","last_peered":"2024-04-25T18:56:17.540084+0000","last_clean":"2024-04-25T18:56:17.540084+0000","last_became_active":"2024-04-25T18:55:19.166036+0000","last_became_peered":"2024-04-25T18:55:19.166036+0000","last_unstale":"2024-04-25T18:56:17.540084+0000","last_undegraded":"2024-04-25T18:56:17.540084+0000","last_fullsized":"2024-04-25T18:56:17.540084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:04.761919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'1","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.249028+0000","last_change":"2024-04-25T18:55:23.236893+0000","last_active":"2024-04-25T18:55:26.249028+0000","last_peered":"2024-04-25T18:55:26.249028+0000","last_clean":"2024-04-25T18:55:26.249028+0000","last_became_active":"2024-04-25T18:55:23.236488+0000","last_became_peered":"2024-04-25T18:55:23.236488+0000","last_unstale":"2024-04-25T18:55:26.249028+0000","last_undegraded":"2024-04-25T18:55:26.249028+0000","last_fullsized":"2024-04-25T18:55:26.249028+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:44.605041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.551149+0000","last_change":"2024-04-25T18:55:19.175076+0000","last_active":"2024-04-25T18:56:17.551149+0000","last_peered":"2024-04-25T18:56:17.551149+0000","last_clean":"2024-04-25T18:56:17.551149+0000","last_became_active":"2024-04-25T18:55:19.174547+0000","last_became_peered":"2024-04-25T18:55:19.174547+0000","last_unstale":"2024-04-25T18:56:17.551149+0000","last_undegraded":"2024-04-25T18:56:17.551149+0000","last_fullsized":"2024-04-25T18:56:17.551149+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:43.791068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.249675+0000","last_change":"2024-04-25T18:55:23.219665+0000","last_active":"2024-04-25T18:55:26.249675+0000","last_peered":"2024-04-25T18:55:26.249675+0000","last_clean":"2024-04-25T18:55:26.249675+0000","last_became_active":"2024-04-25T18:55:23.219264+0000","last_became_peered":"2024-04-25T18:55:23.219264+0000","last_unstale":"2024-04-25T18:55:26.249675+0000","last_undegraded":"2024-04-25T18:55:26.249675+0000","last_fullsized":"2024-04-25T18:55:26.249675+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:19.008120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.249999+0000","last_change":"2024-04-25T18:55:23.220290+0000","last_active":"2024-04-25T18:55:26.249999+0000","last_peered":"2024-04-25T18:55:26.249999+0000","last_clean":"2024-04-25T18:55:26.249999+0000","last_became_active":"2024-04-25T18:55:23.219131+0000","last_became_peered":"2024-04-25T18:55:23.219131+0000","last_unstale":"2024-04-25T18:55:26.249999+0000","last_undegraded":"2024-04-25T18:55:26.249999+0000","last_fullsized":"2024-04-25T18:55:26.249999+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:25:25.678525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536245+0000","last_change":"2024-04-25T18:55:23.216415+0000","last_active":"2024-04-25T18:56:17.536245+0000","last_peered":"2024-04-25T18:56:17.536245+0000","last_clean":"2024-04-25T18:56:17.536245+0000","last_became_active":"2024-04-25T18:55:23.216066+0000","last_became_peered":"2024-04-25T18:55:23.216066+0000","last_unstale":"2024-04-25T18:56:17.536245+0000","last_undegraded":"2024-04-25T18:56:17.536245+0000","last_fullsized":"2024-04-25T18:56:17.536245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:54.633216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536208+0000","last_change":"2024-04-25T18:55:23.234034+0000","last_active":"2024-04-25T18:56:17.536208+0000","last_peered":"2024-04-25T18:56:17.536208+0000","last_clean":"2024-04-25T18:56:17.536208+0000","last_became_active":"2024-04-25T18:55:23.233639+0000","last_became_peered":"2024-04-25T18:55:23.233639+0000","last_unstale":"2024-04-25T18:56:17.536208+0000","last_undegraded":"2024-04-25T18:56:17.536208+0000","last_fullsized":"2024-04-25T18:56:17.536208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:57.117958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.257169+0000","last_change":"2024-04-25T18:55:23.233498+0000","last_active":"2024-04-25T18:55:26.257169+0000","last_peered":"2024-04-25T18:55:26.257169+0000","last_clean":"2024-04-25T18:55:26.257169+0000","last_became_active":"2024-04-25T18:55:23.231661+0000","last_became_peered":"2024-04-25T18:55:23.231661+0000","last_unstale":"2024-04-25T18:55:26.257169+0000","last_undegraded":"2024-04-25T18:55:26.257169+0000","last_fullsized":"2024-04-25T18:55:26.257169+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:58.738426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550041+0000","last_change":"2024-04-25T18:55:23.238068+0000","last_active":"2024-04-25T18:56:17.550041+0000","last_peered":"2024-04-25T18:56:17.550041+0000","last_clean":"2024-04-25T18:56:17.550041+0000","last_became_active":"2024-04-25T18:55:23.237773+0000","last_became_peered":"2024-04-25T18:55:23.237773+0000","last_unstale":"2024-04-25T18:56:17.550041+0000","last_undegraded":"2024-04-25T18:56:17.550041+0000","last_fullsized":"2024-04-25T18:56:17.550041+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.688576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.249405+0000","last_change":"2024-04-25T18:55:23.215463+0000","last_active":"2024-04-25T18:55:26.249405+0000","last_peered":"2024-04-25T18:55:26.249405+0000","last_clean":"2024-04-25T18:55:26.249405+0000","last_became_active":"2024-04-25T18:55:23.215068+0000","last_became_peered":"2024-04-25T18:55:23.215068+0000","last_unstale":"2024-04-25T18:55:26.249405+0000","last_undegraded":"2024-04-25T18:55:26.249405+0000","last_fullsized":"2024-04-25T18:55:26.249405+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.450828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536634+0000","last_change":"2024-04-25T18:55:23.214362+0000","last_active":"2024-04-25T18:56:17.536634+0000","last_peered":"2024-04-25T18:56:17.536634+0000","last_clean":"2024-04-25T18:56:17.536634+0000","last_became_active":"2024-04-25T18:55:23.214035+0000","last_became_peered":"2024-04-25T18:55:23.214035+0000","last_unstale":"2024-04-25T18:56:17.536634+0000","last_undegraded":"2024-04-25T18:56:17.536634+0000","last_fullsized":"2024-04-25T18:56:17.536634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:33.386859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.544771+0000","last_change":"2024-04-25T18:55:23.235709+0000","last_active":"2024-04-25T18:56:16.544771+0000","last_peered":"2024-04-25T18:56:16.544771+0000","last_clean":"2024-04-25T18:56:16.544771+0000","last_became_active":"2024-04-25T18:55:23.226089+0000","last_became_peered":"2024-04-25T18:55:23.226089+0000","last_unstale":"2024-04-25T18:56:16.544771+0000","last_undegraded":"2024-04-25T18:56:16.544771+0000","last_fullsized":"2024-04-25T18:56:16.544771+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:57.161365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.563613+0000","last_change":"2024-04-25T18:56:17.561546+0000","last_active":"2024-04-25T18:56:17.563613+0000","last_peered":"2024-04-25T18:56:17.563613+0000","last_clean":"2024-04-25T18:56:17.563613+0000","last_became_active":"2024-04-25T18:56:17.559767+0000","last_became_peered":"2024-04-25T18:56:17.559767+0000","last_unstale":"2024-04-25T18:56:17.563613+0000","last_undegraded":"2024-04-25T18:56:17.563613+0000","last_fullsized":"2024-04-25T18:56:17.563613+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:59.512008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T18:56:16.539377+0000","last_change":"2024-04-25T18:55:23.227620+0000","last_active":"2024-04-25T18:56:16.539377+0000","last_peered":"2024-04-25T18:56:16.539377+0000","last_clean":"2024-04-25T18:56:16.539377+0000","last_became_active":"2024-04-25T18:55:23.227236+0000","last_became_peered":"2024-04-25T18:55:23.227236+0000","last_unstale":"2024-04-25T18:56:16.539377+0000","last_undegraded":"2024-04-25T18:56:16.539377+0000","last_fullsized":"2024-04-25T18:56:16.539377+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:32.707619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.541206+0000","last_change":"2024-04-25T18:55:23.238393+0000","last_active":"2024-04-25T18:56:17.541206+0000","last_peered":"2024-04-25T18:56:17.541206+0000","last_clean":"2024-04-25T18:56:17.541206+0000","last_became_active":"2024-04-25T18:55:23.237726+0000","last_became_peered":"2024-04-25T18:55:23.237726+0000","last_unstale":"2024-04-25T18:56:17.541206+0000","last_undegraded":"2024-04-25T18:56:17.541206+0000","last_fullsized":"2024-04-25T18:56:17.541206+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:48.543014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550620+0000","last_change":"2024-04-25T18:55:23.231374+0000","last_active":"2024-04-25T18:56:17.550620+0000","last_peered":"2024-04-25T18:56:17.550620+0000","last_clean":"2024-04-25T18:56:17.550620+0000","last_became_active":"2024-04-25T18:55:23.230976+0000","last_became_peered":"2024-04-25T18:55:23.230976+0000","last_unstale":"2024-04-25T18:56:17.550620+0000","last_undegraded":"2024-04-25T18:56:17.550620+0000","last_fullsized":"2024-04-25T18:56:17.550620+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:06.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T18:55:26.248939+0000","last_change":"2024-04-25T18:55:23.217332+0000","last_active":"2024-04-25T18:55:26.248939+0000","last_peered":"2024-04-25T18:55:26.248939+0000","last_clean":"2024-04-25T18:55:26.248939+0000","last_became_active":"2024-04-25T18:55:23.216968+0000","last_became_peered":"2024-04-25T18:55:23.216968+0000","last_unstale":"2024-04-25T18:55:26.248939+0000","last_undegraded":"2024-04-25T18:55:26.248939+0000","last_fullsized":"2024-04-25T18:55:26.248939+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:31.000083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67464,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":218,"num_read_kb":180,"num_write":167,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":202358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252891,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":481481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514530,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36992,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708160,"statfs":{"total":95995035648,"available":95957155840,"internally_reserved":0,"allocated":1695744,"data_stored":1136425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710757,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":498993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972397,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":498913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234034,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36904,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708248,"statfs":{"total":95995035648,"available":95957245952,"internally_reserved":0,"allocated":1605632,"data_stored":1073958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430262,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":498382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1097728,"data_stored":561247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1019904,"data_stored":482377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":562954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117063,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":481417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855425,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691901,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1650688,"data_stored":1090527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:56:22.266 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:56:22.754 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:22.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:22 smithi079 ceph-mon[33002]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:56:22.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:22 smithi045 ceph-mon[27776]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:56:23.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:22 smithi062 ceph-mon[32844]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:56:23.916 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:23 smithi045 ceph-mon[27776]: pgmap v357: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:56:23.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:23 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:23.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:23 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:23.917 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:23 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:23.933 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:23 smithi079 ceph-mon[33002]: pgmap v357: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:56:23.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:23 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:23.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:23 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:23.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:23 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:24.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:23 smithi062 ceph-mon[32844]: pgmap v357: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 1.4 KiB/s wr, 8 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:56:24.193 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:23 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:56:24.193 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:23 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:56:24.193 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:23 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:56:24.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.888+0000 7fa44e800700 1 -- 172.21.15.45:0/2587109509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448104ee0 msgr2=0x7fa4481052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:24.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.888+0000 7fa44e800700 1 --2- 172.21.15.45:0/2587109509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448104ee0 0x7fa4481052c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa438009990 tx=0x7fa438009ca0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.888+0000 7fa44e800700 1 -- 172.21.15.45:0/2587109509 shutdown_connections 2024-04-25T18:56:24.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.888+0000 7fa44e800700 1 --2- 172.21.15.45:0/2587109509 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448104ee0 0x7fa4481052c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.888+0000 7fa44e800700 1 --2- 172.21.15.45:0/2587109509 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa448109fc0 0x7fa448111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.888+0000 7fa44e800700 1 --2- 172.21.15.45:0/2587109509 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa448105890 0x7fa4481098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.888+0000 7fa44e800700 1 -- 172.21.15.45:0/2587109509 >> 172.21.15.45:0/2587109509 conn(0x7fa448100770 msgr2=0x7fa448102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:24.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.888+0000 7fa44e800700 1 -- 172.21.15.45:0/2587109509 shutdown_connections 2024-04-25T18:56:24.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.888+0000 7fa44e800700 1 -- 172.21.15.45:0/2587109509 wait complete. 2024-04-25T18:56:24.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.889+0000 7fa44e800700 1 Processor -- start 2024-04-25T18:56:24.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.889+0000 7fa44e800700 1 -- start start 2024-04-25T18:56:24.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa44e800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa448104ee0 0x7fa44819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa44e800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448105890 0x7fa44819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa44e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa448109fc0 0x7fa44819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa44e800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa448114e70 con 0x7fa448104ee0 2024-04-25T18:56:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa44e800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa448114cf0 con 0x7fa448105890 2024-04-25T18:56:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa44e800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa448114ff0 con 0x7fa448109fc0 2024-04-25T18:56:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa447400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa448104ee0 0x7fa44819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa447400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa448104ee0 0x7fa44819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48190/0 (socket says 172.21.15.45:48190) 2024-04-25T18:56:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa446a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448105890 0x7fa44819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa447400700 1 -- 172.21.15.45:0/3367570657 learned_addr learned my addr 172.21.15.45:0/3367570657 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:24.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.890+0000 7fa447e00700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa448109fc0 0x7fa44819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.891+0000 7fa446a00700 1 -- 172.21.15.45:0/3367570657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa448109fc0 msgr2=0x7fa44819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.891+0000 7fa446a00700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa448109fc0 0x7fa44819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.891+0000 7fa446a00700 1 -- 172.21.15.45:0/3367570657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa448104ee0 msgr2=0x7fa44819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.891+0000 7fa446a00700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa448104ee0 0x7fa44819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:24.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.891+0000 7fa446a00700 1 -- 172.21.15.45:0/3367570657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4481a0330 con 0x7fa448105890 2024-04-25T18:56:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.891+0000 7fa447e00700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa448109fc0 0x7fa44819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.891+0000 7fa446a00700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448105890 0x7fa44819b860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa43400e9c0 tx=0x7fa43400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.892+0000 7fa43fe00700 1 -- 172.21.15.45:0/3367570657 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43400c9c0 con 0x7fa448105890 2024-04-25T18:56:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.892+0000 7fa43fe00700 1 -- 172.21.15.45:0/3367570657 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa43400cb60 con 0x7fa448105890 2024-04-25T18:56:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.892+0000 7fa43fe00700 1 -- 172.21.15.45:0/3367570657 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434018680 con 0x7fa448105890 2024-04-25T18:56:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.892+0000 7fa44e800700 1 -- 172.21.15.45:0/3367570657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4481a0620 con 0x7fa448105890 2024-04-25T18:56:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.892+0000 7fa44e800700 1 -- 172.21.15.45:0/3367570657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4481a7ea0 con 0x7fa448105890 2024-04-25T18:56:24.895 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.893+0000 7fa44e800700 1 -- 172.21.15.45:0/3367570657 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa448069730 con 0x7fa448105890 2024-04-25T18:56:24.897 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.895+0000 7fa43fe00700 1 -- 172.21.15.45:0/3367570657 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa434010430 con 0x7fa448105890 2024-04-25T18:56:24.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.896+0000 7fa43fe00700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa42c0a3d50 0x7fa42c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:24.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.896+0000 7fa43fe00700 1 -- 172.21.15.45:0/3367570657 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa4340d06e0 con 0x7fa448105890 2024-04-25T18:56:24.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.897+0000 7fa447400700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa42c0a3d50 0x7fa42c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:24.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.897+0000 7fa447400700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa42c0a3d50 0x7fa42c0a6210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa43800b2e0 tx=0x7fa43800b610 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:24.903 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:24.901+0000 7fa43fe00700 1 -- 172.21.15.45:0/3367570657 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa43409cee0 con 0x7fa448105890 2024-04-25T18:56:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.131+0000 7fa44e800700 1 -- 172.21.15.45:0/3367570657 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa448062e80 con 0x7fa42c0a3d50 2024-04-25T18:56:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.142+0000 7fa43fe00700 1 -- 172.21.15.45:0/3367570657 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258788 (secure 0 0 0) 0x7fa438067300 con 0x7fa42c0a3d50 2024-04-25T18:56:25.146 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:56:25.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.149+0000 7fa44e800700 1 -- 172.21.15.45:0/3367570657 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa42c0a3d50 msgr2=0x7fa42c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.149+0000 7fa44e800700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa42c0a3d50 0x7fa42c0a6210 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa43800b2e0 tx=0x7fa43800b610 comp rx=0 tx=0).stop 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.149+0000 7fa44e800700 1 -- 172.21.15.45:0/3367570657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448105890 msgr2=0x7fa44819b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.149+0000 7fa44e800700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448105890 0x7fa44819b860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa43400e9c0 tx=0x7fa43400ecd0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.150+0000 7fa44e800700 1 -- 172.21.15.45:0/3367570657 shutdown_connections 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.150+0000 7fa44e800700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa42c0a3d50 0x7fa42c0a6210 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.150+0000 7fa44e800700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa448109fc0 0x7fa44819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.150+0000 7fa44e800700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa448104ee0 0x7fa44819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.150+0000 7fa44e800700 1 --2- 172.21.15.45:0/3367570657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa448105890 0x7fa44819b860 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.150+0000 7fa44e800700 1 -- 172.21.15.45:0/3367570657 >> 172.21.15.45:0/3367570657 conn(0x7fa448100770 msgr2=0x7fa448102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.150+0000 7fa44e800700 1 -- 172.21.15.45:0/3367570657 shutdown_connections 2024-04-25T18:56:25.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:25.150+0000 7fa44e800700 1 -- 172.21.15.45:0/3367570657 wait complete. 2024-04-25T18:56:25.154 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T18:56:25.801 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:25 smithi045 ceph-mon[27776]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:56:25.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:25 smithi079 ceph-mon[33002]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:56:26.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:25 smithi062 ceph-mon[32844]: pgmap v358: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.4 KiB/s wr, 6 op/s; 4 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:56:26.206 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-04-25T18:56:24.264836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658066,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":374,"num_read_kb":309,"num_write":350,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392536,"kb_used_data":14184,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549288,"statfs":{"total":1151940427776,"available":1151538470912,"internally_reserved":0,"allocated":14524416,"data_stored":7952298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8705,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":46,"num_read_kb":34,"num_write":6,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003149"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.538363+0000","last_change":"2024-04-25T18:55:23.241030+0000","last_active":"2024-04-25T18:56:17.538363+0000","last_peered":"2024-04-25T18:56:17.538363+0000","last_clean":"2024-04-25T18:56:17.538363+0000","last_became_active":"2024-04-25T18:55:23.240724+0000","last_became_peered":"2024-04-25T18:55:23.240724+0000","last_unstale":"2024-04-25T18:56:17.538363+0000","last_undegraded":"2024-04-25T18:56:17.538363+0000","last_fullsized":"2024-04-25T18:56:17.538363+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:37.995475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.545865+0000","last_change":"2024-04-25T18:55:23.229875+0000","last_active":"2024-04-25T18:56:17.545865+0000","last_peered":"2024-04-25T18:56:17.545865+0000","last_clean":"2024-04-25T18:56:17.545865+0000","last_became_active":"2024-04-25T18:55:23.229640+0000","last_became_peered":"2024-04-25T18:55:23.229640+0000","last_unstale":"2024-04-25T18:56:17.545865+0000","last_undegraded":"2024-04-25T18:56:17.545865+0000","last_fullsized":"2024-04-25T18:56:17.545865+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:10.262376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.545315+0000","last_change":"2024-04-25T18:55:23.229821+0000","last_active":"2024-04-25T18:56:17.545315+0000","last_peered":"2024-04-25T18:56:17.545315+0000","last_clean":"2024-04-25T18:56:17.545315+0000","last_became_active":"2024-04-25T18:55:23.229414+0000","last_became_peered":"2024-04-25T18:55:23.229414+0000","last_unstale":"2024-04-25T18:56:17.545315+0000","last_undegraded":"2024-04-25T18:56:17.545315+0000","last_fullsized":"2024-04-25T18:56:17.545315+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:56.063786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.562827+0000","last_change":"2024-04-25T18:56:17.560167+0000","last_active":"2024-04-25T18:56:17.562827+0000","last_peered":"2024-04-25T18:56:17.562827+0000","last_clean":"2024-04-25T18:56:17.562827+0000","last_became_active":"2024-04-25T18:56:17.558466+0000","last_became_peered":"2024-04-25T18:56:17.558466+0000","last_unstale":"2024-04-25T18:56:17.562827+0000","last_undegraded":"2024-04-25T18:56:17.562827+0000","last_fullsized":"2024-04-25T18:56:17.562827+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:22.371076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540378+0000","last_change":"2024-04-25T18:55:23.233512+0000","last_active":"2024-04-25T18:56:17.540378+0000","last_peered":"2024-04-25T18:56:17.540378+0000","last_clean":"2024-04-25T18:56:17.540378+0000","last_became_active":"2024-04-25T18:55:23.233158+0000","last_became_peered":"2024-04-25T18:55:23.233158+0000","last_unstale":"2024-04-25T18:56:17.540378+0000","last_undegraded":"2024-04-25T18:56:17.540378+0000","last_fullsized":"2024-04-25T18:56:17.540378+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:44.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540548+0000","last_change":"2024-04-25T18:55:23.225921+0000","last_active":"2024-04-25T18:56:17.540548+0000","last_peered":"2024-04-25T18:56:17.540548+0000","last_clean":"2024-04-25T18:56:17.540548+0000","last_became_active":"2024-04-25T18:55:23.225547+0000","last_became_peered":"2024-04-25T18:55:23.225547+0000","last_unstale":"2024-04-25T18:56:17.540548+0000","last_undegraded":"2024-04-25T18:56:17.540548+0000","last_fullsized":"2024-04-25T18:56:17.540548+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:29.828394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583149+0000","last_change":"2024-04-25T18:55:23.217111+0000","last_active":"2024-04-25T18:56:17.583149+0000","last_peered":"2024-04-25T18:56:17.583149+0000","last_clean":"2024-04-25T18:56:17.583149+0000","last_became_active":"2024-04-25T18:55:23.216731+0000","last_became_peered":"2024-04-25T18:55:23.216731+0000","last_unstale":"2024-04-25T18:56:17.583149+0000","last_undegraded":"2024-04-25T18:56:17.583149+0000","last_fullsized":"2024-04-25T18:56:17.583149+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:17.931928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.558980+0000","last_change":"2024-04-25T18:56:17.555014+0000","last_active":"2024-04-25T18:56:17.558980+0000","last_peered":"2024-04-25T18:56:17.558980+0000","last_clean":"2024-04-25T18:56:17.558980+0000","last_became_active":"2024-04-25T18:56:17.553863+0000","last_became_peered":"2024-04-25T18:56:17.553863+0000","last_unstale":"2024-04-25T18:56:17.558980+0000","last_undegraded":"2024-04-25T18:56:17.558980+0000","last_fullsized":"2024-04-25T18:56:17.558980+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.240754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.541075+0000","last_change":"2024-04-25T18:55:23.218197+0000","last_active":"2024-04-25T18:56:17.541075+0000","last_peered":"2024-04-25T18:56:17.541075+0000","last_clean":"2024-04-25T18:56:17.541075+0000","last_became_active":"2024-04-25T18:55:23.217665+0000","last_became_peered":"2024-04-25T18:55:23.217665+0000","last_unstale":"2024-04-25T18:56:17.541075+0000","last_undegraded":"2024-04-25T18:56:17.541075+0000","last_fullsized":"2024-04-25T18:56:17.541075+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:11.829991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540782+0000","last_change":"2024-04-25T18:55:23.221533+0000","last_active":"2024-04-25T18:56:17.540782+0000","last_peered":"2024-04-25T18:56:17.540782+0000","last_clean":"2024-04-25T18:56:17.540782+0000","last_became_active":"2024-04-25T18:55:23.220711+0000","last_became_peered":"2024-04-25T18:55:23.220711+0000","last_unstale":"2024-04-25T18:56:17.540782+0000","last_undegraded":"2024-04-25T18:56:17.540782+0000","last_fullsized":"2024-04-25T18:56:17.540782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:45.185166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583546+0000","last_change":"2024-04-25T18:55:23.221329+0000","last_active":"2024-04-25T18:56:17.583546+0000","last_peered":"2024-04-25T18:56:17.583546+0000","last_clean":"2024-04-25T18:56:17.583546+0000","last_became_active":"2024-04-25T18:55:23.220156+0000","last_became_peered":"2024-04-25T18:55:23.220156+0000","last_unstale":"2024-04-25T18:56:17.583546+0000","last_undegraded":"2024-04-25T18:56:17.583546+0000","last_fullsized":"2024-04-25T18:56:17.583546+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:46.981199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.562773+0000","last_change":"2024-04-25T18:56:17.558018+0000","last_active":"2024-04-25T18:56:17.562773+0000","last_peered":"2024-04-25T18:56:17.562773+0000","last_clean":"2024-04-25T18:56:17.562773+0000","last_became_active":"2024-04-25T18:56:17.556229+0000","last_became_peered":"2024-04-25T18:56:17.556229+0000","last_unstale":"2024-04-25T18:56:17.562773+0000","last_undegraded":"2024-04-25T18:56:17.562773+0000","last_fullsized":"2024-04-25T18:56:17.562773+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:50.949895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550463+0000","last_change":"2024-04-25T18:55:23.227474+0000","last_active":"2024-04-25T18:56:17.550463+0000","last_peered":"2024-04-25T18:56:17.550463+0000","last_clean":"2024-04-25T18:56:17.550463+0000","last_became_active":"2024-04-25T18:55:23.227068+0000","last_became_peered":"2024-04-25T18:55:23.227068+0000","last_unstale":"2024-04-25T18:56:17.550463+0000","last_undegraded":"2024-04-25T18:56:17.550463+0000","last_fullsized":"2024-04-25T18:56:17.550463+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:13.884833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.542977+0000","last_change":"2024-04-25T18:55:23.230572+0000","last_active":"2024-04-25T18:56:17.542977+0000","last_peered":"2024-04-25T18:56:17.542977+0000","last_clean":"2024-04-25T18:56:17.542977+0000","last_became_active":"2024-04-25T18:55:23.229756+0000","last_became_peered":"2024-04-25T18:55:23.229756+0000","last_unstale":"2024-04-25T18:56:17.542977+0000","last_undegraded":"2024-04-25T18:56:17.542977+0000","last_fullsized":"2024-04-25T18:56:17.542977+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:15.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.543099+0000","last_change":"2024-04-25T18:55:23.227913+0000","last_active":"2024-04-25T18:56:17.543099+0000","last_peered":"2024-04-25T18:56:17.543099+0000","last_clean":"2024-04-25T18:56:17.543099+0000","last_became_active":"2024-04-25T18:55:23.227598+0000","last_became_peered":"2024-04-25T18:55:23.227598+0000","last_unstale":"2024-04-25T18:56:17.543099+0000","last_undegraded":"2024-04-25T18:56:17.543099+0000","last_fullsized":"2024-04-25T18:56:17.543099+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:33.555741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540636+0000","last_change":"2024-04-25T18:55:23.230241+0000","last_active":"2024-04-25T18:56:17.540636+0000","last_peered":"2024-04-25T18:56:17.540636+0000","last_clean":"2024-04-25T18:56:17.540636+0000","last_became_active":"2024-04-25T18:55:23.229898+0000","last_became_peered":"2024-04-25T18:55:23.229898+0000","last_unstale":"2024-04-25T18:56:17.540636+0000","last_undegraded":"2024-04-25T18:56:17.540636+0000","last_fullsized":"2024-04-25T18:56:17.540636+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:27.278475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.538443+0000","last_change":"2024-04-25T18:55:23.240978+0000","last_active":"2024-04-25T18:56:17.538443+0000","last_peered":"2024-04-25T18:56:17.538443+0000","last_clean":"2024-04-25T18:56:17.538443+0000","last_became_active":"2024-04-25T18:55:23.240702+0000","last_became_peered":"2024-04-25T18:55:23.240702+0000","last_unstale":"2024-04-25T18:56:17.538443+0000","last_undegraded":"2024-04-25T18:56:17.538443+0000","last_fullsized":"2024-04-25T18:56:17.538443+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:06.053688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.547162+0000","last_change":"2024-04-25T18:55:23.232116+0000","last_active":"2024-04-25T18:56:17.547162+0000","last_peered":"2024-04-25T18:56:17.547162+0000","last_clean":"2024-04-25T18:56:17.547162+0000","last_became_active":"2024-04-25T18:55:23.231586+0000","last_became_peered":"2024-04-25T18:55:23.231586+0000","last_unstale":"2024-04-25T18:56:17.547162+0000","last_undegraded":"2024-04-25T18:56:17.547162+0000","last_fullsized":"2024-04-25T18:56:17.547162+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:32.640823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536773+0000","last_change":"2024-04-25T18:55:23.224519+0000","last_active":"2024-04-25T18:56:17.536773+0000","last_peered":"2024-04-25T18:56:17.536773+0000","last_clean":"2024-04-25T18:56:17.536773+0000","last_became_active":"2024-04-25T18:55:23.224145+0000","last_became_peered":"2024-04-25T18:55:23.224145+0000","last_unstale":"2024-04-25T18:56:17.536773+0000","last_undegraded":"2024-04-25T18:56:17.536773+0000","last_fullsized":"2024-04-25T18:56:17.536773+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:03.340162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.548353+0000","last_change":"2024-04-25T18:55:19.164977+0000","last_active":"2024-04-25T18:56:17.548353+0000","last_peered":"2024-04-25T18:56:17.548353+0000","last_clean":"2024-04-25T18:56:17.548353+0000","last_became_active":"2024-04-25T18:55:19.164316+0000","last_became_peered":"2024-04-25T18:55:19.164316+0000","last_unstale":"2024-04-25T18:56:17.548353+0000","last_undegraded":"2024-04-25T18:56:17.548353+0000","last_fullsized":"2024-04-25T18:56:17.548353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:08.031680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.559746+0000","last_change":"2024-04-25T18:56:17.559746+0000","last_active":"2024-04-25T18:56:17.559746+0000","last_peered":"2024-04-25T18:56:17.559746+0000","last_clean":"2024-04-25T18:56:17.559746+0000","last_became_active":"2024-04-25T18:56:17.558761+0000","last_became_peered":"2024-04-25T18:56:17.558761+0000","last_unstale":"2024-04-25T18:56:17.559746+0000","last_undegraded":"2024-04-25T18:56:17.559746+0000","last_fullsized":"2024-04-25T18:56:17.559746+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:43.518760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.547448+0000","last_change":"2024-04-25T18:55:19.179943+0000","last_active":"2024-04-25T18:56:17.547448+0000","last_peered":"2024-04-25T18:56:17.547448+0000","last_clean":"2024-04-25T18:56:17.547448+0000","last_became_active":"2024-04-25T18:55:19.179519+0000","last_became_peered":"2024-04-25T18:55:19.179519+0000","last_unstale":"2024-04-25T18:56:17.547448+0000","last_undegraded":"2024-04-25T18:56:17.547448+0000","last_fullsized":"2024-04-25T18:56:17.547448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:46.365187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.545013+0000","last_change":"2024-04-25T18:55:23.237965+0000","last_active":"2024-04-25T18:56:17.545013+0000","last_peered":"2024-04-25T18:56:17.545013+0000","last_clean":"2024-04-25T18:56:17.545013+0000","last_became_active":"2024-04-25T18:55:23.237520+0000","last_became_peered":"2024-04-25T18:55:23.237520+0000","last_unstale":"2024-04-25T18:56:17.545013+0000","last_undegraded":"2024-04-25T18:56:17.545013+0000","last_fullsized":"2024-04-25T18:56:17.545013+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:03.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540660+0000","last_change":"2024-04-25T18:55:19.163965+0000","last_active":"2024-04-25T18:56:17.540660+0000","last_peered":"2024-04-25T18:56:17.540660+0000","last_clean":"2024-04-25T18:56:17.540660+0000","last_became_active":"2024-04-25T18:55:19.163390+0000","last_became_peered":"2024-04-25T18:55:19.163390+0000","last_unstale":"2024-04-25T18:56:17.540660+0000","last_undegraded":"2024-04-25T18:56:17.540660+0000","last_fullsized":"2024-04-25T18:56:17.540660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:54.183730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.541392+0000","last_change":"2024-04-25T18:55:23.227124+0000","last_active":"2024-04-25T18:56:17.541392+0000","last_peered":"2024-04-25T18:56:17.541392+0000","last_clean":"2024-04-25T18:56:17.541392+0000","last_became_active":"2024-04-25T18:55:23.226383+0000","last_became_peered":"2024-04-25T18:55:23.226383+0000","last_unstale":"2024-04-25T18:56:17.541392+0000","last_undegraded":"2024-04-25T18:56:17.541392+0000","last_fullsized":"2024-04-25T18:56:17.541392+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:22.011212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.544679+0000","last_change":"2024-04-25T18:56:17.544303+0000","last_active":"2024-04-25T18:56:17.544679+0000","last_peered":"2024-04-25T18:56:17.544679+0000","last_clean":"2024-04-25T18:56:17.544679+0000","last_became_active":"2024-04-25T18:56:17.542002+0000","last_became_peered":"2024-04-25T18:56:17.542002+0000","last_unstale":"2024-04-25T18:56:17.544679+0000","last_undegraded":"2024-04-25T18:56:17.544679+0000","last_fullsized":"2024-04-25T18:56:17.544679+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:23.848398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540179+0000","last_change":"2024-04-25T18:55:23.217703+0000","last_active":"2024-04-25T18:56:17.540179+0000","last_peered":"2024-04-25T18:56:17.540179+0000","last_clean":"2024-04-25T18:56:17.540179+0000","last_became_active":"2024-04-25T18:55:23.216767+0000","last_became_peered":"2024-04-25T18:55:23.216767+0000","last_unstale":"2024-04-25T18:56:17.540179+0000","last_undegraded":"2024-04-25T18:56:17.540179+0000","last_fullsized":"2024-04-25T18:56:17.540179+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:17.779356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"92'6","reported_seq":69,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:18.467032+0000","last_change":"2024-04-25T18:55:19.164199+0000","last_active":"2024-04-25T18:56:18.467032+0000","last_peered":"2024-04-25T18:56:18.467032+0000","last_clean":"2024-04-25T18:56:18.467032+0000","last_became_active":"2024-04-25T18:55:19.163849+0000","last_became_peered":"2024-04-25T18:55:19.163849+0000","last_unstale":"2024-04-25T18:56:18.467032+0000","last_undegraded":"2024-04-25T18:56:18.467032+0000","last_fullsized":"2024-04-25T18:56:18.467032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:25.234229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540648+0000","last_change":"2024-04-25T18:55:23.216916+0000","last_active":"2024-04-25T18:56:17.540648+0000","last_peered":"2024-04-25T18:56:17.540648+0000","last_clean":"2024-04-25T18:56:17.540648+0000","last_became_active":"2024-04-25T18:55:23.216249+0000","last_became_peered":"2024-04-25T18:55:23.216249+0000","last_unstale":"2024-04-25T18:56:17.540648+0000","last_undegraded":"2024-04-25T18:56:17.540648+0000","last_fullsized":"2024-04-25T18:56:17.540648+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:21.522318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540439+0000","last_change":"2024-04-25T18:55:19.171004+0000","last_active":"2024-04-25T18:56:17.540439+0000","last_peered":"2024-04-25T18:56:17.540439+0000","last_clean":"2024-04-25T18:56:17.540439+0000","last_became_active":"2024-04-25T18:55:19.170570+0000","last_became_peered":"2024-04-25T18:55:19.170570+0000","last_unstale":"2024-04-25T18:56:17.540439+0000","last_undegraded":"2024-04-25T18:56:17.540439+0000","last_fullsized":"2024-04-25T18:56:17.540439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:15:24.954033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.552615+0000","last_change":"2024-04-25T18:56:17.552615+0000","last_active":"2024-04-25T18:56:17.552615+0000","last_peered":"2024-04-25T18:56:17.552615+0000","last_clean":"2024-04-25T18:56:17.552615+0000","last_became_active":"2024-04-25T18:56:17.552154+0000","last_became_peered":"2024-04-25T18:56:17.552154+0000","last_unstale":"2024-04-25T18:56:17.552615+0000","last_undegraded":"2024-04-25T18:56:17.552615+0000","last_fullsized":"2024-04-25T18:56:17.552615+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:01.199859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.549445+0000","last_change":"2024-04-25T18:55:19.173398+0000","last_active":"2024-04-25T18:56:17.549445+0000","last_peered":"2024-04-25T18:56:17.549445+0000","last_clean":"2024-04-25T18:56:17.549445+0000","last_became_active":"2024-04-25T18:55:19.171477+0000","last_became_peered":"2024-04-25T18:55:19.171477+0000","last_unstale":"2024-04-25T18:56:17.549445+0000","last_undegraded":"2024-04-25T18:56:17.549445+0000","last_fullsized":"2024-04-25T18:56:17.549445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:35.230596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.543019+0000","last_change":"2024-04-25T18:55:23.235209+0000","last_active":"2024-04-25T18:56:17.543019+0000","last_peered":"2024-04-25T18:56:17.543019+0000","last_clean":"2024-04-25T18:56:17.543019+0000","last_became_active":"2024-04-25T18:55:23.231898+0000","last_became_peered":"2024-04-25T18:55:23.231898+0000","last_unstale":"2024-04-25T18:56:17.543019+0000","last_undegraded":"2024-04-25T18:56:17.543019+0000","last_fullsized":"2024-04-25T18:56:17.543019+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.521282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539932+0000","last_change":"2024-04-25T18:55:19.163780+0000","last_active":"2024-04-25T18:56:17.539932+0000","last_peered":"2024-04-25T18:56:17.539932+0000","last_clean":"2024-04-25T18:56:17.539932+0000","last_became_active":"2024-04-25T18:55:19.163197+0000","last_became_peered":"2024-04-25T18:55:19.163197+0000","last_unstale":"2024-04-25T18:56:17.539932+0000","last_undegraded":"2024-04-25T18:56:17.539932+0000","last_fullsized":"2024-04-25T18:56:17.539932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:02:26.519160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.556804+0000","last_change":"2024-04-25T18:56:17.556804+0000","last_active":"2024-04-25T18:56:17.556804+0000","last_peered":"2024-04-25T18:56:17.556804+0000","last_clean":"2024-04-25T18:56:17.556804+0000","last_became_active":"2024-04-25T18:56:17.554885+0000","last_became_peered":"2024-04-25T18:56:17.554885+0000","last_unstale":"2024-04-25T18:56:17.556804+0000","last_undegraded":"2024-04-25T18:56:17.556804+0000","last_fullsized":"2024-04-25T18:56:17.556804+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:36.316156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"92'3","reported_seq":65,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:20.469171+0000","last_change":"2024-04-25T18:55:19.170565+0000","last_active":"2024-04-25T18:56:20.469171+0000","last_peered":"2024-04-25T18:56:20.469171+0000","last_clean":"2024-04-25T18:56:20.469171+0000","last_became_active":"2024-04-25T18:55:19.170162+0000","last_became_peered":"2024-04-25T18:55:19.170162+0000","last_unstale":"2024-04-25T18:56:20.469171+0000","last_undegraded":"2024-04-25T18:56:20.469171+0000","last_fullsized":"2024-04-25T18:56:20.469171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:36.606661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.541196+0000","last_change":"2024-04-25T18:55:23.227436+0000","last_active":"2024-04-25T18:56:17.541196+0000","last_peered":"2024-04-25T18:56:17.541196+0000","last_clean":"2024-04-25T18:56:17.541196+0000","last_became_active":"2024-04-25T18:55:23.227044+0000","last_became_peered":"2024-04-25T18:55:23.227044+0000","last_unstale":"2024-04-25T18:56:17.541196+0000","last_undegraded":"2024-04-25T18:56:17.541196+0000","last_fullsized":"2024-04-25T18:56:17.541196+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:34.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583525+0000","last_change":"2024-04-25T18:55:19.167514+0000","last_active":"2024-04-25T18:56:17.583525+0000","last_peered":"2024-04-25T18:56:17.583525+0000","last_clean":"2024-04-25T18:56:17.583525+0000","last_became_active":"2024-04-25T18:55:19.167102+0000","last_became_peered":"2024-04-25T18:55:19.167102+0000","last_unstale":"2024-04-25T18:56:17.583525+0000","last_undegraded":"2024-04-25T18:56:17.583525+0000","last_fullsized":"2024-04-25T18:56:17.583525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:41.472274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.544876+0000","last_change":"2024-04-25T18:55:23.237012+0000","last_active":"2024-04-25T18:56:17.544876+0000","last_peered":"2024-04-25T18:56:17.544876+0000","last_clean":"2024-04-25T18:56:17.544876+0000","last_became_active":"2024-04-25T18:55:23.230263+0000","last_became_peered":"2024-04-25T18:55:23.230263+0000","last_unstale":"2024-04-25T18:56:17.544876+0000","last_undegraded":"2024-04-25T18:56:17.544876+0000","last_fullsized":"2024-04-25T18:56:17.544876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:30.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540250+0000","last_change":"2024-04-25T18:55:19.179555+0000","last_active":"2024-04-25T18:56:17.540250+0000","last_peered":"2024-04-25T18:56:17.540250+0000","last_clean":"2024-04-25T18:56:17.540250+0000","last_became_active":"2024-04-25T18:55:19.179190+0000","last_became_peered":"2024-04-25T18:55:19.179190+0000","last_unstale":"2024-04-25T18:56:17.540250+0000","last_undegraded":"2024-04-25T18:56:17.540250+0000","last_fullsized":"2024-04-25T18:56:17.540250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:57.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.541440+0000","last_change":"2024-04-25T18:55:23.219978+0000","last_active":"2024-04-25T18:56:17.541440+0000","last_peered":"2024-04-25T18:56:17.541440+0000","last_clean":"2024-04-25T18:56:17.541440+0000","last_became_active":"2024-04-25T18:55:23.219630+0000","last_became_peered":"2024-04-25T18:55:23.219630+0000","last_unstale":"2024-04-25T18:56:17.541440+0000","last_undegraded":"2024-04-25T18:56:17.541440+0000","last_fullsized":"2024-04-25T18:56:17.541440+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:31.294500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'30","reported_seq":137,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:21.475029+0000","last_change":"2024-04-25T18:55:19.164042+0000","last_active":"2024-04-25T18:56:21.475029+0000","last_peered":"2024-04-25T18:56:21.475029+0000","last_clean":"2024-04-25T18:56:21.475029+0000","last_became_active":"2024-04-25T18:55:19.163604+0000","last_became_peered":"2024-04-25T18:55:19.163604+0000","last_unstale":"2024-04-25T18:56:21.475029+0000","last_undegraded":"2024-04-25T18:56:21.475029+0000","last_fullsized":"2024-04-25T18:56:21.475029+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:55.030715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63072,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":81,"num_read_kb":101,"num_write":31,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540306+0000","last_change":"2024-04-25T18:55:23.222702+0000","last_active":"2024-04-25T18:56:17.540306+0000","last_peered":"2024-04-25T18:56:17.540306+0000","last_clean":"2024-04-25T18:56:17.540306+0000","last_became_active":"2024-04-25T18:55:23.221849+0000","last_became_peered":"2024-04-25T18:55:23.221849+0000","last_unstale":"2024-04-25T18:56:17.540306+0000","last_undegraded":"2024-04-25T18:56:17.540306+0000","last_fullsized":"2024-04-25T18:56:17.540306+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.957027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.562287+0000","last_change":"2024-04-25T18:56:17.561912+0000","last_active":"2024-04-25T18:56:17.562287+0000","last_peered":"2024-04-25T18:56:17.562287+0000","last_clean":"2024-04-25T18:56:17.562287+0000","last_became_active":"2024-04-25T18:56:17.552120+0000","last_became_peered":"2024-04-25T18:56:17.552120+0000","last_unstale":"2024-04-25T18:56:17.562287+0000","last_undegraded":"2024-04-25T18:56:17.562287+0000","last_fullsized":"2024-04-25T18:56:17.562287+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:30.209802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550782+0000","last_change":"2024-04-25T18:55:23.233074+0000","last_active":"2024-04-25T18:56:17.550782+0000","last_peered":"2024-04-25T18:56:17.550782+0000","last_clean":"2024-04-25T18:56:17.550782+0000","last_became_active":"2024-04-25T18:55:23.232751+0000","last_became_peered":"2024-04-25T18:55:23.232751+0000","last_unstale":"2024-04-25T18:56:17.550782+0000","last_undegraded":"2024-04-25T18:56:17.550782+0000","last_fullsized":"2024-04-25T18:56:17.550782+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.584929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539986+0000","last_change":"2024-04-25T18:55:19.166093+0000","last_active":"2024-04-25T18:56:17.539986+0000","last_peered":"2024-04-25T18:56:17.539986+0000","last_clean":"2024-04-25T18:56:17.539986+0000","last_became_active":"2024-04-25T18:55:19.165715+0000","last_became_peered":"2024-04-25T18:55:19.165715+0000","last_unstale":"2024-04-25T18:56:17.539986+0000","last_undegraded":"2024-04-25T18:56:17.539986+0000","last_fullsized":"2024-04-25T18:56:17.539986+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:32.177943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583380+0000","last_change":"2024-04-25T18:55:23.224617+0000","last_active":"2024-04-25T18:56:17.583380+0000","last_peered":"2024-04-25T18:56:17.583380+0000","last_clean":"2024-04-25T18:56:17.583380+0000","last_became_active":"2024-04-25T18:55:23.224269+0000","last_became_peered":"2024-04-25T18:55:23.224269+0000","last_unstale":"2024-04-25T18:56:17.583380+0000","last_undegraded":"2024-04-25T18:56:17.583380+0000","last_fullsized":"2024-04-25T18:56:17.583380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:19.079910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"92'4","reported_seq":67,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:18.489938+0000","last_change":"2024-04-25T18:55:19.164971+0000","last_active":"2024-04-25T18:56:18.489938+0000","last_peered":"2024-04-25T18:56:18.489938+0000","last_clean":"2024-04-25T18:56:18.489938+0000","last_became_active":"2024-04-25T18:55:19.164579+0000","last_became_peered":"2024-04-25T18:55:19.164579+0000","last_unstale":"2024-04-25T18:56:18.489938+0000","last_undegraded":"2024-04-25T18:56:18.489938+0000","last_fullsized":"2024-04-25T18:56:18.489938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:37.419379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583097+0000","last_change":"2024-04-25T18:55:23.222160+0000","last_active":"2024-04-25T18:56:17.583097+0000","last_peered":"2024-04-25T18:56:17.583097+0000","last_clean":"2024-04-25T18:56:17.583097+0000","last_became_active":"2024-04-25T18:55:23.221812+0000","last_became_peered":"2024-04-25T18:55:23.221812+0000","last_unstale":"2024-04-25T18:56:17.583097+0000","last_undegraded":"2024-04-25T18:56:17.583097+0000","last_fullsized":"2024-04-25T18:56:17.583097+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:53:53.388764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540070+0000","last_change":"2024-04-25T18:55:19.179970+0000","last_active":"2024-04-25T18:56:17.540070+0000","last_peered":"2024-04-25T18:56:17.540070+0000","last_clean":"2024-04-25T18:56:17.540070+0000","last_became_active":"2024-04-25T18:55:19.179545+0000","last_became_peered":"2024-04-25T18:55:19.179545+0000","last_unstale":"2024-04-25T18:56:17.540070+0000","last_undegraded":"2024-04-25T18:56:17.540070+0000","last_fullsized":"2024-04-25T18:56:17.540070+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:42.841661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539929+0000","last_change":"2024-04-25T18:55:23.216134+0000","last_active":"2024-04-25T18:56:17.539929+0000","last_peered":"2024-04-25T18:56:17.539929+0000","last_clean":"2024-04-25T18:56:17.539929+0000","last_became_active":"2024-04-25T18:55:23.215535+0000","last_became_peered":"2024-04-25T18:55:23.215535+0000","last_unstale":"2024-04-25T18:56:17.539929+0000","last_undegraded":"2024-04-25T18:56:17.539929+0000","last_fullsized":"2024-04-25T18:56:17.539929+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:08.042095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540676+0000","last_change":"2024-04-25T18:55:19.162437+0000","last_active":"2024-04-25T18:56:17.540676+0000","last_peered":"2024-04-25T18:56:17.540676+0000","last_clean":"2024-04-25T18:56:17.540676+0000","last_became_active":"2024-04-25T18:55:19.162005+0000","last_became_peered":"2024-04-25T18:55:19.162005+0000","last_unstale":"2024-04-25T18:56:17.540676+0000","last_undegraded":"2024-04-25T18:56:17.540676+0000","last_fullsized":"2024-04-25T18:56:17.540676+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:50:53.967081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.538221+0000","last_change":"2024-04-25T18:55:23.222345+0000","last_active":"2024-04-25T18:56:17.538221+0000","last_peered":"2024-04-25T18:56:17.538221+0000","last_clean":"2024-04-25T18:56:17.538221+0000","last_became_active":"2024-04-25T18:55:23.221142+0000","last_became_peered":"2024-04-25T18:55:23.221142+0000","last_unstale":"2024-04-25T18:56:17.538221+0000","last_undegraded":"2024-04-25T18:56:17.538221+0000","last_fullsized":"2024-04-25T18:56:17.538221+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:53.823585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550605+0000","last_change":"2024-04-25T18:55:19.181153+0000","last_active":"2024-04-25T18:56:17.550605+0000","last_peered":"2024-04-25T18:56:17.550605+0000","last_clean":"2024-04-25T18:56:17.550605+0000","last_became_active":"2024-04-25T18:55:19.180553+0000","last_became_peered":"2024-04-25T18:55:19.180553+0000","last_unstale":"2024-04-25T18:56:17.550605+0000","last_undegraded":"2024-04-25T18:56:17.550605+0000","last_fullsized":"2024-04-25T18:56:17.550605+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:07.829617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"92'1","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540422+0000","last_change":"2024-04-25T18:55:23.238824+0000","last_active":"2024-04-25T18:56:17.540422+0000","last_peered":"2024-04-25T18:56:17.540422+0000","last_clean":"2024-04-25T18:56:17.540422+0000","last_became_active":"2024-04-25T18:55:23.238228+0000","last_became_peered":"2024-04-25T18:55:23.238228+0000","last_unstale":"2024-04-25T18:56:17.540422+0000","last_undegraded":"2024-04-25T18:56:17.540422+0000","last_fullsized":"2024-04-25T18:56:17.540422+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:16.279254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583314+0000","last_change":"2024-04-25T18:55:19.171186+0000","last_active":"2024-04-25T18:56:17.583314+0000","last_peered":"2024-04-25T18:56:17.583314+0000","last_clean":"2024-04-25T18:56:17.583314+0000","last_became_active":"2024-04-25T18:55:19.170800+0000","last_became_peered":"2024-04-25T18:55:19.170800+0000","last_unstale":"2024-04-25T18:56:17.583314+0000","last_undegraded":"2024-04-25T18:56:17.583314+0000","last_fullsized":"2024-04-25T18:56:17.583314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.599119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540524+0000","last_change":"2024-04-25T18:55:23.236951+0000","last_active":"2024-04-25T18:56:17.540524+0000","last_peered":"2024-04-25T18:56:17.540524+0000","last_clean":"2024-04-25T18:56:17.540524+0000","last_became_active":"2024-04-25T18:55:23.236596+0000","last_became_peered":"2024-04-25T18:55:23.236596+0000","last_unstale":"2024-04-25T18:56:17.540524+0000","last_undegraded":"2024-04-25T18:56:17.540524+0000","last_fullsized":"2024-04-25T18:56:17.540524+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:14.634653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540262+0000","last_change":"2024-04-25T18:55:23.211984+0000","last_active":"2024-04-25T18:56:17.540262+0000","last_peered":"2024-04-25T18:56:17.540262+0000","last_clean":"2024-04-25T18:56:17.540262+0000","last_became_active":"2024-04-25T18:55:23.211591+0000","last_became_peered":"2024-04-25T18:55:23.211591+0000","last_unstale":"2024-04-25T18:56:17.540262+0000","last_undegraded":"2024-04-25T18:56:17.540262+0000","last_fullsized":"2024-04-25T18:56:17.540262+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:09.024891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"92'4","reported_seq":63,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.549379+0000","last_change":"2024-04-25T18:55:19.180378+0000","last_active":"2024-04-25T18:56:17.549379+0000","last_peered":"2024-04-25T18:56:17.549379+0000","last_clean":"2024-04-25T18:56:17.549379+0000","last_became_active":"2024-04-25T18:55:19.179905+0000","last_became_peered":"2024-04-25T18:55:19.179905+0000","last_unstale":"2024-04-25T18:56:17.549379+0000","last_undegraded":"2024-04-25T18:56:17.549379+0000","last_fullsized":"2024-04-25T18:56:17.549379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:36.434477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":33,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.546088+0000","last_change":"2024-04-25T18:55:23.231489+0000","last_active":"2024-04-25T18:56:17.546088+0000","last_peered":"2024-04-25T18:56:17.546088+0000","last_clean":"2024-04-25T18:56:17.546088+0000","last_became_active":"2024-04-25T18:55:23.231049+0000","last_became_peered":"2024-04-25T18:55:23.231049+0000","last_unstale":"2024-04-25T18:56:17.546088+0000","last_undegraded":"2024-04-25T18:56:17.546088+0000","last_fullsized":"2024-04-25T18:56:17.546088+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:25.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539259+0000","last_change":"2024-04-25T18:55:19.171694+0000","last_active":"2024-04-25T18:56:17.539259+0000","last_peered":"2024-04-25T18:56:17.539259+0000","last_clean":"2024-04-25T18:56:17.539259+0000","last_became_active":"2024-04-25T18:55:19.171296+0000","last_became_peered":"2024-04-25T18:55:19.171296+0000","last_unstale":"2024-04-25T18:56:17.539259+0000","last_undegraded":"2024-04-25T18:56:17.539259+0000","last_fullsized":"2024-04-25T18:56:17.539259+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:33.216980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":362,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583202+0000","last_change":"2024-04-25T18:53:49.127506+0000","last_active":"2024-04-25T18:56:17.583202+0000","last_peered":"2024-04-25T18:56:17.583202+0000","last_clean":"2024-04-25T18:56:17.583202+0000","last_became_active":"2024-04-25T18:53:49.126974+0000","last_became_peered":"2024-04-25T18:53:49.126974+0000","last_unstale":"2024-04-25T18:56:17.583202+0000","last_undegraded":"2024-04-25T18:56:17.583202+0000","last_fullsized":"2024-04-25T18:56:17.583202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_clean_scrub_stamp":"2024-04-25T18:50:00.765468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.488525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.549219+0000","last_change":"2024-04-25T18:55:23.211697+0000","last_active":"2024-04-25T18:56:17.549219+0000","last_peered":"2024-04-25T18:56:17.549219+0000","last_clean":"2024-04-25T18:56:17.549219+0000","last_became_active":"2024-04-25T18:55:23.211160+0000","last_became_peered":"2024-04-25T18:55:23.211160+0000","last_unstale":"2024-04-25T18:56:17.549219+0000","last_undegraded":"2024-04-25T18:56:17.549219+0000","last_fullsized":"2024-04-25T18:56:17.549219+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:03.452578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583299+0000","last_change":"2024-04-25T18:55:19.165955+0000","last_active":"2024-04-25T18:56:17.583299+0000","last_peered":"2024-04-25T18:56:17.583299+0000","last_clean":"2024-04-25T18:56:17.583299+0000","last_became_active":"2024-04-25T18:55:19.165594+0000","last_became_peered":"2024-04-25T18:55:19.165594+0000","last_unstale":"2024-04-25T18:56:17.583299+0000","last_undegraded":"2024-04-25T18:56:17.583299+0000","last_fullsized":"2024-04-25T18:56:17.583299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:11.692374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540424+0000","last_change":"2024-04-25T18:55:19.182698+0000","last_active":"2024-04-25T18:56:17.540424+0000","last_peered":"2024-04-25T18:56:17.540424+0000","last_clean":"2024-04-25T18:56:17.540424+0000","last_became_active":"2024-04-25T18:55:19.182393+0000","last_became_peered":"2024-04-25T18:55:19.182393+0000","last_unstale":"2024-04-25T18:56:17.540424+0000","last_undegraded":"2024-04-25T18:56:17.540424+0000","last_fullsized":"2024-04-25T18:56:17.540424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.513613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550144+0000","last_change":"2024-04-25T18:55:23.223240+0000","last_active":"2024-04-25T18:56:17.550144+0000","last_peered":"2024-04-25T18:56:17.550144+0000","last_clean":"2024-04-25T18:56:17.550144+0000","last_became_active":"2024-04-25T18:55:23.218253+0000","last_became_peered":"2024-04-25T18:55:23.218253+0000","last_unstale":"2024-04-25T18:56:17.550144+0000","last_undegraded":"2024-04-25T18:56:17.550144+0000","last_fullsized":"2024-04-25T18:56:17.550144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:40.435633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'12","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.547262+0000","last_change":"2024-04-25T18:55:19.168535+0000","last_active":"2024-04-25T18:56:17.547262+0000","last_peered":"2024-04-25T18:56:17.547262+0000","last_clean":"2024-04-25T18:56:17.547262+0000","last_became_active":"2024-04-25T18:55:19.166735+0000","last_became_peered":"2024-04-25T18:55:19.166735+0000","last_unstale":"2024-04-25T18:56:17.547262+0000","last_undegraded":"2024-04-25T18:56:17.547262+0000","last_fullsized":"2024-04-25T18:56:17.547262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:50:14.008290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.546844+0000","last_change":"2024-04-25T18:55:23.233989+0000","last_active":"2024-04-25T18:56:17.546844+0000","last_peered":"2024-04-25T18:56:17.546844+0000","last_clean":"2024-04-25T18:56:17.546844+0000","last_became_active":"2024-04-25T18:55:23.226867+0000","last_became_peered":"2024-04-25T18:55:23.226867+0000","last_unstale":"2024-04-25T18:56:17.546844+0000","last_undegraded":"2024-04-25T18:56:17.546844+0000","last_fullsized":"2024-04-25T18:56:17.546844+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:02.314368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539501+0000","last_change":"2024-04-25T18:55:19.870412+0000","last_active":"2024-04-25T18:56:17.539501+0000","last_peered":"2024-04-25T18:56:17.539501+0000","last_clean":"2024-04-25T18:56:17.539501+0000","last_became_active":"2024-04-25T18:55:19.869970+0000","last_became_peered":"2024-04-25T18:55:19.869970+0000","last_unstale":"2024-04-25T18:56:17.539501+0000","last_undegraded":"2024-04-25T18:56:17.539501+0000","last_fullsized":"2024-04-25T18:56:17.539501+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:22.236417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.546016+0000","last_change":"2024-04-25T18:55:23.239172+0000","last_active":"2024-04-25T18:56:17.546016+0000","last_peered":"2024-04-25T18:56:17.546016+0000","last_clean":"2024-04-25T18:56:17.546016+0000","last_became_active":"2024-04-25T18:55:23.238732+0000","last_became_peered":"2024-04-25T18:55:23.238732+0000","last_unstale":"2024-04-25T18:56:17.546016+0000","last_undegraded":"2024-04-25T18:56:17.546016+0000","last_fullsized":"2024-04-25T18:56:17.546016+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:27.459659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540749+0000","last_change":"2024-04-25T18:55:19.179687+0000","last_active":"2024-04-25T18:56:17.540749+0000","last_peered":"2024-04-25T18:56:17.540749+0000","last_clean":"2024-04-25T18:56:17.540749+0000","last_became_active":"2024-04-25T18:55:19.179225+0000","last_became_peered":"2024-04-25T18:55:19.179225+0000","last_unstale":"2024-04-25T18:56:17.540749+0000","last_undegraded":"2024-04-25T18:56:17.540749+0000","last_fullsized":"2024-04-25T18:56:17.540749+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:16:16.266021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540122+0000","last_change":"2024-04-25T18:55:23.221379+0000","last_active":"2024-04-25T18:56:17.540122+0000","last_peered":"2024-04-25T18:56:17.540122+0000","last_clean":"2024-04-25T18:56:17.540122+0000","last_became_active":"2024-04-25T18:55:23.221037+0000","last_became_peered":"2024-04-25T18:55:23.221037+0000","last_unstale":"2024-04-25T18:56:17.540122+0000","last_undegraded":"2024-04-25T18:56:17.540122+0000","last_fullsized":"2024-04-25T18:56:17.540122+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:27.581084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.546247+0000","last_change":"2024-04-25T18:55:19.171298+0000","last_active":"2024-04-25T18:56:17.546247+0000","last_peered":"2024-04-25T18:56:17.546247+0000","last_clean":"2024-04-25T18:56:17.546247+0000","last_became_active":"2024-04-25T18:55:19.170738+0000","last_became_peered":"2024-04-25T18:55:19.170738+0000","last_unstale":"2024-04-25T18:56:17.546247+0000","last_undegraded":"2024-04-25T18:56:17.546247+0000","last_fullsized":"2024-04-25T18:56:17.546247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.236331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.538060+0000","last_change":"2024-04-25T18:55:23.226577+0000","last_active":"2024-04-25T18:56:17.538060+0000","last_peered":"2024-04-25T18:56:17.538060+0000","last_clean":"2024-04-25T18:56:17.538060+0000","last_became_active":"2024-04-25T18:55:23.226198+0000","last_became_peered":"2024-04-25T18:55:23.226198+0000","last_unstale":"2024-04-25T18:56:17.538060+0000","last_undegraded":"2024-04-25T18:56:17.538060+0000","last_fullsized":"2024-04-25T18:56:17.538060+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:34:59.124523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536353+0000","last_change":"2024-04-25T18:55:19.171917+0000","last_active":"2024-04-25T18:56:17.536353+0000","last_peered":"2024-04-25T18:56:17.536353+0000","last_clean":"2024-04-25T18:56:17.536353+0000","last_became_active":"2024-04-25T18:55:19.171506+0000","last_became_peered":"2024-04-25T18:55:19.171506+0000","last_unstale":"2024-04-25T18:56:17.536353+0000","last_undegraded":"2024-04-25T18:56:17.536353+0000","last_fullsized":"2024-04-25T18:56:17.536353+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:53.446504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.544887+0000","last_change":"2024-04-25T18:55:23.237828+0000","last_active":"2024-04-25T18:56:17.544887+0000","last_peered":"2024-04-25T18:56:17.544887+0000","last_clean":"2024-04-25T18:56:17.544887+0000","last_became_active":"2024-04-25T18:55:23.237310+0000","last_became_peered":"2024-04-25T18:55:23.237310+0000","last_unstale":"2024-04-25T18:56:17.544887+0000","last_undegraded":"2024-04-25T18:56:17.544887+0000","last_fullsized":"2024-04-25T18:56:17.544887+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:10.987032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540249+0000","last_change":"2024-04-25T18:55:19.163824+0000","last_active":"2024-04-25T18:56:17.540249+0000","last_peered":"2024-04-25T18:56:17.540249+0000","last_clean":"2024-04-25T18:56:17.540249+0000","last_became_active":"2024-04-25T18:55:19.163465+0000","last_became_peered":"2024-04-25T18:55:19.163465+0000","last_unstale":"2024-04-25T18:56:17.540249+0000","last_undegraded":"2024-04-25T18:56:17.540249+0000","last_fullsized":"2024-04-25T18:56:17.540249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:38.347100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583336+0000","last_change":"2024-04-25T18:55:23.224648+0000","last_active":"2024-04-25T18:56:17.583336+0000","last_peered":"2024-04-25T18:56:17.583336+0000","last_clean":"2024-04-25T18:56:17.583336+0000","last_became_active":"2024-04-25T18:55:23.224323+0000","last_became_peered":"2024-04-25T18:55:23.224323+0000","last_unstale":"2024-04-25T18:56:17.583336+0000","last_undegraded":"2024-04-25T18:56:17.583336+0000","last_fullsized":"2024-04-25T18:56:17.583336+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.117792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536684+0000","last_change":"2024-04-25T18:55:19.870213+0000","last_active":"2024-04-25T18:56:17.536684+0000","last_peered":"2024-04-25T18:56:17.536684+0000","last_clean":"2024-04-25T18:56:17.536684+0000","last_became_active":"2024-04-25T18:55:19.869756+0000","last_became_peered":"2024-04-25T18:55:19.869756+0000","last_unstale":"2024-04-25T18:56:17.536684+0000","last_undegraded":"2024-04-25T18:56:17.536684+0000","last_fullsized":"2024-04-25T18:56:17.536684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:44.695876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583631+0000","last_change":"2024-04-25T18:55:23.217094+0000","last_active":"2024-04-25T18:56:17.583631+0000","last_peered":"2024-04-25T18:56:17.583631+0000","last_clean":"2024-04-25T18:56:17.583631+0000","last_became_active":"2024-04-25T18:55:23.216696+0000","last_became_peered":"2024-04-25T18:55:23.216696+0000","last_unstale":"2024-04-25T18:56:17.583631+0000","last_undegraded":"2024-04-25T18:56:17.583631+0000","last_fullsized":"2024-04-25T18:56:17.583631+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:07.049349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540084+0000","last_change":"2024-04-25T18:55:19.166377+0000","last_active":"2024-04-25T18:56:17.540084+0000","last_peered":"2024-04-25T18:56:17.540084+0000","last_clean":"2024-04-25T18:56:17.540084+0000","last_became_active":"2024-04-25T18:55:19.166036+0000","last_became_peered":"2024-04-25T18:55:19.166036+0000","last_unstale":"2024-04-25T18:56:17.540084+0000","last_undegraded":"2024-04-25T18:56:17.540084+0000","last_fullsized":"2024-04-25T18:56:17.540084+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:04.761919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"92'1","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583448+0000","last_change":"2024-04-25T18:55:23.236893+0000","last_active":"2024-04-25T18:56:17.583448+0000","last_peered":"2024-04-25T18:56:17.583448+0000","last_clean":"2024-04-25T18:56:17.583448+0000","last_became_active":"2024-04-25T18:55:23.236488+0000","last_became_peered":"2024-04-25T18:55:23.236488+0000","last_unstale":"2024-04-25T18:56:17.583448+0000","last_undegraded":"2024-04-25T18:56:17.583448+0000","last_fullsized":"2024-04-25T18:56:17.583448+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:44.605041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.551149+0000","last_change":"2024-04-25T18:55:19.175076+0000","last_active":"2024-04-25T18:56:17.551149+0000","last_peered":"2024-04-25T18:56:17.551149+0000","last_clean":"2024-04-25T18:56:17.551149+0000","last_became_active":"2024-04-25T18:55:19.174547+0000","last_became_peered":"2024-04-25T18:55:19.174547+0000","last_unstale":"2024-04-25T18:56:17.551149+0000","last_undegraded":"2024-04-25T18:56:17.551149+0000","last_fullsized":"2024-04-25T18:56:17.551149+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:43.791068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583617+0000","last_change":"2024-04-25T18:55:23.219665+0000","last_active":"2024-04-25T18:56:17.583617+0000","last_peered":"2024-04-25T18:56:17.583617+0000","last_clean":"2024-04-25T18:56:17.583617+0000","last_became_active":"2024-04-25T18:55:23.219264+0000","last_became_peered":"2024-04-25T18:55:23.219264+0000","last_unstale":"2024-04-25T18:56:17.583617+0000","last_undegraded":"2024-04-25T18:56:17.583617+0000","last_fullsized":"2024-04-25T18:56:17.583617+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:19.008120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.538101+0000","last_change":"2024-04-25T18:55:23.220290+0000","last_active":"2024-04-25T18:56:17.538101+0000","last_peered":"2024-04-25T18:56:17.538101+0000","last_clean":"2024-04-25T18:56:17.538101+0000","last_became_active":"2024-04-25T18:55:23.219131+0000","last_became_peered":"2024-04-25T18:55:23.219131+0000","last_unstale":"2024-04-25T18:56:17.538101+0000","last_undegraded":"2024-04-25T18:56:17.538101+0000","last_fullsized":"2024-04-25T18:56:17.538101+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:25:25.678525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536245+0000","last_change":"2024-04-25T18:55:23.216415+0000","last_active":"2024-04-25T18:56:17.536245+0000","last_peered":"2024-04-25T18:56:17.536245+0000","last_clean":"2024-04-25T18:56:17.536245+0000","last_became_active":"2024-04-25T18:55:23.216066+0000","last_became_peered":"2024-04-25T18:55:23.216066+0000","last_unstale":"2024-04-25T18:56:17.536245+0000","last_undegraded":"2024-04-25T18:56:17.536245+0000","last_fullsized":"2024-04-25T18:56:17.536245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:54.633216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536208+0000","last_change":"2024-04-25T18:55:23.234034+0000","last_active":"2024-04-25T18:56:17.536208+0000","last_peered":"2024-04-25T18:56:17.536208+0000","last_clean":"2024-04-25T18:56:17.536208+0000","last_became_active":"2024-04-25T18:55:23.233639+0000","last_became_peered":"2024-04-25T18:55:23.233639+0000","last_unstale":"2024-04-25T18:56:17.536208+0000","last_undegraded":"2024-04-25T18:56:17.536208+0000","last_fullsized":"2024-04-25T18:56:17.536208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:57.117958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.584511+0000","last_change":"2024-04-25T18:56:17.584511+0000","last_active":"2024-04-25T18:56:17.584511+0000","last_peered":"2024-04-25T18:56:17.584511+0000","last_clean":"2024-04-25T18:56:17.584511+0000","last_became_active":"2024-04-25T18:56:17.583959+0000","last_became_peered":"2024-04-25T18:56:17.583959+0000","last_unstale":"2024-04-25T18:56:17.584511+0000","last_undegraded":"2024-04-25T18:56:17.584511+0000","last_fullsized":"2024-04-25T18:56:17.584511+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.622823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550041+0000","last_change":"2024-04-25T18:55:23.238068+0000","last_active":"2024-04-25T18:56:17.550041+0000","last_peered":"2024-04-25T18:56:17.550041+0000","last_clean":"2024-04-25T18:56:17.550041+0000","last_became_active":"2024-04-25T18:55:23.237773+0000","last_became_peered":"2024-04-25T18:55:23.237773+0000","last_unstale":"2024-04-25T18:56:17.550041+0000","last_undegraded":"2024-04-25T18:56:17.550041+0000","last_fullsized":"2024-04-25T18:56:17.550041+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.688576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.538286+0000","last_change":"2024-04-25T18:55:23.215463+0000","last_active":"2024-04-25T18:56:17.538286+0000","last_peered":"2024-04-25T18:56:17.538286+0000","last_clean":"2024-04-25T18:56:17.538286+0000","last_became_active":"2024-04-25T18:55:23.215068+0000","last_became_peered":"2024-04-25T18:55:23.215068+0000","last_unstale":"2024-04-25T18:56:17.538286+0000","last_undegraded":"2024-04-25T18:56:17.538286+0000","last_fullsized":"2024-04-25T18:56:17.538286+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.450828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536634+0000","last_change":"2024-04-25T18:55:23.214362+0000","last_active":"2024-04-25T18:56:17.536634+0000","last_peered":"2024-04-25T18:56:17.536634+0000","last_clean":"2024-04-25T18:56:17.536634+0000","last_became_active":"2024-04-25T18:55:23.214035+0000","last_became_peered":"2024-04-25T18:55:23.214035+0000","last_unstale":"2024-04-25T18:56:17.536634+0000","last_undegraded":"2024-04-25T18:56:17.536634+0000","last_fullsized":"2024-04-25T18:56:17.536634+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:33.386859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.546644+0000","last_change":"2024-04-25T18:55:23.235709+0000","last_active":"2024-04-25T18:56:17.546644+0000","last_peered":"2024-04-25T18:56:17.546644+0000","last_clean":"2024-04-25T18:56:17.546644+0000","last_became_active":"2024-04-25T18:55:23.226089+0000","last_became_peered":"2024-04-25T18:55:23.226089+0000","last_unstale":"2024-04-25T18:56:17.546644+0000","last_undegraded":"2024-04-25T18:56:17.546644+0000","last_fullsized":"2024-04-25T18:56:17.546644+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:57.161365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.563613+0000","last_change":"2024-04-25T18:56:17.561546+0000","last_active":"2024-04-25T18:56:17.563613+0000","last_peered":"2024-04-25T18:56:17.563613+0000","last_clean":"2024-04-25T18:56:17.563613+0000","last_became_active":"2024-04-25T18:56:17.559767+0000","last_became_peered":"2024-04-25T18:56:17.559767+0000","last_unstale":"2024-04-25T18:56:17.563613+0000","last_undegraded":"2024-04-25T18:56:17.563613+0000","last_fullsized":"2024-04-25T18:56:17.563613+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:59.512008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.544365+0000","last_change":"2024-04-25T18:55:23.227620+0000","last_active":"2024-04-25T18:56:17.544365+0000","last_peered":"2024-04-25T18:56:17.544365+0000","last_clean":"2024-04-25T18:56:17.544365+0000","last_became_active":"2024-04-25T18:55:23.227236+0000","last_became_peered":"2024-04-25T18:55:23.227236+0000","last_unstale":"2024-04-25T18:56:17.544365+0000","last_undegraded":"2024-04-25T18:56:17.544365+0000","last_fullsized":"2024-04-25T18:56:17.544365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:32.707619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.541206+0000","last_change":"2024-04-25T18:55:23.238393+0000","last_active":"2024-04-25T18:56:17.541206+0000","last_peered":"2024-04-25T18:56:17.541206+0000","last_clean":"2024-04-25T18:56:17.541206+0000","last_became_active":"2024-04-25T18:55:23.237726+0000","last_became_peered":"2024-04-25T18:55:23.237726+0000","last_unstale":"2024-04-25T18:56:17.541206+0000","last_undegraded":"2024-04-25T18:56:17.541206+0000","last_fullsized":"2024-04-25T18:56:17.541206+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:48.543014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550620+0000","last_change":"2024-04-25T18:55:23.231374+0000","last_active":"2024-04-25T18:56:17.550620+0000","last_peered":"2024-04-25T18:56:17.550620+0000","last_clean":"2024-04-25T18:56:17.550620+0000","last_became_active":"2024-04-25T18:55:23.230976+0000","last_became_peered":"2024-04-25T18:55:23.230976+0000","last_unstale":"2024-04-25T18:56:17.550620+0000","last_undegraded":"2024-04-25T18:56:17.550620+0000","last_fullsized":"2024-04-25T18:56:17.550620+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:06.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.538371+0000","last_change":"2024-04-25T18:55:23.217332+0000","last_active":"2024-04-25T18:56:17.538371+0000","last_peered":"2024-04-25T18:56:17.538371+0000","last_clean":"2024-04-25T18:56:17.538371+0000","last_became_active":"2024-04-25T18:55:23.216968+0000","last_became_peered":"2024-04-25T18:55:23.216968+0000","last_unstale":"2024-04-25T18:56:17.538371+0000","last_undegraded":"2024-04-25T18:56:17.538371+0000","last_fullsized":"2024-04-25T18:56:17.538371+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:31.000083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67464,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":238,"num_read_kb":190,"num_write":167,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":202392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252892,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1032192,"data_stored":498791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514531,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37032,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708120,"statfs":{"total":95995035648,"available":95957114880,"internally_reserved":0,"allocated":1728512,"data_stored":1153701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710758,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1044480,"data_stored":498993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972398,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":498913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234035,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36944,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708208,"statfs":{"total":95995035648,"available":95957204992,"internally_reserved":0,"allocated":1638400,"data_stored":1091234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430262,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":498382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1097728,"data_stored":561247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1052672,"data_stored":499653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1118208,"data_stored":562954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117064,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1048576,"data_stored":498693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":499210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691901,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28632,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716520,"statfs":{"total":95995035648,"available":95965716480,"internally_reserved":0,"allocated":1650688,"data_stored":1090527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":63072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":65041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:56:26.212 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T18:56:26.212 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T18:56:26.213 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T18:56:26.737 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:26.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:26 smithi079 ceph-mon[33002]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:56:26.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:26 smithi045 ceph-mon[27776]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:56:27.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:26 smithi062 ceph-mon[32844]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:56:27.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:27 smithi079 ceph-mon[33002]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:56:27.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:27 smithi045 ceph-mon[27776]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:56:28.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:27 smithi062 ceph-mon[32844]: pgmap v359: 97 pgs: 97 active+clean; 643 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 3 B/s, 1 keys/s, 0 objects/s recovering 2024-04-25T18:56:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.013+0000 7f9df4e00700 1 -- 172.21.15.45:0/2111398437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df01087b0 msgr2=0x7f9df010ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.013+0000 7f9df4e00700 1 --2- 172.21.15.45:0/2111398437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df01087b0 0x7f9df010ac20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009f90 tx=0x7f9de800b330 comp rx=0 tx=0).stop 2024-04-25T18:56:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.014+0000 7f9df4e00700 1 -- 172.21.15.45:0/2111398437 shutdown_connections 2024-04-25T18:56:29.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.014+0000 7f9df4e00700 1 --2- 172.21.15.45:0/2111398437 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df0103550 0x7f9df0105940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:29.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.014+0000 7f9df4e00700 1 --2- 172.21.15.45:0/2111398437 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df01087b0 0x7f9df010ac20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:29.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.014+0000 7f9df4e00700 1 --2- 172.21.15.45:0/2111398437 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9df0105e80 0x7f9df0108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:29.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.014+0000 7f9df4e00700 1 -- 172.21.15.45:0/2111398437 >> 172.21.15.45:0/2111398437 conn(0x7f9df00fcf60 msgr2=0x7f9df00ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:29.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.014+0000 7f9df4e00700 1 -- 172.21.15.45:0/2111398437 shutdown_connections 2024-04-25T18:56:29.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.014+0000 7f9df4e00700 1 -- 172.21.15.45:0/2111398437 wait complete. 2024-04-25T18:56:29.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.015+0000 7f9df4e00700 1 Processor -- start 2024-04-25T18:56:29.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.015+0000 7f9df4e00700 1 -- start start 2024-04-25T18:56:29.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.015+0000 7f9df4e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9df0103550 0x7f9df0198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:29.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.015+0000 7f9df4e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df0105e80 0x7f9df0199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:29.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.015+0000 7f9df4e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df01087b0 0x7f9df01a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:29.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.015+0000 7f9df4e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9df010e570 con 0x7f9df0103550 2024-04-25T18:56:29.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.015+0000 7f9df4e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9df010e3f0 con 0x7f9df0105e80 2024-04-25T18:56:29.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.015+0000 7f9df4e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9df010e6f0 con 0x7f9df01087b0 2024-04-25T18:56:29.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9dede00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9df0103550 0x7f9df0198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:29.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9dede00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9df0103550 0x7f9df0198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48214/0 (socket says 172.21.15.45:48214) 2024-04-25T18:56:29.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9dede00700 1 -- 172.21.15.45:0/2013039860 learned_addr learned my addr 172.21.15.45:0/2013039860 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:29.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9dee800700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df01087b0 0x7f9df01a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:29.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9ded400700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df0105e80 0x7f9df0199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:29.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9dee800700 1 -- 172.21.15.45:0/2013039860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df0105e80 msgr2=0x7f9df0199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:29.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9dee800700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df0105e80 0x7f9df0199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:29.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9dee800700 1 -- 172.21.15.45:0/2013039860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9df0103550 msgr2=0x7f9df0198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:29.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9dee800700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9df0103550 0x7f9df0198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:29.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9dee800700 1 -- 172.21.15.45:0/2013039860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df01a0740 con 0x7f9df01087b0 2024-04-25T18:56:29.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.016+0000 7f9dede00700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9df0103550 0x7f9df0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:29.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.017+0000 7f9dee800700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df01087b0 0x7f9df01a00a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009550 tx=0x7f9de800cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:29.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.017+0000 7f9de6a00700 1 -- 172.21.15.45:0/2013039860 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de8021020 con 0x7f9df01087b0 2024-04-25T18:56:29.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.017+0000 7f9de6a00700 1 -- 172.21.15.45:0/2013039860 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9de8007ac0 con 0x7f9df01087b0 2024-04-25T18:56:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.017+0000 7f9df4e00700 1 -- 172.21.15.45:0/2013039860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df01a09d0 con 0x7f9df01087b0 2024-04-25T18:56:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.017+0000 7f9de6a00700 1 -- 172.21.15.45:0/2013039860 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de80052a0 con 0x7f9df01087b0 2024-04-25T18:56:29.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.017+0000 7f9df4e00700 1 -- 172.21.15.45:0/2013039860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df01a0ed0 con 0x7f9df01087b0 2024-04-25T18:56:29.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.018+0000 7f9df4e00700 1 -- 172.21.15.45:0/2013039860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df0050f80 con 0x7f9df01087b0 2024-04-25T18:56:29.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.020+0000 7f9de6a00700 1 -- 172.21.15.45:0/2013039860 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9de8024020 con 0x7f9df01087b0 2024-04-25T18:56:29.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.022+0000 7f9de6a00700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9dd40a3e00 0x7f9dd40a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:29.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.022+0000 7f9de6a00700 1 -- 172.21.15.45:0/2013039860 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f9de800f020 con 0x7f9df01087b0 2024-04-25T18:56:29.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.022+0000 7f9dede00700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9dd40a3e00 0x7f9dd40a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:29.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.023+0000 7f9dede00700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9dd40a3e00 0x7f9dd40a62c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9de0005950 tx=0x7f9de0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:29.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.026+0000 7f9de6a00700 1 -- 172.21.15.45:0/2013039860 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9de8011e50 con 0x7f9df01087b0 2024-04-25T18:56:29.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.351+0000 7f9df4e00700 1 -- 172.21.15.45:0/2013039860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f9df002dfb0 con 0x7f9df01087b0 2024-04-25T18:56:29.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.358+0000 7f9de6a00700 1 -- 172.21.15.45:0/2013039860 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f9de809deb0 con 0x7f9df01087b0 2024-04-25T18:56:29.361 INFO:teuthology.orchestra.run.smithi045.stdout:[client.0] 2024-04-25T18:56:29.361 INFO:teuthology.orchestra.run.smithi045.stdout: key = AQBdpypmR28eFRAAu1F3GwRCtnWREmH5clhfzg== 2024-04-25T18:56:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 -- 172.21.15.45:0/2013039860 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9dd40a3e00 msgr2=0x7f9dd40a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9dd40a3e00 0x7f9dd40a62c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9de0005950 tx=0x7f9de0009310 comp rx=0 tx=0).stop 2024-04-25T18:56:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 -- 172.21.15.45:0/2013039860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df01087b0 msgr2=0x7f9df01a00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:29.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df01087b0 0x7f9df01a00a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9de8009550 tx=0x7f9de800cfa0 comp rx=0 tx=0).stop 2024-04-25T18:56:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 -- 172.21.15.45:0/2013039860 shutdown_connections 2024-04-25T18:56:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f9dd40a3e00 0x7f9dd40a62c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9df01087b0 0x7f9df01a00a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9df0103550 0x7f9df0198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 --2- 172.21.15.45:0/2013039860 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9df0105e80 0x7f9df0199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 -- 172.21.15.45:0/2013039860 >> 172.21.15.45:0/2013039860 conn(0x7f9df00fcf60 msgr2=0x7f9df0103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 -- 172.21.15.45:0/2013039860 shutdown_connections 2024-04-25T18:56:29.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:29.362+0000 7f9df4e00700 1 -- 172.21.15.45:0/2013039860 wait complete. 2024-04-25T18:56:29.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:29 smithi045 ceph-mon[27776]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 286 B/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:56:29.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:29 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2013039860' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:56:29.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:29 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:56:29.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:29 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T18:56:30.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:29 smithi062 ceph-mon[32844]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 286 B/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:56:30.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:29 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2013039860' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:56:30.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:29 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:56:30.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:29 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T18:56:30.169 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:56:30.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T18:56:30.170 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T18:56:30.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:29 smithi079 ceph-mon[33002]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 286 B/s wr, 6 op/s; 3 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:56:30.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:29 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2013039860' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:56:30.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:29 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T18:56:30.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:29 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T18:56:30.301 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T18:56:30.301 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T18:56:30.301 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T18:56:30.835 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:31.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:31 smithi045 ceph-mon[27776]: pgmap v361: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:56:32.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:31 smithi062 ceph-mon[32844]: pgmap v361: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:56:32.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:31 smithi079 ceph-mon[33002]: pgmap v361: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:56:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.885+0000 7fe287e00700 1 -- 172.21.15.45:0/806378134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe288074de0 msgr2=0x7fe2880751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.885+0000 7fe287e00700 1 --2- 172.21.15.45:0/806378134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe288074de0 0x7fe2880751c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe278009f90 tx=0x7fe27800b330 comp rx=0 tx=0).stop 2024-04-25T18:56:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.885+0000 7fe287e00700 1 -- 172.21.15.45:0/806378134 shutdown_connections 2024-04-25T18:56:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.885+0000 7fe287e00700 1 --2- 172.21.15.45:0/806378134 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe288074de0 0x7fe2880751c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.885+0000 7fe287e00700 1 --2- 172.21.15.45:0/806378134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe288113960 0x7fe288115d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.885+0000 7fe287e00700 1 --2- 172.21.15.45:0/806378134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288075700 0x7fe288113420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.885+0000 7fe287e00700 1 -- 172.21.15.45:0/806378134 >> 172.21.15.45:0/806378134 conn(0x7fe2880fe420 msgr2=0x7fe288100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.885+0000 7fe287e00700 1 -- 172.21.15.45:0/806378134 shutdown_connections 2024-04-25T18:56:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.885+0000 7fe287e00700 1 -- 172.21.15.45:0/806378134 wait complete. 2024-04-25T18:56:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.887+0000 7fe287e00700 1 Processor -- start 2024-04-25T18:56:32.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.887+0000 7fe287e00700 1 -- start start 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.887+0000 7fe287e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe288074de0 0x7fe28819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.887+0000 7fe287e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288075700 0x7fe28819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.887+0000 7fe287e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe288113960 0x7fe2881a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.887+0000 7fe287e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe288119180 con 0x7fe288113960 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.887+0000 7fe287e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe288119000 con 0x7fe288074de0 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.887+0000 7fe287e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe288119300 con 0x7fe288075700 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe287400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe288113960 0x7fe2881a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe286a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe288074de0 0x7fe28819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe27fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288075700 0x7fe28819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe287400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe288113960 0x7fe2881a3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48230/0 (socket says 172.21.15.45:48230) 2024-04-25T18:56:32.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe287400700 1 -- 172.21.15.45:0/2633251875 learned_addr learned my addr 172.21.15.45:0/2633251875 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe286a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe288074de0 0x7fe28819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:51068/0 (socket says 172.21.15.45:51068) 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe286a00700 1 -- 172.21.15.45:0/2633251875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288075700 msgr2=0x7fe28819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe286a00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288075700 0x7fe28819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe286a00700 1 -- 172.21.15.45:0/2633251875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe288113960 msgr2=0x7fe2881a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe286a00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe288113960 0x7fe2881a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.888+0000 7fe286a00700 1 -- 172.21.15.45:0/2633251875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2881a46f0 con 0x7fe288074de0 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.889+0000 7fe286a00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe288074de0 0x7fe28819f730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe278000f80 tx=0x7fe27800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.889+0000 7fe27f400700 1 -- 172.21.15.45:0/2633251875 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278021070 con 0x7fe288074de0 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.889+0000 7fe27f400700 1 -- 172.21.15.45:0/2633251875 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe278007ac0 con 0x7fe288074de0 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.889+0000 7fe27f400700 1 -- 172.21.15.45:0/2633251875 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe278005040 con 0x7fe288074de0 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.889+0000 7fe287e00700 1 -- 172.21.15.45:0/2633251875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2881a4980 con 0x7fe288074de0 2024-04-25T18:56:32.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.889+0000 7fe287e00700 1 -- 172.21.15.45:0/2633251875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2881ac260 con 0x7fe288074de0 2024-04-25T18:56:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.890+0000 7fe287e00700 1 -- 172.21.15.45:0/2633251875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe288076520 con 0x7fe288074de0 2024-04-25T18:56:32.894 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.892+0000 7fe27f400700 1 -- 172.21.15.45:0/2633251875 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe278024070 con 0x7fe288074de0 2024-04-25T18:56:32.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.894+0000 7fe27f400700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe26c0a3cd0 0x7fe26c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:32.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.894+0000 7fe27f400700 1 -- 172.21.15.45:0/2633251875 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fe27800f020 con 0x7fe288074de0 2024-04-25T18:56:32.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.894+0000 7fe27fe00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe26c0a3cd0 0x7fe26c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:32.896 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.895+0000 7fe27fe00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe26c0a3cd0 0x7fe26c0a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe288106710 tx=0x7fe270009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:32.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:32.896+0000 7fe27f400700 1 -- 172.21.15.45:0/2633251875 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe27802b020 con 0x7fe288074de0 2024-04-25T18:56:33.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.192+0000 7fe287e00700 1 -- 172.21.15.45:0/2633251875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fe2881a0a40 con 0x7fe288074de0 2024-04-25T18:56:33.200 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.198+0000 7fe27f400700 1 -- 172.21.15.45:0/2633251875 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264994 (secure 0 0 0) 0x7fe27809e580 con 0x7fe288074de0 2024-04-25T18:56:33.201 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:56:33.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 -- 172.21.15.45:0/2633251875 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe26c0a3cd0 msgr2=0x7fe26c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe26c0a3cd0 0x7fe26c0a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe288106710 tx=0x7fe270009260 comp rx=0 tx=0).stop 2024-04-25T18:56:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 -- 172.21.15.45:0/2633251875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe288074de0 msgr2=0x7fe28819f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe288074de0 0x7fe28819f730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe278000f80 tx=0x7fe27800ccb0 comp rx=0 tx=0).stop 2024-04-25T18:56:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 -- 172.21.15.45:0/2633251875 shutdown_connections 2024-04-25T18:56:33.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe26c0a3cd0 0x7fe26c0a6190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:33.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe288074de0 0x7fe28819f730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:33.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe288113960 0x7fe2881a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:33.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 --2- 172.21.15.45:0/2633251875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe288075700 0x7fe28819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:33.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 -- 172.21.15.45:0/2633251875 >> 172.21.15.45:0/2633251875 conn(0x7fe2880fe420 msgr2=0x7fe2880ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:33.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 -- 172.21.15.45:0/2633251875 shutdown_connections 2024-04-25T18:56:33.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:33.205+0000 7fe287e00700 1 -- 172.21.15.45:0/2633251875 wait complete. 2024-04-25T18:56:33.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:33 smithi045 ceph-mon[27776]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:56:33.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:33 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2633251875' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T18:56:33.986 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:33 smithi079 ceph-mon[33002]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:56:33.986 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:33 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2633251875' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T18:56:34.072 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":943541413},{"type":"v1","addr":"172.21.15.45:6801","nonce":943541413}]},"active_addr":"172.21.15.45:6801/943541413","active_change":"2024-04-25T18:47:16.094881+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.45:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":2599503580}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":4010951391}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3590060181}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":3367087840}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.45:0","nonce":300244986}]}]} 2024-04-25T18:56:34.077 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T18:56:34.077 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T18:56:34.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:56:34.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:33 smithi062 ceph-mon[32844]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s; 2 B/s, 0 keys/s, 0 objects/s recovering 2024-04-25T18:56:34.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:33 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2633251875' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T18:56:34.571 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:35.763 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:35 smithi045 ceph-mon[27776]: pgmap v363: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:56:36.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:35 smithi062 ceph-mon[32844]: pgmap v363: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:56:36.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:35 smithi079 ceph-mon[33002]: pgmap v363: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T18:56:36.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.761+0000 7fb1e7200700 1 -- 172.21.15.45:0/713943267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e0109c80 msgr2=0x7fb1e010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.761+0000 7fb1e7200700 1 --2- 172.21.15.45:0/713943267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e0109c80 0x7fb1e010dcd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009990 tx=0x7fb1cc009ca0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.762+0000 7fb1e7200700 1 -- 172.21.15.45:0/713943267 shutdown_connections 2024-04-25T18:56:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.762+0000 7fb1e7200700 1 --2- 172.21.15.45:0/713943267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e010e3b0 0x7fb1e0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.762+0000 7fb1e7200700 1 --2- 172.21.15.45:0/713943267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e01092d0 0x7fb1e01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.762+0000 7fb1e7200700 1 --2- 172.21.15.45:0/713943267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e0109c80 0x7fb1e010dcd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.762+0000 7fb1e7200700 1 -- 172.21.15.45:0/713943267 >> 172.21.15.45:0/713943267 conn(0x7fb1e0076210 msgr2=0x7fb1e0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.762+0000 7fb1e7200700 1 -- 172.21.15.45:0/713943267 shutdown_connections 2024-04-25T18:56:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.762+0000 7fb1e7200700 1 -- 172.21.15.45:0/713943267 wait complete. 2024-04-25T18:56:36.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.763+0000 7fb1e7200700 1 Processor -- start 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.763+0000 7fb1e7200700 1 -- start start 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.763+0000 7fb1e7200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e01092d0 0x7fb1e019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.763+0000 7fb1e7200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e0109c80 0x7fb1e019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.763+0000 7fb1e7200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e010e3b0 0x7fb1e01a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.763+0000 7fb1e7200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e0119270 con 0x7fb1e0109c80 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.763+0000 7fb1e7200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01190f0 con 0x7fb1e01092d0 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.763+0000 7fb1e7200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e01193f0 con 0x7fb1e010e3b0 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.764+0000 7fb1dfe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e01092d0 0x7fb1e019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.764+0000 7fb1df400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e0109c80 0x7fb1e019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.764+0000 7fb1dfe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e01092d0 0x7fb1e019f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:45946/0 (socket says 172.21.15.45:45946) 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.764+0000 7fb1dfe00700 1 -- 172.21.15.45:0/3724716795 learned_addr learned my addr 172.21.15.45:0/3724716795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:36.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.764+0000 7fb1e5200700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e010e3b0 0x7fb1e01a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.764+0000 7fb1dfe00700 1 -- 172.21.15.45:0/3724716795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e010e3b0 msgr2=0x7fb1e01a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.764+0000 7fb1dfe00700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e010e3b0 0x7fb1e01a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.764+0000 7fb1dfe00700 1 -- 172.21.15.45:0/3724716795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e0109c80 msgr2=0x7fb1e019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.764+0000 7fb1dfe00700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e0109c80 0x7fb1e019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.764+0000 7fb1dfe00700 1 -- 172.21.15.45:0/3724716795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e01a4700 con 0x7fb1e01092d0 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.765+0000 7fb1dfe00700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e01092d0 0x7fb1e019f740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000e9a0 tx=0x7fb1d000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.765+0000 7fb1dcc00700 1 -- 172.21.15.45:0/3724716795 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d000ca10 con 0x7fb1e01092d0 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.765+0000 7fb1dcc00700 1 -- 172.21.15.45:0/3724716795 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1d000cbb0 con 0x7fb1e01092d0 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.765+0000 7fb1e7200700 1 -- 172.21.15.45:0/3724716795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e01a49f0 con 0x7fb1e01092d0 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.765+0000 7fb1dcc00700 1 -- 172.21.15.45:0/3724716795 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d0018680 con 0x7fb1e01092d0 2024-04-25T18:56:36.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.765+0000 7fb1e7200700 1 -- 172.21.15.45:0/3724716795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e01ac270 con 0x7fb1e01092d0 2024-04-25T18:56:36.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.766+0000 7fb1e7200700 1 -- 172.21.15.45:0/3724716795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e0069730 con 0x7fb1e01092d0 2024-04-25T18:56:36.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.768+0000 7fb1dcc00700 1 -- 172.21.15.45:0/3724716795 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb1d0010430 con 0x7fb1e01092d0 2024-04-25T18:56:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.769+0000 7fb1dcc00700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb1c40a3d50 0x7fb1c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:36.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.770+0000 7fb1dcc00700 1 -- 172.21.15.45:0/3724716795 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fb1d00d0740 con 0x7fb1e01092d0 2024-04-25T18:56:36.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.770+0000 7fb1df400700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb1c40a3d50 0x7fb1c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:36.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.770+0000 7fb1df400700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb1c40a3d50 0x7fb1c40a6210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb1e010e120 tx=0x7fb1cc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:36.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:36.774+0000 7fb1dcc00700 1 -- 172.21.15.45:0/3724716795 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb1d009cf40 con 0x7fb1e01092d0 2024-04-25T18:56:37.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.007+0000 7fb1e7200700 1 -- 172.21.15.45:0/3724716795 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1e002df80 con 0x7fb1e01092d0 2024-04-25T18:56:37.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.009+0000 7fb1dcc00700 1 -- 172.21.15.45:0/3724716795 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+22654 (secure 0 0 0) 0x7fb1d009c840 con 0x7fb1e01092d0 2024-04-25T18:56:37.012 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:56:37.012 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T18:56:17.531513+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:55:17.591864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:55:22.070218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:56:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.012+0000 7fb1e7200700 1 -- 172.21.15.45:0/3724716795 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb1c40a3d50 msgr2=0x7fb1c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.012+0000 7fb1e7200700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb1c40a3d50 0x7fb1c40a6210 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb1e010e120 tx=0x7fb1cc005fd0 comp rx=0 tx=0).stop 2024-04-25T18:56:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.012+0000 7fb1e7200700 1 -- 172.21.15.45:0/3724716795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e01092d0 msgr2=0x7fb1e019f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.012+0000 7fb1e7200700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e01092d0 0x7fb1e019f740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb1d000e9a0 tx=0x7fb1d000ecb0 comp rx=0 tx=0).stop 2024-04-25T18:56:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.013+0000 7fb1e7200700 1 -- 172.21.15.45:0/3724716795 shutdown_connections 2024-04-25T18:56:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.013+0000 7fb1e7200700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb1c40a3d50 0x7fb1c40a6210 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.013+0000 7fb1e7200700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb1e01092d0 0x7fb1e019f740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.013+0000 7fb1e7200700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb1e010e3b0 0x7fb1e01a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:37.015 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.013+0000 7fb1e7200700 1 --2- 172.21.15.45:0/3724716795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb1e0109c80 0x7fb1e019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.013+0000 7fb1e7200700 1 -- 172.21.15.45:0/3724716795 >> 172.21.15.45:0/3724716795 conn(0x7fb1e0076210 msgr2=0x7fb1e00767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.013+0000 7fb1e7200700 1 -- 172.21.15.45:0/3724716795 shutdown_connections 2024-04-25T18:56:37.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:37.013+0000 7fb1e7200700 1 -- 172.21.15.45:0/3724716795 wait complete. 2024-04-25T18:56:37.880 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T18:56:37.880 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:56:37.961 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:37 smithi045 ceph-mon[27776]: pgmap v364: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:56:37.962 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:37 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3724716795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:56:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:37 smithi062 ceph-mon[32844]: pgmap v364: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:56:38.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:37 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3724716795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:56:38.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:37 smithi079 ceph-mon[33002]: pgmap v364: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:56:38.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:37 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3724716795' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:56:38.366 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:39.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:39 smithi045 ceph-mon[27776]: pgmap v365: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:56:40.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:39 smithi062 ceph-mon[32844]: pgmap v365: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:56:40.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:39 smithi079 ceph-mon[33002]: pgmap v365: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:56:40.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.677+0000 7f220c000700 1 -- 172.21.15.45:0/3872445016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f220406a750 msgr2=0x7f220406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:40.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.677+0000 7f220c000700 1 --2- 172.21.15.45:0/3872445016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f220406a750 0x7f220406abb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f21f0009990 tx=0x7f21f0009ca0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.677+0000 7f220c000700 1 -- 172.21.15.45:0/3872445016 shutdown_connections 2024-04-25T18:56:40.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.677+0000 7f220c000700 1 --2- 172.21.15.45:0/3872445016 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220406b0f0 0x7f220410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.677+0000 7f220c000700 1 --2- 172.21.15.45:0/3872445016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2204108a30 0x7f2204108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.677+0000 7f220c000700 1 --2- 172.21.15.45:0/3872445016 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f220406a750 0x7f220406abb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.677+0000 7f220c000700 1 -- 172.21.15.45:0/3872445016 >> 172.21.15.45:0/3872445016 conn(0x7f22040788f0 msgr2=0x7f2204078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.677+0000 7f220c000700 1 -- 172.21.15.45:0/3872445016 shutdown_connections 2024-04-25T18:56:40.680 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.677+0000 7f220c000700 1 -- 172.21.15.45:0/3872445016 wait complete. 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.678+0000 7f220c000700 1 Processor -- start 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.678+0000 7f220c000700 1 -- start start 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.678+0000 7f220c000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220406a750 0x7f220419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.678+0000 7f220c000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f220406b0f0 0x7f220419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.678+0000 7f220c000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2204108a30 0x7f220419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.678+0000 7f220c000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2204112bd0 con 0x7f220406a750 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.678+0000 7f220c000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2204112a50 con 0x7f2204108a30 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.678+0000 7f220c000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2204112d50 con 0x7f220406b0f0 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.679+0000 7f2209600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220406a750 0x7f220419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.679+0000 7f220a000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2204108a30 0x7f220419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.679+0000 7f2209600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220406a750 0x7f220419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58892/0 (socket says 172.21.15.45:58892) 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.679+0000 7f2209600700 1 -- 172.21.15.45:0/4192721359 learned_addr learned my addr 172.21.15.45:0/4192721359 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:40.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.679+0000 7f2208c00700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f220406b0f0 0x7f220419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:40.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.679+0000 7f220a000700 1 -- 172.21.15.45:0/4192721359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f220406b0f0 msgr2=0x7f220419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.679+0000 7f220a000700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f220406b0f0 0x7f220419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.679+0000 7f220a000700 1 -- 172.21.15.45:0/4192721359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220406a750 msgr2=0x7f220419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.679+0000 7f220a000700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220406a750 0x7f220419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.679+0000 7f220a000700 1 -- 172.21.15.45:0/4192721359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22041a02f0 con 0x7f2204108a30 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.680+0000 7f220a000700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2204108a30 0x7f220419fbb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00efc0 tx=0x7f21fc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.680+0000 7f2202000700 1 -- 172.21.15.45:0/4192721359 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc0092e0 con 0x7f2204108a30 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.680+0000 7f2202000700 1 -- 172.21.15.45:0/4192721359 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21fc004bb0 con 0x7f2204108a30 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.680+0000 7f220c000700 1 -- 172.21.15.45:0/4192721359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22041a0580 con 0x7f2204108a30 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.680+0000 7f2202000700 1 -- 172.21.15.45:0/4192721359 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc004020 con 0x7f2204108a30 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.680+0000 7f220c000700 1 -- 172.21.15.45:0/4192721359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22041a7e60 con 0x7f2204108a30 2024-04-25T18:56:40.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.681+0000 7f220c000700 1 -- 172.21.15.45:0/4192721359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2204069730 con 0x7f2204108a30 2024-04-25T18:56:40.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.683+0000 7f2202000700 1 -- 172.21.15.45:0/4192721359 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f21fc01b030 con 0x7f2204108a30 2024-04-25T18:56:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.685+0000 7f2202000700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f21ec0a3d50 0x7f21ec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.685+0000 7f2202000700 1 -- 172.21.15.45:0/4192721359 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f21fc0d09b0 con 0x7f2204108a30 2024-04-25T18:56:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.685+0000 7f2209600700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f21ec0a3d50 0x7f21ec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:40.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.686+0000 7f2209600700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f21ec0a3d50 0x7f21ec0a6210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009ca0 tx=0x7f21f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:40.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.687+0000 7f2202000700 1 -- 172.21.15.45:0/4192721359 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21fc022020 con 0x7f2204108a30 2024-04-25T18:56:40.923 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.920+0000 7f220c000700 1 -- 172.21.15.45:0/4192721359 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22041a8030 con 0x7f2204108a30 2024-04-25T18:56:40.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.922+0000 7f2202000700 1 -- 172.21.15.45:0/4192721359 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+22654 (secure 0 0 0) 0x7f21fc010040 con 0x7f2204108a30 2024-04-25T18:56:40.925 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:56:40.925 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T18:56:17.531513+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:55:17.591864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:55:22.070218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:56:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 -- 172.21.15.45:0/4192721359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f21ec0a3d50 msgr2=0x7f21ec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f21ec0a3d50 0x7f21ec0a6210 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f21f8009ca0 tx=0x7f21f8009260 comp rx=0 tx=0).stop 2024-04-25T18:56:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 -- 172.21.15.45:0/4192721359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2204108a30 msgr2=0x7f220419fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2204108a30 0x7f220419fbb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00efc0 tx=0x7f21fc00c3a0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 -- 172.21.15.45:0/4192721359 shutdown_connections 2024-04-25T18:56:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f21ec0a3d50 0x7f21ec0a6210 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2204108a30 0x7f220419fbb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f220406a750 0x7f220419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 --2- 172.21.15.45:0/4192721359 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f220406b0f0 0x7f220419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:40.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 -- 172.21.15.45:0/4192721359 >> 172.21.15.45:0/4192721359 conn(0x7f22040788f0 msgr2=0x7f22040ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:40.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 -- 172.21.15.45:0/4192721359 shutdown_connections 2024-04-25T18:56:40.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:40.926+0000 7f220c000700 1 -- 172.21.15.45:0/4192721359 wait complete. 2024-04-25T18:56:41.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T18:56:41.828 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T18:56:41.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T18:56:41.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T18:56:41.829 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T18:56:41.830 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T18:56:41.830 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T18:56:41.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T18:56:41.831 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T18:56:41.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T18:56:41.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T18:56:41.832 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T18:56:41.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:41 smithi045 ceph-mon[27776]: pgmap v366: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:56:41.965 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:41 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/4192721359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:56:42.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:41 smithi062 ceph-mon[32844]: pgmap v366: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:56:42.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:41 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/4192721359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:56:42.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:41 smithi079 ceph-mon[33002]: pgmap v366: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:56:42.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:41 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/4192721359' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:56:42.893 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:42.907 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.268 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.270 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.275 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.278 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.509 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.644 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.646 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.648 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:56:43.921 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:43 smithi045 ceph-mon[27776]: pgmap v367: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:56:44.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:43 smithi062 ceph-mon[32844]: pgmap v367: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:56:44.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:43 smithi079 ceph-mon[33002]: pgmap v367: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:56:45.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:45 smithi045 ceph-mon[27776]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:56:46.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:45 smithi062 ceph-mon[32844]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:56:46.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:45 smithi079 ceph-mon[33002]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:56:47.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:47 smithi045 ceph-mon[27776]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:56:48.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:47 smithi062 ceph-mon[32844]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:56:48.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:47 smithi079 ceph-mon[33002]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:56:49.773 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:49 smithi045 ceph-mon[27776]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:56:50.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:49 smithi062 ceph-mon[32844]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:56:50.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:49 smithi079 ceph-mon[33002]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:56:52.011 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:51 smithi045 ceph-mon[27776]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:56:52.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:51 smithi062 ceph-mon[32844]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:56:52.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:51 smithi079 ceph-mon[33002]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:56:53.778 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:53 smithi045 ceph-mon[27776]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:56:54.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:53 smithi062 ceph-mon[32844]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:56:54.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:53 smithi079 ceph-mon[33002]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:56:54.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.753+0000 7fd51bc00700 1 -- 172.21.15.45:0/2391990360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd514102a90 msgr2=0x7fd514102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:54.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.753+0000 7fd51bc00700 1 --2- 172.21.15.45:0/2391990360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd514102a90 0x7fd514102f10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd500009990 tx=0x7fd500009ca0 comp rx=0 tx=0).stop 2024-04-25T18:56:54.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.753+0000 7fd51bc00700 1 -- 172.21.15.45:0/2391990360 shutdown_connections 2024-04-25T18:56:54.912 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.753+0000 7fd51bc00700 1 --2- 172.21.15.45:0/2391990360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd514108a80 0x7fd514108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.753+0000 7fd51bc00700 1 --2- 172.21.15.45:0/2391990360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd514103450 0x7fd51410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.753+0000 7fd51bc00700 1 --2- 172.21.15.45:0/2391990360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd514102a90 0x7fd514102f10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.753+0000 7fd51bc00700 1 -- 172.21.15.45:0/2391990360 >> 172.21.15.45:0/2391990360 conn(0x7fd514079bf0 msgr2=0x7fd51407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.753+0000 7fd51bc00700 1 -- 172.21.15.45:0/2391990360 shutdown_connections 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.753+0000 7fd51bc00700 1 -- 172.21.15.45:0/2391990360 wait complete. 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.754+0000 7fd51bc00700 1 Processor -- start 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd51bc00700 1 -- start start 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd51bc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd514102a90 0x7fd514074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd51bc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd514103450 0x7fd514075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd51bc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd514108a80 0x7fd5140757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd51bc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd514112c10 con 0x7fd514108a80 2024-04-25T18:56:54.913 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd51bc00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd514112a90 con 0x7fd514102a90 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd51bc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd514112d90 con 0x7fd514103450 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd519200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd514102a90 0x7fd514074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd519200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd514102a90 0x7fd514074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:55896/0 (socket says 172.21.15.45:55896) 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd519200700 1 -- 172.21.15.45:0/892147206 learned_addr learned my addr 172.21.15.45:0/892147206 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.755+0000 7fd513e00700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd514103450 0x7fd514075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.756+0000 7fd519c00700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd514108a80 0x7fd5140757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.756+0000 7fd519200700 1 -- 172.21.15.45:0/892147206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd514103450 msgr2=0x7fd514075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.756+0000 7fd519200700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd514103450 0x7fd514075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.756+0000 7fd519200700 1 -- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd514108a80 msgr2=0x7fd5140757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.756+0000 7fd519200700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd514108a80 0x7fd5140757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.756+0000 7fd519200700 1 -- 172.21.15.45:0/892147206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5141abad0 con 0x7fd514102a90 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.756+0000 7fd513e00700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd514103450 0x7fd514075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.756+0000 7fd519200700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd514102a90 0x7fd514074d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd50800e9a0 tx=0x7fd50800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:54.914 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.756+0000 7fd511600700 1 -- 172.21.15.45:0/892147206 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd50800ca10 con 0x7fd514102a90 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.757+0000 7fd511600700 1 -- 172.21.15.45:0/892147206 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd50800cbb0 con 0x7fd514102a90 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.757+0000 7fd511600700 1 -- 172.21.15.45:0/892147206 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd508018680 con 0x7fd514102a90 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.757+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5141abd60 con 0x7fd514102a90 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.757+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5141ac1c0 con 0x7fd514102a90 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.758+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5140630d0 con 0x7fd514102a90 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.760+0000 7fd511600700 1 -- 172.21.15.45:0/892147206 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd508010430 con 0x7fd514102a90 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.761+0000 7fd511600700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd4fc0a3cd0 0x7fd4fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.761+0000 7fd511600700 1 -- 172.21.15.45:0/892147206 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fd5080d07a0 con 0x7fd514102a90 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.762+0000 7fd513e00700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd4fc0a3cd0 0x7fd4fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.762+0000 7fd511600700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] conn(0x7fd4fc0ad910 0x7fd4fc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.762+0000 7fd511600700 1 -- 172.21.15.45:0/892147206 --> [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd50800be50 con 0x7fd4fc0ad910 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.762+0000 7fd511600700 1 -- 172.21.15.45:0/892147206 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fd5080d0b50 con 0x7fd514102a90 2024-04-25T18:56:54.915 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.762+0000 7fd519c00700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] conn(0x7fd4fc0ad910 0x7fd4fc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.762+0000 7fd513e00700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd4fc0a3cd0 0x7fd4fc0a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd51419d260 tx=0x7fd500005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.763+0000 7fd519c00700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] conn(0x7fd4fc0ad910 0x7fd4fc0afdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.764+0000 7fd511600700 1 -- 172.21.15.45:0/892147206 <== osd.9 v2:172.21.15.45:6826/2318794689 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd50800be50 con 0x7fd4fc0ad910 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.782+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 --> [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd514063550 con 0x7fd4fc0ad910 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.783+0000 7fd511600700 1 -- 172.21.15.45:0/892147206 <== osd.9 v2:172.21.15.45:6826/2318794689 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd514063550 con 0x7fd4fc0ad910 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.783+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] conn(0x7fd4fc0ad910 msgr2=0x7fd4fc0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.783+0000 7fd51bc00700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] conn(0x7fd4fc0ad910 0x7fd4fc0afdb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.783+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd4fc0a3cd0 msgr2=0x7fd4fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.784+0000 7fd51bc00700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd4fc0a3cd0 0x7fd4fc0a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd51419d260 tx=0x7fd500005960 comp rx=0 tx=0).stop 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.784+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd514102a90 msgr2=0x7fd514074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.784+0000 7fd51bc00700 1 --2- 172.21.15.45:0/892147206 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd514102a90 0x7fd514074d40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd50800e9a0 tx=0x7fd50800ecb0 comp rx=0 tx=0).stop 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.784+0000 7fd519c00700 1 -- 172.21.15.45:0/892147206 reap_dead start 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.784+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 shutdown_connections 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.784+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 >> 172.21.15.45:0/892147206 conn(0x7fd514079bf0 msgr2=0x7fd514101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:54.916 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.784+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 shutdown_connections 2024-04-25T18:56:54.917 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.784+0000 7fd51bc00700 1 -- 172.21.15.45:0/892147206 wait complete. 2024-04-25T18:56:55.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.881+0000 7fd11ba00700 1 -- 172.21.15.45:0/1733665097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd114076f70 msgr2=0x7fd114077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:55.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.881+0000 7fd11ba00700 1 --2- 172.21.15.45:0/1733665097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd114076f70 0x7fd114077350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd1080098f0 tx=0x7fd108009c00 comp rx=0 tx=0).stop 2024-04-25T18:56:55.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.881+0000 7fd11ba00700 1 -- 172.21.15.45:0/1733665097 shutdown_connections 2024-04-25T18:56:55.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.881+0000 7fd11ba00700 1 --2- 172.21.15.45:0/1733665097 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd114113920 0x7fd114115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:55.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.881+0000 7fd11ba00700 1 --2- 172.21.15.45:0/1733665097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd114076f70 0x7fd114077350 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.881+0000 7fd11ba00700 1 --2- 172.21.15.45:0/1733665097 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd114077890 0x7fd1141133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.881+0000 7fd11ba00700 1 -- 172.21.15.45:0/1733665097 >> 172.21.15.45:0/1733665097 conn(0x7fd114100770 msgr2=0x7fd114102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.881+0000 7fd11ba00700 1 -- 172.21.15.45:0/1733665097 shutdown_connections 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.881+0000 7fd11ba00700 1 -- 172.21.15.45:0/1733665097 wait complete. 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.882+0000 7fd11ba00700 1 Processor -- start 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd11ba00700 1 -- start start 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd11ba00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd114076f70 0x7fd1141a5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd11ba00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd114077890 0x7fd1141a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd11ba00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd114113920 0x7fd11419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd11ba00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd114119310 con 0x7fd114077890 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd11ba00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd114119190 con 0x7fd114113920 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd11ba00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd114119490 con 0x7fd114076f70 2024-04-25T18:56:55.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd113e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd114077890 0x7fd1141a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd113e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd114077890 0x7fd1141a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51896/0 (socket says 172.21.15.45:51896) 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd119000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd114076f70 0x7fd1141a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.883+0000 7fd113e00700 1 -- 172.21.15.45:0/1472653852 learned_addr learned my addr 172.21.15.45:0/1472653852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.884+0000 7fd119a00700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd114113920 0x7fd11419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.884+0000 7fd113e00700 1 -- 172.21.15.45:0/1472653852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd114076f70 msgr2=0x7fd1141a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.884+0000 7fd113e00700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd114076f70 0x7fd1141a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.884+0000 7fd113e00700 1 -- 172.21.15.45:0/1472653852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd114113920 msgr2=0x7fd11419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.884+0000 7fd113e00700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd114113920 0x7fd11419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.884+0000 7fd113e00700 1 -- 172.21.15.45:0/1472653852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1141a0020 con 0x7fd114077890 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.884+0000 7fd113e00700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd114077890 0x7fd1141a5b60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd10000e9c0 tx=0x7fd10000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.884+0000 7fd111600700 1 -- 172.21.15.45:0/1472653852 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd10000cbd0 con 0x7fd114077890 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.884+0000 7fd111600700 1 -- 172.21.15.45:0/1472653852 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd10000ee60 con 0x7fd114077890 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.885+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1141a0310 con 0x7fd114077890 2024-04-25T18:56:55.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.885+0000 7fd111600700 1 -- 172.21.15.45:0/1472653852 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1000055c0 con 0x7fd114077890 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.885+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1141ac5a0 con 0x7fd114077890 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.885+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1140630d0 con 0x7fd114077890 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.888+0000 7fd111600700 1 -- 172.21.15.45:0/1472653852 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd100004750 con 0x7fd114077890 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.889+0000 7fd111600700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd0fc0a3cd0 0x7fd0fc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.889+0000 7fd111600700 1 -- 172.21.15.45:0/1472653852 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fd1000d0c70 con 0x7fd114077890 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.889+0000 7fd119000700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd0fc0a3cd0 0x7fd0fc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.890+0000 7fd111600700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] conn(0x7fd0fc0ad910 0x7fd0fc0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.890+0000 7fd111600700 1 -- 172.21.15.45:0/1472653852 --> [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd10001ee50 con 0x7fd0fc0ad910 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.890+0000 7fd111600700 1 -- 172.21.15.45:0/1472653852 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fd1000d1020 con 0x7fd114077890 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.890+0000 7fd119a00700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] conn(0x7fd0fc0ad910 0x7fd0fc0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.890+0000 7fd119000700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd0fc0a3cd0 0x7fd0fc0a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1080098c0 tx=0x7fd108019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.891+0000 7fd119a00700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] conn(0x7fd0fc0ad910 0x7fd0fc0afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:55.955 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.892+0000 7fd111600700 1 -- 172.21.15.45:0/1472653852 <== osd.8 v2:172.21.15.79:6816/2481950968 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd10001ee50 con 0x7fd0fc0ad910 2024-04-25T18:56:55.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.910+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 --> [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd114063550 con 0x7fd0fc0ad910 2024-04-25T18:56:55.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.911+0000 7fd111600700 1 -- 172.21.15.45:0/1472653852 <== osd.8 v2:172.21.15.79:6816/2481950968 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd114063550 con 0x7fd0fc0ad910 2024-04-25T18:56:55.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.911+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 >> [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] conn(0x7fd0fc0ad910 msgr2=0x7fd0fc0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:55.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.911+0000 7fd11ba00700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] conn(0x7fd0fc0ad910 0x7fd0fc0afdb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:55.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.911+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd0fc0a3cd0 msgr2=0x7fd0fc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:55.956 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.911+0000 7fd11ba00700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd0fc0a3cd0 0x7fd0fc0a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1080098c0 tx=0x7fd108019040 comp rx=0 tx=0).stop 2024-04-25T18:56:55.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.911+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd114077890 msgr2=0x7fd1141a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:55.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.911+0000 7fd11ba00700 1 --2- 172.21.15.45:0/1472653852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd114077890 0x7fd1141a5b60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd10000e9c0 tx=0x7fd10000ecd0 comp rx=0 tx=0).stop 2024-04-25T18:56:55.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.911+0000 7fd119a00700 1 -- 172.21.15.45:0/1472653852 reap_dead start 2024-04-25T18:56:55.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.912+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 shutdown_connections 2024-04-25T18:56:55.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.912+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 >> 172.21.15.45:0/1472653852 conn(0x7fd114100770 msgr2=0x7fd114102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:55.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.912+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 shutdown_connections 2024-04-25T18:56:55.957 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:54.912+0000 7fd11ba00700 1 -- 172.21.15.45:0/1472653852 wait complete. 2024-04-25T18:56:56.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:55 smithi062 ceph-mon[32844]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:56:56.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:55 smithi079 ceph-mon[33002]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:56:56.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:55 smithi045 ceph-mon[27776]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:56:57.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.443+0000 7f8494a00700 1 -- 172.21.15.45:0/2336581158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490102970 msgr2=0x7f8490102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:57.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.443+0000 7f8494a00700 1 --2- 172.21.15.45:0/2336581158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490102970 0x7f8490102df0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f847c009990 tx=0x7f847c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:56:57.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.443+0000 7f8494a00700 1 -- 172.21.15.45:0/2336581158 shutdown_connections 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.443+0000 7f8494a00700 1 --2- 172.21.15.45:0/2336581158 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490103330 0x7f849010f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.443+0000 7f8494a00700 1 --2- 172.21.15.45:0/2336581158 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490108970 0x7f8490108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.443+0000 7f8494a00700 1 --2- 172.21.15.45:0/2336581158 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490102970 0x7f8490102df0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.443+0000 7f8494a00700 1 -- 172.21.15.45:0/2336581158 >> 172.21.15.45:0/2336581158 conn(0x7f84900fe4c0 msgr2=0x7f84901008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.443+0000 7f8494a00700 1 -- 172.21.15.45:0/2336581158 shutdown_connections 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.443+0000 7f8494a00700 1 -- 172.21.15.45:0/2336581158 wait complete. 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.445+0000 7f8494a00700 1 Processor -- start 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.445+0000 7f8494a00700 1 -- start start 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f8494a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490102970 0x7f849019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f8494a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490103330 0x7f849019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f8494a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490108970 0x7f849019fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f8494a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8490112c00 con 0x7f8490103330 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f8494a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8490112a80 con 0x7f8490108970 2024-04-25T18:56:57.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f8494a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8490112d80 con 0x7f8490102970 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f848de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490103330 0x7f849019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f848e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490102970 0x7f849019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f848f200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490108970 0x7f849019fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f848e800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490102970 0x7f849019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48594/0 (socket says 172.21.15.45:48594) 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f848e800700 1 -- 172.21.15.45:0/840426336 learned_addr learned my addr 172.21.15.45:0/840426336 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.446+0000 7f848e800700 1 -- 172.21.15.45:0/840426336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490108970 msgr2=0x7f849019fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.447+0000 7f848e800700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8490108970 0x7f849019fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.447+0000 7f848e800700 1 -- 172.21.15.45:0/840426336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490103330 msgr2=0x7f849019b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.447+0000 7f848e800700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8490103330 0x7f849019b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.447+0000 7f848e800700 1 -- 172.21.15.45:0/840426336 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84901a0260 con 0x7f8490102970 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.447+0000 7f848e800700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490102970 0x7f849019b250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f848000e9c0 tx=0x7f848000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.447+0000 7f8487400700 1 -- 172.21.15.45:0/840426336 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848000cbd0 con 0x7f8490102970 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.447+0000 7f8487400700 1 -- 172.21.15.45:0/840426336 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f848000ee60 con 0x7f8490102970 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.447+0000 7f8487400700 1 -- 172.21.15.45:0/840426336 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84800055c0 con 0x7f8490102970 2024-04-25T18:56:57.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.447+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84901a0550 con 0x7f8490102970 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.448+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84901a7dd0 con 0x7f8490102970 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.449+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f84900ffd30 con 0x7f8490102970 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.451+0000 7f8487400700 1 -- 172.21.15.45:0/840426336 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8480004750 con 0x7f8490102970 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.452+0000 7f8487400700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f84740a3d50 0x7f84740a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.453+0000 7f8487400700 1 -- 172.21.15.45:0/840426336 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f84800d0760 con 0x7f8490102970 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.453+0000 7f848de00700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f84740a3d50 0x7f84740a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.453+0000 7f8487400700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] conn(0x7f84740ad990 0x7f84740afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.453+0000 7f8487400700 1 -- 172.21.15.45:0/840426336 --> [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84740b0500 con 0x7f84740ad990 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.453+0000 7f8487400700 1 -- 172.21.15.45:0/840426336 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f84800d7050 con 0x7f8490102970 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.454+0000 7f848f200700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] conn(0x7f84740ad990 0x7f84740afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.454+0000 7f848de00700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f84740a3d50 0x7f84740a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f849010fb30 tx=0x7f847c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.454+0000 7f848f200700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] conn(0x7f84740ad990 0x7f84740afe30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.455+0000 7f8487400700 1 -- 172.21.15.45:0/840426336 <== osd.5 v2:172.21.15.79:6808/188682646 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f84740b0500 con 0x7f84740ad990 2024-04-25T18:56:57.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.475+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 --> [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84901040b0 con 0x7f84740ad990 2024-04-25T18:56:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.476+0000 7f8487400700 1 -- 172.21.15.45:0/840426336 <== osd.5 v2:172.21.15.79:6808/188682646 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f84901040b0 con 0x7f84740ad990 2024-04-25T18:56:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.476+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 >> [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] conn(0x7f84740ad990 msgr2=0x7f84740afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.476+0000 7f8494a00700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] conn(0x7f84740ad990 0x7f84740afe30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:56:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.477+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f84740a3d50 msgr2=0x7f84740a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.477+0000 7f8494a00700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f84740a3d50 0x7f84740a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f849010fb30 tx=0x7f847c00b260 comp rx=0 tx=0).stop 2024-04-25T18:56:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.477+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490102970 msgr2=0x7f849019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:56:57.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.477+0000 7f8494a00700 1 --2- 172.21.15.45:0/840426336 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8490102970 0x7f849019b250 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f848000e9c0 tx=0x7f848000ecd0 comp rx=0 tx=0).stop 2024-04-25T18:56:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.477+0000 7f848f200700 1 -- 172.21.15.45:0/840426336 reap_dead start 2024-04-25T18:56:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.477+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 shutdown_connections 2024-04-25T18:56:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.477+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 >> 172.21.15.45:0/840426336 conn(0x7f84900fe4c0 msgr2=0x7f849010dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:56:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.477+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 shutdown_connections 2024-04-25T18:56:57.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:55.477+0000 7f8494a00700 1 -- 172.21.15.45:0/840426336 wait complete. 2024-04-25T18:56:58.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:57 smithi062 ceph-mon[32844]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:56:58.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:57 smithi079 ceph-mon[33002]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:56:58.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:57 smithi045 ceph-mon[27776]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:57:00.110 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:56:59 smithi045 ceph-mon[27776]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:00.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:56:59 smithi062 ceph-mon[32844]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:00.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:56:59 smithi079 ceph-mon[33002]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:00.693 INFO:teuthology.orchestra.run.smithi045.stdout:171798691909 2024-04-25T18:57:00.694 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T18:57:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.745+0000 7f6e0d400700 1 -- 172.21.15.45:0/232019347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e081020e0 msgr2=0x7f6e0810eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.745+0000 7f6e0d400700 1 --2- 172.21.15.45:0/232019347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e081020e0 0x7f6e0810eda0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009990 tx=0x7f6df4009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.745+0000 7f6e0d400700 1 -- 172.21.15.45:0/232019347 shutdown_connections 2024-04-25T18:57:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.745+0000 7f6e0d400700 1 --2- 172.21.15.45:0/232019347 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e08101730 0x7f6e08101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:00.925 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.745+0000 7f6e0d400700 1 --2- 172.21.15.45:0/232019347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e0810f2e0 0x7f6e081116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.745+0000 7f6e0d400700 1 --2- 172.21.15.45:0/232019347 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e081020e0 0x7f6e0810eda0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.745+0000 7f6e0d400700 1 -- 172.21.15.45:0/232019347 >> 172.21.15.45:0/232019347 conn(0x7f6e080fd3f0 msgr2=0x7f6e080ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.746+0000 7f6e0d400700 1 -- 172.21.15.45:0/232019347 shutdown_connections 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.746+0000 7f6e0d400700 1 -- 172.21.15.45:0/232019347 wait complete. 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.747+0000 7f6e0d400700 1 Processor -- start 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.747+0000 7f6e0d400700 1 -- start start 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.747+0000 7f6e0d400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e08101730 0x7f6e0819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.747+0000 7f6e0d400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e081020e0 0x7f6e0819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.747+0000 7f6e0d400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e0810f2e0 0x7f6e0819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.747+0000 7f6e0d400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08114de0 con 0x7f6e081020e0 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.747+0000 7f6e0d400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08114c60 con 0x7f6e0810f2e0 2024-04-25T18:57:00.926 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.747+0000 7f6e0d400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08114f60 con 0x7f6e08101730 2024-04-25T18:57:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e07400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e081020e0 0x7f6e0819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e07e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e08101730 0x7f6e0819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e07e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e08101730 0x7f6e0819b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48604/0 (socket says 172.21.15.45:48604) 2024-04-25T18:57:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e07e00700 1 -- 172.21.15.45:0/1861743146 learned_addr learned my addr 172.21.15.45:0/1861743146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e0ca00700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e0810f2e0 0x7f6e0819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e07e00700 1 -- 172.21.15.45:0/1861743146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e0810f2e0 msgr2=0x7f6e0819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e07e00700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6e0810f2e0 0x7f6e0819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:00.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e07e00700 1 -- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e081020e0 msgr2=0x7f6e0819b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e07e00700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6e081020e0 0x7f6e0819b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e07e00700 1 -- 172.21.15.45:0/1861743146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e081a02e0 con 0x7f6e08101730 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.748+0000 7f6e07e00700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e08101730 0x7f6e0819b2d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6df800e9a0 tx=0x7f6df800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.749+0000 7f6e04c00700 1 -- 172.21.15.45:0/1861743146 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df800ca10 con 0x7f6e08101730 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.749+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e081a05d0 con 0x7f6e08101730 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.749+0000 7f6e04c00700 1 -- 172.21.15.45:0/1861743146 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df800cbb0 con 0x7f6e08101730 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.749+0000 7f6e04c00700 1 -- 172.21.15.45:0/1861743146 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df8018680 con 0x7f6e08101730 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.749+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e081a7e50 con 0x7f6e08101730 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.750+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e080630d0 con 0x7f6e08101730 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.752+0000 7f6e04c00700 1 -- 172.21.15.45:0/1861743146 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6df8010430 con 0x7f6e08101730 2024-04-25T18:57:00.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.754+0000 7f6e04c00700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6dec0a3cd0 0x7f6dec0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.754+0000 7f6e04c00700 1 -- 172.21.15.45:0/1861743146 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f6df80d0cc0 con 0x7f6e08101730 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.754+0000 7f6e07400700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6dec0a3cd0 0x7f6dec0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.754+0000 7f6e04c00700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] conn(0x7f6dec0ad910 0x7f6dec0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.754+0000 7f6e04c00700 1 -- 172.21.15.45:0/1861743146 --> [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6df8016ec0 con 0x7f6dec0ad910 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.754+0000 7f6e04c00700 1 -- 172.21.15.45:0/1861743146 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f6df80d1110 con 0x7f6e08101730 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.755+0000 7f6e0ca00700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] conn(0x7f6dec0ad910 0x7f6dec0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.755+0000 7f6e07400700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6dec0a3cd0 0x7f6dec0a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6df40098e0 tx=0x7f6df4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.755+0000 7f6e0ca00700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] conn(0x7f6dec0ad910 0x7f6dec0afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.756+0000 7f6e04c00700 1 -- 172.21.15.45:0/1861743146 <== osd.6 v2:172.21.15.45:6818/401127899 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6df8016ec0 con 0x7f6dec0ad910 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.775+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 --> [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6e08063550 con 0x7f6dec0ad910 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.775+0000 7f6e04c00700 1 -- 172.21.15.45:0/1861743146 <== osd.6 v2:172.21.15.45:6818/401127899 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6e08063550 con 0x7f6dec0ad910 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] conn(0x7f6dec0ad910 msgr2=0x7f6dec0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:00.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0d400700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] conn(0x7f6dec0ad910 0x7f6dec0afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:00.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6dec0a3cd0 msgr2=0x7f6dec0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:00.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0d400700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f6dec0a3cd0 0x7f6dec0a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6df40098e0 tx=0x7f6df4005960 comp rx=0 tx=0).stop 2024-04-25T18:57:00.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e08101730 msgr2=0x7f6e0819b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:00.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0d400700 1 --2- 172.21.15.45:0/1861743146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6e08101730 0x7f6e0819b2d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6df800e9a0 tx=0x7f6df800ecb0 comp rx=0 tx=0).stop 2024-04-25T18:57:00.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0ca00700 1 -- 172.21.15.45:0/1861743146 reap_dead start 2024-04-25T18:57:00.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 shutdown_connections 2024-04-25T18:57:00.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 >> 172.21.15.45:0/1861743146 conn(0x7f6e080fd3f0 msgr2=0x7f6e080ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:00.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 shutdown_connections 2024-04-25T18:57:00.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:56:59.776+0000 7f6e0d400700 1 -- 172.21.15.45:0/1861743146 wait complete. 2024-04-25T18:57:02.010 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:01 smithi045 ceph-mon[27776]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T18:57:02.086 INFO:teuthology.orchestra.run.smithi045.stdout:300647710765 2024-04-25T18:57:02.087 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T18:57:02.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:01 smithi062 ceph-mon[32844]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T18:57:02.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:01 smithi079 ceph-mon[33002]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T18:57:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.306+0000 7f984ae00700 1 -- 172.21.15.45:0/2527843887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844104450 msgr2=0x7f984410c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:02.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.306+0000 7f984ae00700 1 --2- 172.21.15.45:0/2527843887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844104450 0x7f984410c940 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f98300098f0 tx=0x7f9830009c00 comp rx=0 tx=0).stop 2024-04-25T18:57:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.306+0000 7f984ae00700 1 -- 172.21.15.45:0/2527843887 shutdown_connections 2024-04-25T18:57:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.306+0000 7f984ae00700 1 --2- 172.21.15.45:0/2527843887 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f984410cf60 0x7f984410f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.306+0000 7f984ae00700 1 --2- 172.21.15.45:0/2527843887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9844103b30 0x7f9844103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.306+0000 7f984ae00700 1 --2- 172.21.15.45:0/2527843887 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9844104450 0x7f984410c940 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.306+0000 7f984ae00700 1 -- 172.21.15.45:0/2527843887 >> 172.21.15.45:0/2527843887 conn(0x7f98440fd360 msgr2=0x7f98440ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.306+0000 7f984ae00700 1 -- 172.21.15.45:0/2527843887 shutdown_connections 2024-04-25T18:57:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.306+0000 7f984ae00700 1 -- 172.21.15.45:0/2527843887 wait complete. 2024-04-25T18:57:02.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.307+0000 7f984ae00700 1 Processor -- start 2024-04-25T18:57:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f984ae00700 1 -- start start 2024-04-25T18:57:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f984ae00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9844103b30 0x7f984419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f984ae00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9844104450 0x7f984419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f984ae00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984410cf60 0x7f984419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:02.309 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f984ae00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9844112ba0 con 0x7f984410cf60 2024-04-25T18:57:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f984ae00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9844112a20 con 0x7f9844104450 2024-04-25T18:57:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f984ae00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9844112d20 con 0x7f9844103b30 2024-04-25T18:57:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f9849000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9844104450 0x7f984419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f984a400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984410cf60 0x7f984419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f984a400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984410cf60 0x7f984419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51968/0 (socket says 172.21.15.45:51968) 2024-04-25T18:57:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f9849000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9844104450 0x7f984419b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:55990/0 (socket says 172.21.15.45:55990) 2024-04-25T18:57:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f984a400700 1 -- 172.21.15.45:0/1445168184 learned_addr learned my addr 172.21.15.45:0/1445168184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:02.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.308+0000 7f9849a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9844103b30 0x7f984419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.309+0000 7f9849000700 1 -- 172.21.15.45:0/1445168184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9844103b30 msgr2=0x7f984419b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.309+0000 7f9849000700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9844103b30 0x7f984419b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.309+0000 7f9849000700 1 -- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984410cf60 msgr2=0x7f984419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.309+0000 7f9849000700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984410cf60 0x7f984419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.309+0000 7f9849000700 1 -- 172.21.15.45:0/1445168184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98441a0320 con 0x7f9844104450 2024-04-25T18:57:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.309+0000 7f9849a00700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9844103b30 0x7f984419b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:02.311 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.309+0000 7f984a400700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f984410cf60 0x7f984419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.309+0000 7f9849000700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9844104450 0x7f984419b850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9830000c00 tx=0x7f983000f950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.310+0000 7f9842000700 1 -- 172.21.15.45:0/1445168184 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830005040 con 0x7f9844104450 2024-04-25T18:57:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.310+0000 7f9842000700 1 -- 172.21.15.45:0/1445168184 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9830005690 con 0x7f9844104450 2024-04-25T18:57:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.310+0000 7f9842000700 1 -- 172.21.15.45:0/1445168184 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830003970 con 0x7f9844104450 2024-04-25T18:57:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.310+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98441a05b0 con 0x7f9844104450 2024-04-25T18:57:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.310+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a7e90 con 0x7f9844104450 2024-04-25T18:57:02.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.311+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9844103210 con 0x7f9844104450 2024-04-25T18:57:02.314 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.313+0000 7f9842000700 1 -- 172.21.15.45:0/1445168184 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f983000b780 con 0x7f9844104450 2024-04-25T18:57:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.314+0000 7f9842000700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f982c0a3d50 0x7f982c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:02.315 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.315+0000 7f9842000700 1 -- 172.21.15.45:0/1445168184 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f98300d1a30 con 0x7f9844104450 2024-04-25T18:57:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.315+0000 7f9849a00700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f982c0a3d50 0x7f982c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.315+0000 7f9842000700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] conn(0x7f982c0ad860 0x7f982c0afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:02.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.315+0000 7f9842000700 1 -- 172.21.15.45:0/1445168184 --> [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9830018e50 con 0x7f982c0ad860 2024-04-25T18:57:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.315+0000 7f9842000700 1 -- 172.21.15.45:0/1445168184 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f98300d1de0 con 0x7f9844104450 2024-04-25T18:57:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.315+0000 7f984a400700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] conn(0x7f982c0ad860 0x7f982c0afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.316+0000 7f9849a00700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f982c0a3d50 0x7f982c0a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9838011490 tx=0x7f9838009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:02.317 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.316+0000 7f984a400700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] conn(0x7f982c0ad860 0x7f982c0afd00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:02.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.317+0000 7f9842000700 1 -- 172.21.15.45:0/1445168184 <== osd.3 v2:172.21.15.45:6810/3944994121 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9830018e50 con 0x7f982c0ad860 2024-04-25T18:57:02.336 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.335+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 --> [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f984406b000 con 0x7f982c0ad860 2024-04-25T18:57:02.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.336+0000 7f9842000700 1 -- 172.21.15.45:0/1445168184 <== osd.3 v2:172.21.15.45:6810/3944994121 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f984406b000 con 0x7f982c0ad860 2024-04-25T18:57:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.336+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] conn(0x7f982c0ad860 msgr2=0x7f982c0afd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.336+0000 7f984ae00700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] conn(0x7f982c0ad860 0x7f982c0afd00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.336+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f982c0a3d50 msgr2=0x7f982c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.336+0000 7f984ae00700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f982c0a3d50 0x7f982c0a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9838011490 tx=0x7f9838009260 comp rx=0 tx=0).stop 2024-04-25T18:57:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.336+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9844104450 msgr2=0x7f984419b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.336+0000 7f984ae00700 1 --2- 172.21.15.45:0/1445168184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9844104450 0x7f984419b850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9830000c00 tx=0x7f983000f950 comp rx=0 tx=0).stop 2024-04-25T18:57:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.336+0000 7f984a400700 1 -- 172.21.15.45:0/1445168184 reap_dead start 2024-04-25T18:57:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.337+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 shutdown_connections 2024-04-25T18:57:02.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.337+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 >> 172.21.15.45:0/1445168184 conn(0x7f98440fd360 msgr2=0x7f984410e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:02.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.337+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 shutdown_connections 2024-04-25T18:57:02.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:02.337+0000 7f984ae00700 1 -- 172.21.15.45:0/1445168184 wait complete. 2024-04-25T18:57:03.586 INFO:teuthology.orchestra.run.smithi045.stdout:206158430271 2024-04-25T18:57:03.586 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T18:57:03.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.500+0000 7fe704000700 1 -- 172.21.15.45:0/359200745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6fc06b980 msgr2=0x7fe6fc10f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.500+0000 7fe704000700 1 --2- 172.21.15.45:0/359200745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6fc06b980 0x7fe6fc10f8f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe6f400b0a0 tx=0x7fe6f400b3b0 comp rx=0 tx=0).stop 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.500+0000 7fe704000700 1 -- 172.21.15.45:0/359200745 shutdown_connections 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.500+0000 7fe704000700 1 --2- 172.21.15.45:0/359200745 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6fc06b980 0x7fe6fc10f8f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.500+0000 7fe704000700 1 --2- 172.21.15.45:0/359200745 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6fc108a20 0x7fe6fc108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.500+0000 7fe704000700 1 --2- 172.21.15.45:0/359200745 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6fc06afe0 0x7fe6fc06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.500+0000 7fe704000700 1 -- 172.21.15.45:0/359200745 >> 172.21.15.45:0/359200745 conn(0x7fe6fc0788b0 msgr2=0x7fe6fc078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.500+0000 7fe704000700 1 -- 172.21.15.45:0/359200745 shutdown_connections 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.500+0000 7fe704000700 1 -- 172.21.15.45:0/359200745 wait complete. 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.501+0000 7fe704000700 1 Processor -- start 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.501+0000 7fe704000700 1 -- start start 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe704000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6fc06afe0 0x7fe6fc19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:03.661 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe704000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6fc06b980 0x7fe6fc19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe704000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6fc108a20 0x7fe6fc19fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe704000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc112bc0 con 0x7fe6fc06b980 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe704000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc112a40 con 0x7fe6fc108a20 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe704000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe6fc112d40 con 0x7fe6fc06afe0 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe700c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6fc06b980 0x7fe6fc19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe700c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6fc06b980 0x7fe6fc19b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51954/0 (socket says 172.21.15.45:51954) 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe701600700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6fc06afe0 0x7fe6fc19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe700c00700 1 -- 172.21.15.45:0/3989966797 learned_addr learned my addr 172.21.15.45:0/3989966797 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe702000700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6fc108a20 0x7fe6fc19fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe700c00700 1 -- 172.21.15.45:0/3989966797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6fc06afe0 msgr2=0x7fe6fc19b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe700c00700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe6fc06afe0 0x7fe6fc19b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe700c00700 1 -- 172.21.15.45:0/3989966797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6fc108a20 msgr2=0x7fe6fc19fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:03.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe700c00700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe6fc108a20 0x7fe6fc19fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.502+0000 7fe700c00700 1 -- 172.21.15.45:0/3989966797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6fc1a02a0 con 0x7fe6fc06b980 2024-04-25T18:57:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.503+0000 7fe700c00700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6fc06b980 0x7fe6fc19b7d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800e9a0 tx=0x7fe6e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.503+0000 7fe6fa000700 1 -- 172.21.15.45:0/3989966797 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e800ca10 con 0x7fe6fc06b980 2024-04-25T18:57:03.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.503+0000 7fe6fa000700 1 -- 172.21.15.45:0/3989966797 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e800cbb0 con 0x7fe6fc06b980 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.503+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6fc1a0590 con 0x7fe6fc06b980 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.503+0000 7fe6fa000700 1 -- 172.21.15.45:0/3989966797 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8018680 con 0x7fe6fc06b980 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.503+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6fc1a7e10 con 0x7fe6fc06b980 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.504+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6fc0631d0 con 0x7fe6fc06b980 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.506+0000 7fe6fa000700 1 -- 172.21.15.45:0/3989966797 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe6e8010430 con 0x7fe6fc06b980 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.508+0000 7fe6fa000700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe6e40a3cd0 0x7fe6e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.508+0000 7fe6fa000700 1 -- 172.21.15.45:0/3989966797 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fe6e80d0cd0 con 0x7fe6fc06b980 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.508+0000 7fe701600700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe6e40a3cd0 0x7fe6e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.508+0000 7fe6fa000700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] conn(0x7fe6e40ad910 0x7fe6e40afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.508+0000 7fe6fa000700 1 -- 172.21.15.45:0/3989966797 --> [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6e8019ec0 con 0x7fe6e40ad910 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.508+0000 7fe6fa000700 1 -- 172.21.15.45:0/3989966797 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fe6e80d1080 con 0x7fe6fc06b980 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.509+0000 7fe702000700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] conn(0x7fe6e40ad910 0x7fe6e40afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:03.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.509+0000 7fe701600700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe6e40a3cd0 0x7fe6e40a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009be0 tx=0x7fe6f0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.509+0000 7fe702000700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] conn(0x7fe6e40ad910 0x7fe6e40afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.510+0000 7fe6fa000700 1 -- 172.21.15.45:0/3989966797 <== osd.0 v2:172.21.15.45:6802/4137498586 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6e8019ec0 con 0x7fe6e40ad910 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.529+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 --> [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe6fc063650 con 0x7fe6e40ad910 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.530+0000 7fe6fa000700 1 -- 172.21.15.45:0/3989966797 <== osd.0 v2:172.21.15.45:6802/4137498586 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe6fc063650 con 0x7fe6e40ad910 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.530+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] conn(0x7fe6e40ad910 msgr2=0x7fe6e40afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.530+0000 7fe704000700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] conn(0x7fe6e40ad910 0x7fe6e40afdb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.530+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe6e40a3cd0 msgr2=0x7fe6e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.530+0000 7fe704000700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe6e40a3cd0 0x7fe6e40a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0009be0 tx=0x7fe6f0009260 comp rx=0 tx=0).stop 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.530+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6fc06b980 msgr2=0x7fe6fc19b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.530+0000 7fe704000700 1 --2- 172.21.15.45:0/3989966797 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6fc06b980 0x7fe6fc19b7d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800e9a0 tx=0x7fe6e800ecb0 comp rx=0 tx=0).stop 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.531+0000 7fe702000700 1 -- 172.21.15.45:0/3989966797 reap_dead start 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.531+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 shutdown_connections 2024-04-25T18:57:03.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.531+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 >> 172.21.15.45:0/3989966797 conn(0x7fe6fc0788b0 msgr2=0x7fe6fc10de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.531+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 shutdown_connections 2024-04-25T18:57:03.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:00.531+0000 7fe704000700 1 -- 172.21.15.45:0/3989966797 wait complete. 2024-04-25T18:57:04.013 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:03 smithi045 ceph-mon[27776]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-25T18:57:04.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:03 smithi062 ceph-mon[32844]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-25T18:57:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:03 smithi079 ceph-mon[33002]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-04-25T18:57:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.595+0000 7f0498e00700 1 -- 172.21.15.45:0/2685782041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0494077890 msgr2=0x7f0494113390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.595+0000 7f0498e00700 1 --2- 172.21.15.45:0/2685782041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0494077890 0x7f0494113390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f047c009990 tx=0x7f047c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:04.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.595+0000 7f0498e00700 1 -- 172.21.15.45:0/2685782041 shutdown_connections 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.595+0000 7f0498e00700 1 --2- 172.21.15.45:0/2685782041 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f04941138d0 0x7f0494115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.595+0000 7f0498e00700 1 --2- 172.21.15.45:0/2685782041 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0494076f70 0x7f0494077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.595+0000 7f0498e00700 1 --2- 172.21.15.45:0/2685782041 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0494077890 0x7f0494113390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.595+0000 7f0498e00700 1 -- 172.21.15.45:0/2685782041 >> 172.21.15.45:0/2685782041 conn(0x7f0494100720 msgr2=0x7f0494102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.595+0000 7f0498e00700 1 -- 172.21.15.45:0/2685782041 shutdown_connections 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.595+0000 7f0498e00700 1 -- 172.21.15.45:0/2685782041 wait complete. 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.597+0000 7f0498e00700 1 Processor -- start 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.597+0000 7f0498e00700 1 -- start start 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.597+0000 7f0498e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0494076f70 0x7f04941a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.597+0000 7f0498e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0494077890 0x7f04941a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.597+0000 7f0498e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04941138d0 0x7f049419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.597+0000 7f0498e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f04941192c0 con 0x7f04941138d0 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.597+0000 7f0498e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0494119140 con 0x7f0494077890 2024-04-25T18:57:04.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.597+0000 7f0498e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0494119440 con 0x7f0494076f70 2024-04-25T18:57:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.597+0000 7f0491e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0494076f70 0x7f04941a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.597+0000 7f0492800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04941138d0 0x7f049419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.598+0000 7f0492800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04941138d0 0x7f049419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44286/0 (socket says 172.21.15.45:44286) 2024-04-25T18:57:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.598+0000 7f0492800700 1 -- 172.21.15.45:0/1888466462 learned_addr learned my addr 172.21.15.45:0/1888466462 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.598+0000 7f0491e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0494076f70 0x7f04941a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56304/0 (socket says 172.21.15.45:56304) 2024-04-25T18:57:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.598+0000 7f048be00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0494077890 0x7f04941a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:04.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.598+0000 7f0491e00700 1 -- 172.21.15.45:0/1888466462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0494077890 msgr2=0x7f04941a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.598+0000 7f0491e00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0494077890 0x7f04941a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.598+0000 7f0491e00700 1 -- 172.21.15.45:0/1888466462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04941138d0 msgr2=0x7f049419f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.598+0000 7f0491e00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f04941138d0 0x7f049419f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.598+0000 7f0491e00700 1 -- 172.21.15.45:0/1888466462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049419ffd0 con 0x7f0494076f70 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.599+0000 7f0491e00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0494076f70 0x7f04941a55d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f048400e9f0 tx=0x7f048400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.599+0000 7f048aa00700 1 -- 172.21.15.45:0/1888466462 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484004220 con 0x7f0494076f70 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.599+0000 7f048aa00700 1 -- 172.21.15.45:0/1888466462 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04840043c0 con 0x7f0494076f70 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.599+0000 7f048aa00700 1 -- 172.21.15.45:0/1888466462 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0484005020 con 0x7f0494076f70 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.599+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04941a02c0 con 0x7f0494076f70 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.599+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04941ac550 con 0x7f0494076f70 2024-04-25T18:57:04.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.601+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0494063110 con 0x7f0494076f70 2024-04-25T18:57:04.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.603+0000 7f048aa00700 1 -- 172.21.15.45:0/1888466462 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0484004750 con 0x7f0494076f70 2024-04-25T18:57:04.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.604+0000 7f048aa00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f04780a3d50 0x7f04780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:04.605 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.604+0000 7f048aa00700 1 -- 172.21.15.45:0/1888466462 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f04840d04f0 con 0x7f0494076f70 2024-04-25T18:57:04.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.604+0000 7f048be00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f04780a3d50 0x7f04780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:04.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.605+0000 7f048aa00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] conn(0x7f04780ad990 0x7f04780afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:04.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.605+0000 7f048aa00700 1 -- 172.21.15.45:0/1888466462 --> [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f04780b0500 con 0x7f04780ad990 2024-04-25T18:57:04.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.605+0000 7f048aa00700 1 -- 172.21.15.45:0/1888466462 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f04840d7050 con 0x7f0494076f70 2024-04-25T18:57:04.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.605+0000 7f0492800700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] conn(0x7f04780ad990 0x7f04780afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:04.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.605+0000 7f048be00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f04780a3d50 0x7f04780a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f04941a0e60 tx=0x7f047c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:04.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.606+0000 7f0492800700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] conn(0x7f04780ad990 0x7f04780afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:04.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.607+0000 7f048aa00700 1 -- 172.21.15.45:0/1888466462 <== osd.1 v2:172.21.15.62:6800/237724307 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f04780b0500 con 0x7f04780ad990 2024-04-25T18:57:04.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.626+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 --> [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0494063590 con 0x7f04780ad990 2024-04-25T18:57:04.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.626+0000 7f048aa00700 1 -- 172.21.15.45:0/1888466462 <== osd.1 v2:172.21.15.62:6800/237724307 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0494063590 con 0x7f04780ad990 2024-04-25T18:57:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 >> [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] conn(0x7f04780ad990 msgr2=0x7f04780afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0498e00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] conn(0x7f04780ad990 0x7f04780afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f04780a3d50 msgr2=0x7f04780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0498e00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f04780a3d50 0x7f04780a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f04941a0e60 tx=0x7f047c005fd0 comp rx=0 tx=0).stop 2024-04-25T18:57:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0494076f70 msgr2=0x7f04941a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0498e00700 1 --2- 172.21.15.45:0/1888466462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0494076f70 0x7f04941a55d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f048400e9f0 tx=0x7f048400ed00 comp rx=0 tx=0).stop 2024-04-25T18:57:04.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0492800700 1 -- 172.21.15.45:0/1888466462 reap_dead start 2024-04-25T18:57:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 shutdown_connections 2024-04-25T18:57:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 >> 172.21.15.45:0/1888466462 conn(0x7f0494100720 msgr2=0x7f0494102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 shutdown_connections 2024-04-25T18:57:04.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:04.627+0000 7f0498e00700 1 -- 172.21.15.45:0/1888466462 wait complete. 2024-04-25T18:57:05.906 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:05 smithi045 ceph-mon[27776]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T18:57:06.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:05 smithi062 ceph-mon[32844]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T18:57:06.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:05 smithi079 ceph-mon[33002]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T18:57:06.207 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.206+0000 7fbe37200700 1 -- 172.21.15.45:0/1625866012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe38107d40 msgr2=0x7fbe3810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.206+0000 7fbe37200700 1 --2- 172.21.15.45:0/1625866012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe38107d40 0x7fbe3810f8f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009f90 tx=0x7fbe3000b330 comp rx=0 tx=0).stop 2024-04-25T18:57:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.206+0000 7fbe37200700 1 -- 172.21.15.45:0/1625866012 shutdown_connections 2024-04-25T18:57:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.206+0000 7fbe37200700 1 --2- 172.21.15.45:0/1625866012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe38107d40 0x7fbe3810f8f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.206+0000 7fbe37200700 1 --2- 172.21.15.45:0/1625866012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe381014d0 0x7fbe381018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.206+0000 7fbe37200700 1 --2- 172.21.15.45:0/1625866012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe38101e80 0x7fbe38107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.206+0000 7fbe37200700 1 -- 172.21.15.45:0/1625866012 >> 172.21.15.45:0/1625866012 conn(0x7fbe38076bf0 msgr2=0x7fbe38077000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.206+0000 7fbe37200700 1 -- 172.21.15.45:0/1625866012 shutdown_connections 2024-04-25T18:57:06.208 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.206+0000 7fbe37200700 1 -- 172.21.15.45:0/1625866012 wait complete. 2024-04-25T18:57:06.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.207+0000 7fbe37200700 1 Processor -- start 2024-04-25T18:57:06.209 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.208+0000 7fbe37200700 1 -- start start 2024-04-25T18:57:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.208+0000 7fbe37200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe381014d0 0x7fbe38102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.208+0000 7fbe37200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe38101e80 0x7fbe38102990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.208+0000 7fbe37200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe38107d40 0x7fbe38102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.208+0000 7fbe37200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38112c40 con 0x7fbe38107d40 2024-04-25T18:57:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.208+0000 7fbe37200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38112ac0 con 0x7fbe381014d0 2024-04-25T18:57:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.208+0000 7fbe37200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38112dc0 con 0x7fbe38101e80 2024-04-25T18:57:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe35400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe38101e80 0x7fbe38102990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:06.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe35e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe381014d0 0x7fbe38102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe35400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe38101e80 0x7fbe38102990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56334/0 (socket says 172.21.15.45:56334) 2024-04-25T18:57:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe35400700 1 -- 172.21.15.45:0/188137692 learned_addr learned my addr 172.21.15.45:0/188137692 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe36800700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe38107d40 0x7fbe38102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe35e00700 1 -- 172.21.15.45:0/188137692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe38101e80 msgr2=0x7fbe38102990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe35e00700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe38101e80 0x7fbe38102990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe35e00700 1 -- 172.21.15.45:0/188137692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe38107d40 msgr2=0x7fbe38102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:06.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe35e00700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe38107d40 0x7fbe38102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe35e00700 1 -- 172.21.15.45:0/188137692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe381a7540 con 0x7fbe381014d0 2024-04-25T18:57:06.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe36800700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbe38107d40 0x7fbe38102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:57:06.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.209+0000 7fbe35e00700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe381014d0 0x7fbe38102450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800eb90 tx=0x7fbe2800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:06.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.210+0000 7fbe2ea00700 1 -- 172.21.15.45:0/188137692 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800ccf0 con 0x7fbe381014d0 2024-04-25T18:57:06.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.210+0000 7fbe2ea00700 1 -- 172.21.15.45:0/188137692 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe280044d0 con 0x7fbe381014d0 2024-04-25T18:57:06.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.210+0000 7fbe2ea00700 1 -- 172.21.15.45:0/188137692 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28010430 con 0x7fbe381014d0 2024-04-25T18:57:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.210+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe381a7830 con 0x7fbe381014d0 2024-04-25T18:57:06.214 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.210+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe381a7d20 con 0x7fbe381014d0 2024-04-25T18:57:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.211+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe380ffbc0 con 0x7fbe381014d0 2024-04-25T18:57:06.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.213+0000 7fbe2ea00700 1 -- 172.21.15.45:0/188137692 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbe28003680 con 0x7fbe381014d0 2024-04-25T18:57:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.214+0000 7fbe2ea00700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbe1c0a3d50 0x7fbe1c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.215+0000 7fbe2ea00700 1 -- 172.21.15.45:0/188137692 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fbe280d0af0 con 0x7fbe381014d0 2024-04-25T18:57:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.215+0000 7fbe35400700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbe1c0a3d50 0x7fbe1c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:06.217 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.215+0000 7fbe2ea00700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] conn(0x7fbe1c0ad990 0x7fbe1c0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.215+0000 7fbe2ea00700 1 -- 172.21.15.45:0/188137692 --> [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe2801aec0 con 0x7fbe1c0ad990 2024-04-25T18:57:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.215+0000 7fbe2ea00700 1 -- 172.21.15.45:0/188137692 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fbe280d0ea0 con 0x7fbe381014d0 2024-04-25T18:57:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.215+0000 7fbe36800700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] conn(0x7fbe1c0ad990 0x7fbe1c0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.216+0000 7fbe35400700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbe1c0a3d50 0x7fbe1c0a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009680 tx=0x7fbe24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:06.218 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.216+0000 7fbe36800700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] conn(0x7fbe1c0ad990 0x7fbe1c0afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:06.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.217+0000 7fbe2ea00700 1 -- 172.21.15.45:0/188137692 <== osd.7 v2:172.21.15.62:6816/3708428772 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe2801aec0 con 0x7fbe1c0ad990 2024-04-25T18:57:06.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.236+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 --> [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe3806a2b0 con 0x7fbe1c0ad990 2024-04-25T18:57:06.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.236+0000 7fbe2ea00700 1 -- 172.21.15.45:0/188137692 <== osd.7 v2:172.21.15.62:6816/3708428772 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe3806a2b0 con 0x7fbe1c0ad990 2024-04-25T18:57:06.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 >> [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] conn(0x7fbe1c0ad990 msgr2=0x7fbe1c0afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:06.238 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe37200700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] conn(0x7fbe1c0ad990 0x7fbe1c0afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbe1c0a3d50 msgr2=0x7fbe1c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe37200700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbe1c0a3d50 0x7fbe1c0a6210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009680 tx=0x7fbe24009260 comp rx=0 tx=0).stop 2024-04-25T18:57:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe381014d0 msgr2=0x7fbe38102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe37200700 1 --2- 172.21.15.45:0/188137692 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbe381014d0 0x7fbe38102450 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800eb90 tx=0x7fbe2800eea0 comp rx=0 tx=0).stop 2024-04-25T18:57:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe36800700 1 -- 172.21.15.45:0/188137692 reap_dead start 2024-04-25T18:57:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 shutdown_connections 2024-04-25T18:57:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 >> 172.21.15.45:0/188137692 conn(0x7fbe38076bf0 msgr2=0x7fbe380ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 shutdown_connections 2024-04-25T18:57:06.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:06.237+0000 7fbe37200700 1 -- 172.21.15.45:0/188137692 wait complete. 2024-04-25T18:57:07.192 INFO:teuthology.orchestra.run.smithi045.stdout:115964117072 2024-04-25T18:57:07.192 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T18:57:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.417+0000 7f7165c00700 1 -- 172.21.15.45:0/274043837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601034b0 msgr2=0x7f716010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.417+0000 7f7165c00700 1 --2- 172.21.15.45:0/274043837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601034b0 0x7f716010f920 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f715800b0d0 tx=0x7f715800b3e0 comp rx=0 tx=0).stop 2024-04-25T18:57:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.417+0000 7f7165c00700 1 -- 172.21.15.45:0/274043837 shutdown_connections 2024-04-25T18:57:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.417+0000 7f7165c00700 1 --2- 172.21.15.45:0/274043837 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601034b0 0x7f716010f920 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.417+0000 7f7165c00700 1 --2- 172.21.15.45:0/274043837 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7160107940 0x7f7160107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.417+0000 7f7165c00700 1 --2- 172.21.15.45:0/274043837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7160102af0 0x7f7160102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.417+0000 7f7165c00700 1 -- 172.21.15.45:0/274043837 >> 172.21.15.45:0/274043837 conn(0x7f7160079ba0 msgr2=0x7f7160079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.417+0000 7f7165c00700 1 -- 172.21.15.45:0/274043837 shutdown_connections 2024-04-25T18:57:07.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.417+0000 7f7165c00700 1 -- 172.21.15.45:0/274043837 wait complete. 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.418+0000 7f7165c00700 1 Processor -- start 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.418+0000 7f7165c00700 1 -- start start 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f7165c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7160102af0 0x7f716019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f7165c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601034b0 0x7f716019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f7165c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7160107940 0x7f716019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f7165c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7160112c60 con 0x7f7160107940 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f7165c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7160112ae0 con 0x7f71601034b0 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f7165c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7160112de0 con 0x7f7160102af0 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f715f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7160107940 0x7f716019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f715ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7160102af0 0x7f716019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f715f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7160107940 0x7f716019fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44294/0 (socket says 172.21.15.45:44294) 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f715f400700 1 -- 172.21.15.45:0/2520015496 learned_addr learned my addr 172.21.15.45:0/2520015496 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f715ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7160102af0 0x7f716019b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56320/0 (socket says 172.21.15.45:56320) 2024-04-25T18:57:07.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.419+0000 7f715e000700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601034b0 0x7f716019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f715f400700 1 -- 172.21.15.45:0/2520015496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7160102af0 msgr2=0x7f716019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f715f400700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7160102af0 0x7f716019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f715f400700 1 -- 172.21.15.45:0/2520015496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601034b0 msgr2=0x7f716019b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f715f400700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601034b0 0x7f716019b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f715f400700 1 -- 172.21.15.45:0/2520015496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71601a0330 con 0x7f7160107940 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f715ea00700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7160102af0 0x7f716019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f715e000700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f71601034b0 0x7f716019b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f715f400700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7160107940 0x7f716019fbf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7158007a20 tx=0x7f7158007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f7157400700 1 -- 172.21.15.45:0/2520015496 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715801f070 con 0x7f7160107940 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f7157400700 1 -- 172.21.15.45:0/2520015496 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7158004c50 con 0x7f7160107940 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71601a05c0 con 0x7f7160107940 2024-04-25T18:57:07.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f7157400700 1 -- 172.21.15.45:0/2520015496 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715801b430 con 0x7f7160107940 2024-04-25T18:57:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.420+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71601a7ea0 con 0x7f7160107940 2024-04-25T18:57:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.421+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7160101300 con 0x7f7160107940 2024-04-25T18:57:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.424+0000 7f7157400700 1 -- 172.21.15.45:0/2520015496 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7158022070 con 0x7f7160107940 2024-04-25T18:57:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.425+0000 7f7157400700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f71440a3cd0 0x7f71440a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.425+0000 7f7157400700 1 -- 172.21.15.45:0/2520015496 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f715800f020 con 0x7f7160107940 2024-04-25T18:57:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.425+0000 7f715ea00700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f71440a3cd0 0x7f71440a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.425+0000 7f7157400700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] conn(0x7f71440ad7e0 0x7f71440afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:07.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.425+0000 7f7157400700 1 -- 172.21.15.45:0/2520015496 --> [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71440b0350 con 0x7f71440ad7e0 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.425+0000 7f7157400700 1 -- 172.21.15.45:0/2520015496 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f71580d1b00 con 0x7f7160107940 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.426+0000 7f715e000700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] conn(0x7f71440ad7e0 0x7f71440afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.426+0000 7f715ea00700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f71440a3cd0 0x7f71440a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7150009fd0 tx=0x7f7150009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.426+0000 7f715e000700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] conn(0x7f71440ad7e0 0x7f71440afc80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.427+0000 7f7157400700 1 -- 172.21.15.45:0/2520015496 <== osd.10 v2:172.21.15.62:6824/2698516716 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71440b0350 con 0x7f71440ad7e0 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.446+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 --> [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f71601041b0 con 0x7f71440ad7e0 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.447+0000 7f7157400700 1 -- 172.21.15.45:0/2520015496 <== osd.10 v2:172.21.15.62:6824/2698516716 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f71601041b0 con 0x7f71440ad7e0 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.447+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 >> [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] conn(0x7f71440ad7e0 msgr2=0x7f71440afc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.447+0000 7f7165c00700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] conn(0x7f71440ad7e0 0x7f71440afc80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.448+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f71440a3cd0 msgr2=0x7f71440a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.448+0000 7f7165c00700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f71440a3cd0 0x7f71440a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7150009fd0 tx=0x7f7150009260 comp rx=0 tx=0).stop 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.448+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7160107940 msgr2=0x7f716019fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.448+0000 7f7165c00700 1 --2- 172.21.15.45:0/2520015496 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7160107940 0x7f716019fbf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7158007a20 tx=0x7f7158007a50 comp rx=0 tx=0).stop 2024-04-25T18:57:07.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.448+0000 7f715f400700 1 -- 172.21.15.45:0/2520015496 reap_dead start 2024-04-25T18:57:07.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.448+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 shutdown_connections 2024-04-25T18:57:07.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.448+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 >> 172.21.15.45:0/2520015496 conn(0x7f7160079ba0 msgr2=0x7f7160100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:07.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.448+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 shutdown_connections 2024-04-25T18:57:07.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.448+0000 7f7165c00700 1 -- 172.21.15.45:0/2520015496 wait complete. 2024-04-25T18:57:07.983 INFO:teuthology.orchestra.run.smithi045.stdout:55834574940 2024-04-25T18:57:07.983 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T18:57:08.050 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:07 smithi045 ceph-mon[27776]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-04-25T18:57:08.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:07 smithi062 ceph-mon[32844]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-04-25T18:57:08.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:07 smithi079 ceph-mon[33002]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 7 op/s 2024-04-25T18:57:08.493 INFO:teuthology.orchestra.run.smithi045.stdout:266287972405 2024-04-25T18:57:08.494 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T18:57:08.886 INFO:teuthology.orchestra.run.smithi045.stdout:326417514540 2024-04-25T18:57:08.886 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T18:57:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.366+0000 7f3cf9200700 1 -- 172.21.15.45:0/3169538501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4074c70 msgr2=0x7f3cf4075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.366+0000 7f3cf9200700 1 --2- 172.21.15.45:0/3169538501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4074c70 0x7f3cf4075050 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3ce40098f0 tx=0x7f3ce4009c00 comp rx=0 tx=0).stop 2024-04-25T18:57:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.366+0000 7f3cf9200700 1 -- 172.21.15.45:0/3169538501 shutdown_connections 2024-04-25T18:57:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.366+0000 7f3cf9200700 1 --2- 172.21.15.45:0/3169538501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf410f440 0x7f3cf4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.366+0000 7f3cf9200700 1 --2- 172.21.15.45:0/3169538501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4074c70 0x7f3cf4075050 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.366+0000 7f3cf9200700 1 --2- 172.21.15.45:0/3169538501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf4075590 0x7f3cf410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.366+0000 7f3cf9200700 1 -- 172.21.15.45:0/3169538501 >> 172.21.15.45:0/3169538501 conn(0x7f3cf4079ba0 msgr2=0x7f3cf4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.366+0000 7f3cf9200700 1 -- 172.21.15.45:0/3169538501 shutdown_connections 2024-04-25T18:57:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.366+0000 7f3cf9200700 1 -- 172.21.15.45:0/3169538501 wait complete. 2024-04-25T18:57:08.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.368+0000 7f3cf9200700 1 Processor -- start 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.368+0000 7f3cf9200700 1 -- start start 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.368+0000 7f3cf9200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4074c70 0x7f3cf419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.368+0000 7f3cf9200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4075590 0x7f3cf419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.368+0000 7f3cf9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf410f440 0x7f3cf419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.368+0000 7f3cf9200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4114e40 con 0x7f3cf410f440 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.368+0000 7f3cf9200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4114cc0 con 0x7f3cf4075590 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.368+0000 7f3cf9200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4114fc0 con 0x7f3cf4074c70 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.368+0000 7f3cf1e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4074c70 0x7f3cf419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.369+0000 7f3cf1e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4074c70 0x7f3cf419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56378/0 (socket says 172.21.15.45:56378) 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.369+0000 7f3cf1e00700 1 -- 172.21.15.45:0/2508223310 learned_addr learned my addr 172.21.15.45:0/2508223310 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.369+0000 7f3cf1400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4075590 0x7f3cf419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.369+0000 7f3cf2800700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf410f440 0x7f3cf419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.369+0000 7f3cf1e00700 1 -- 172.21.15.45:0/2508223310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4075590 msgr2=0x7f3cf419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.369+0000 7f3cf1e00700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cf4075590 0x7f3cf419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:08.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.369+0000 7f3cf1e00700 1 -- 172.21.15.45:0/2508223310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf410f440 msgr2=0x7f3cf419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.369+0000 7f3cf1e00700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cf410f440 0x7f3cf419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.369+0000 7f3cf1e00700 1 -- 172.21.15.45:0/2508223310 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf41a0380 con 0x7f3cf4074c70 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.369+0000 7f3cf1e00700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4074c70 0x7f3cf419b370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009230 tx=0x7f3ce400fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.370+0000 7f3ceaa00700 1 -- 172.21.15.45:0/2508223310 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce40050c0 con 0x7f3cf4074c70 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.370+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf41a0610 con 0x7f3cf4074c70 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.370+0000 7f3ceaa00700 1 -- 172.21.15.45:0/2508223310 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ce400b560 con 0x7f3cf4074c70 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.370+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf41a7ef0 con 0x7f3cf4074c70 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.370+0000 7f3ceaa00700 1 -- 172.21.15.45:0/2508223310 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce4003a40 con 0x7f3cf4074c70 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.371+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3cf4063150 con 0x7f3cf4074c70 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.373+0000 7f3ceaa00700 1 -- 172.21.15.45:0/2508223310 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3ce400b9d0 con 0x7f3cf4074c70 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.374+0000 7f3ceaa00700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3cd80a3d50 0x7f3cd80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.374+0000 7f3ceaa00700 1 -- 172.21.15.45:0/2508223310 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f3ce40d1900 con 0x7f3cf4074c70 2024-04-25T18:57:08.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.375+0000 7f3cf1400700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3cd80a3d50 0x7f3cd80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.375+0000 7f3ceaa00700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] conn(0x7f3cd80ad860 0x7f3cd80afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.375+0000 7f3ceaa00700 1 -- 172.21.15.45:0/2508223310 --> [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3cd80b03d0 con 0x7f3cd80ad860 2024-04-25T18:57:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.375+0000 7f3ceaa00700 1 -- 172.21.15.45:0/2508223310 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f3ce40d1d50 con 0x7f3cf4074c70 2024-04-25T18:57:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.375+0000 7f3cf2800700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] conn(0x7f3cd80ad860 0x7f3cd80afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.375+0000 7f3cf1400700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3cd80a3d50 0x7f3cd80a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009750 tx=0x7f3cdc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.376+0000 7f3cf2800700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] conn(0x7f3cd80ad860 0x7f3cd80afd00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.377+0000 7f3ceaa00700 1 -- 172.21.15.45:0/2508223310 <== osd.11 v2:172.21.15.79:6824/1965921727 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3cd80b03d0 con 0x7f3cd80ad860 2024-04-25T18:57:08.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.395+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 --> [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3cf40635d0 con 0x7f3cd80ad860 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3ceaa00700 1 -- 172.21.15.45:0/2508223310 <== osd.11 v2:172.21.15.79:6824/1965921727 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3cf40635d0 con 0x7f3cd80ad860 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 >> [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] conn(0x7f3cd80ad860 msgr2=0x7f3cd80afd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3cf9200700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] conn(0x7f3cd80ad860 0x7f3cd80afd00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3cd80a3d50 msgr2=0x7f3cd80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3cf9200700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3cd80a3d50 0x7f3cd80a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009750 tx=0x7f3cdc009310 comp rx=0 tx=0).stop 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4074c70 msgr2=0x7f3cf419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3cf9200700 1 --2- 172.21.15.45:0/2508223310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4074c70 0x7f3cf419b370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ce4009230 tx=0x7f3ce400fa70 comp rx=0 tx=0).stop 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3cf2800700 1 -- 172.21.15.45:0/2508223310 reap_dead start 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 shutdown_connections 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 >> 172.21.15.45:0/2508223310 conn(0x7f3cf4079ba0 msgr2=0x7f3cf40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:08.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.396+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 shutdown_connections 2024-04-25T18:57:08.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:08.397+0000 7f3cf9200700 1 -- 172.21.15.45:0/2508223310 wait complete. 2024-04-25T18:57:09.679 INFO:teuthology.orchestra.run.smithi045.stdout:360777252902 2024-04-25T18:57:09.680 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T18:57:09.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.045+0000 7efe11000700 1 -- 172.21.15.45:0/2049501451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0c074e10 msgr2=0x7efe0c0751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:09.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.045+0000 7efe11000700 1 --2- 172.21.15.45:0/2049501451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0c074e10 0x7efe0c0751f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009990 tx=0x7efdfc009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:09.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.046+0000 7efe11000700 1 -- 172.21.15.45:0/2049501451 shutdown_connections 2024-04-25T18:57:09.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.046+0000 7efe11000700 1 --2- 172.21.15.45:0/2049501451 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe0c10f210 0x7efe0c111620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:09.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.046+0000 7efe11000700 1 --2- 172.21.15.45:0/2049501451 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0c074e10 0x7efe0c0751f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:09.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.046+0000 7efe11000700 1 --2- 172.21.15.45:0/2049501451 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe0c075730 0x7efe0c10ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:09.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.046+0000 7efe11000700 1 -- 172.21.15.45:0/2049501451 >> 172.21.15.45:0/2049501451 conn(0x7efe0c0fe4c0 msgr2=0x7efe0c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:09.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.046+0000 7efe11000700 1 -- 172.21.15.45:0/2049501451 shutdown_connections 2024-04-25T18:57:09.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.046+0000 7efe11000700 1 -- 172.21.15.45:0/2049501451 wait complete. 2024-04-25T18:57:09.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.047+0000 7efe11000700 1 Processor -- start 2024-04-25T18:57:09.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe11000700 1 -- start start 2024-04-25T18:57:09.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe11000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe0c075730 0x7efe0c19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:09.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe11000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe0c10f210 0x7efe0c19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:09.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe11000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0c19fdd0 0x7efe0c1a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:09.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe11000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c114db0 con 0x7efe0c19fdd0 2024-04-25T18:57:09.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe11000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c114c30 con 0x7efe0c10f210 2024-04-25T18:57:09.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe11000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efe0c114f30 con 0x7efe0c075730 2024-04-25T18:57:09.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe0b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe0c075730 0x7efe0c19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:09.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe0b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe0c075730 0x7efe0c19b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56356/0 (socket says 172.21.15.45:56356) 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe0b400700 1 -- 172.21.15.45:0/1526501098 learned_addr learned my addr 172.21.15.45:0/1526501098 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe0aa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe0c10f210 0x7efe0c19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.048+0000 7efe0be00700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0c19fdd0 0x7efe0c1a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.049+0000 7efe0b400700 1 -- 172.21.15.45:0/1526501098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe0c10f210 msgr2=0x7efe0c19ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.049+0000 7efe0b400700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe0c10f210 0x7efe0c19ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.049+0000 7efe0b400700 1 -- 172.21.15.45:0/1526501098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0c19fdd0 msgr2=0x7efe0c1a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.049+0000 7efe0b400700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efe0c19fdd0 0x7efe0c1a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.049+0000 7efe0b400700 1 -- 172.21.15.45:0/1526501098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c1a08f0 con 0x7efe0c075730 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.049+0000 7efe0aa00700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efe0c10f210 0x7efe0c19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.049+0000 7efe0b400700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe0c075730 0x7efe0c19b500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efdfc000c00 tx=0x7efdfc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.049+0000 7efe03e00700 1 -- 172.21.15.45:0/1526501098 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00fb10 con 0x7efe0c075730 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.049+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe0c1a0b80 con 0x7efe0c075730 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.050+0000 7efe03e00700 1 -- 172.21.15.45:0/1526501098 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdfc00fcb0 con 0x7efe0c075730 2024-04-25T18:57:09.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.050+0000 7efe03e00700 1 -- 172.21.15.45:0/1526501098 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0213c0 con 0x7efe0c075730 2024-04-25T18:57:09.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.050+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe0c1a8300 con 0x7efe0c075730 2024-04-25T18:57:09.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.051+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efe0c0631d0 con 0x7efe0c075730 2024-04-25T18:57:09.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.053+0000 7efe03e00700 1 -- 172.21.15.45:0/1526501098 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efdfc009d10 con 0x7efe0c075730 2024-04-25T18:57:09.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.054+0000 7efe03e00700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efdf00a3d50 0x7efdf00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:09.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.054+0000 7efe03e00700 1 -- 172.21.15.45:0/1526501098 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7efdfc0d1920 con 0x7efe0c075730 2024-04-25T18:57:09.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.054+0000 7efe0aa00700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efdf00a3d50 0x7efdf00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:09.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.054+0000 7efe03e00700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] conn(0x7efdf00ad990 0x7efdf00afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:09.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.055+0000 7efe03e00700 1 -- 172.21.15.45:0/1526501098 --> [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdf00b0500 con 0x7efdf00ad990 2024-04-25T18:57:09.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.055+0000 7efe03e00700 1 -- 172.21.15.45:0/1526501098 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7efdfc0d1cd0 con 0x7efe0c075730 2024-04-25T18:57:09.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.055+0000 7efe0be00700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] conn(0x7efdf00ad990 0x7efdf00afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:09.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.055+0000 7efe0aa00700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efdf00a3d50 0x7efdf00a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe0c1107a0 tx=0x7efdf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:09.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.056+0000 7efe0be00700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] conn(0x7efdf00ad990 0x7efdf00afe30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:09.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.057+0000 7efe03e00700 1 -- 172.21.15.45:0/1526501098 <== osd.2 v2:172.21.15.79:6800/2930472364 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efdf00b0500 con 0x7efdf00ad990 2024-04-25T18:57:09.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.075+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 --> [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efe0c063650 con 0x7efdf00ad990 2024-04-25T18:57:09.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.075+0000 7efe03e00700 1 -- 172.21.15.45:0/1526501098 <== osd.2 v2:172.21.15.79:6800/2930472364 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efe0c063650 con 0x7efdf00ad990 2024-04-25T18:57:09.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 >> [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] conn(0x7efdf00ad990 msgr2=0x7efdf00afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:09.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe11000700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] conn(0x7efdf00ad990 0x7efdf00afe30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:09.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efdf00a3d50 msgr2=0x7efdf00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:09.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe11000700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7efdf00a3d50 0x7efdf00a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe0c1107a0 tx=0x7efdf8009260 comp rx=0 tx=0).stop 2024-04-25T18:57:09.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe0c075730 msgr2=0x7efe0c19b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:09.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe11000700 1 --2- 172.21.15.45:0/1526501098 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efe0c075730 0x7efe0c19b500 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7efdfc000c00 tx=0x7efdfc0047b0 comp rx=0 tx=0).stop 2024-04-25T18:57:09.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe0be00700 1 -- 172.21.15.45:0/1526501098 reap_dead start 2024-04-25T18:57:09.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 shutdown_connections 2024-04-25T18:57:09.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 >> 172.21.15.45:0/1526501098 conn(0x7efe0c0fe4c0 msgr2=0x7efe0c0ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:09.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 shutdown_connections 2024-04-25T18:57:09.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:07.076+0000 7efe11000700 1 -- 172.21.15.45:0/1526501098 wait complete. 2024-04-25T18:57:09.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:09 smithi045 ceph-mon[27776]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:10.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:09 smithi062 ceph-mon[32844]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:10.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:09 smithi079 ceph-mon[33002]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:10.426 INFO:teuthology.orchestra.run.smithi045.stdout:77309411416 2024-04-25T18:57:10.427 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T18:57:10.710 INFO:teuthology.orchestra.run.smithi045.stdout:34359738465 2024-04-25T18:57:10.710 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.742+0000 7f4dfd400700 1 -- 172.21.15.45:0/3857380638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4df806b0f0 msgr2=0x7f4df810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.742+0000 7f4dfd400700 1 --2- 172.21.15.45:0/3857380638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4df806b0f0 0x7f4df810f900 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4df00098f0 tx=0x7f4df0009c00 comp rx=0 tx=0).stop 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.743+0000 7f4dfd400700 1 -- 172.21.15.45:0/3857380638 shutdown_connections 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.743+0000 7f4dfd400700 1 --2- 172.21.15.45:0/3857380638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8108a30 0x7f4df8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.743+0000 7f4dfd400700 1 --2- 172.21.15.45:0/3857380638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4df806b0f0 0x7f4df810f900 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.743+0000 7f4dfd400700 1 --2- 172.21.15.45:0/3857380638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df806a750 0x7f4df806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.743+0000 7f4dfd400700 1 -- 172.21.15.45:0/3857380638 >> 172.21.15.45:0/3857380638 conn(0x7f4df80788f0 msgr2=0x7f4df8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.743+0000 7f4dfd400700 1 -- 172.21.15.45:0/3857380638 shutdown_connections 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.743+0000 7f4dfd400700 1 -- 172.21.15.45:0/3857380638 wait complete. 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.744+0000 7f4dfd400700 1 Processor -- start 2024-04-25T18:57:11.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4dfd400700 1 -- start start 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4dfd400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df806a750 0x7f4df8199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4dfd400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4df806b0f0 0x7f4df81996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4dfd400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8108a30 0x7f4df819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4dfd400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8112c20 con 0x7f4df806a750 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4dfd400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8112aa0 con 0x7f4df8108a30 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4dfd400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4df8112da0 con 0x7f4df806b0f0 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4df6800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df806a750 0x7f4df8199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4df6800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df806a750 0x7f4df8199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:44306/0 (socket says 172.21.15.45:44306) 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4df6800700 1 -- 172.21.15.45:0/1535853848 learned_addr learned my addr 172.21.15.45:0/1535853848 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4df5e00700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4df806b0f0 0x7f4df81996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.745+0000 7f4df7200700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8108a30 0x7f4df819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.746+0000 7f4df5e00700 1 -- 172.21.15.45:0/1535853848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8108a30 msgr2=0x7f4df819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.746+0000 7f4df5e00700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8108a30 0x7f4df819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:11.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.746+0000 7f4df5e00700 1 -- 172.21.15.45:0/1535853848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df806a750 msgr2=0x7f4df8199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.746+0000 7f4df5e00700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df806a750 0x7f4df8199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.746+0000 7f4df5e00700 1 -- 172.21.15.45:0/1535853848 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4df819e180 con 0x7f4df806b0f0 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.746+0000 7f4df6800700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4df806a750 0x7f4df8199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.746+0000 7f4df7200700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4df8108a30 0x7f4df819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.746+0000 7f4df5e00700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4df806b0f0 0x7f4df81996b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009fd0 tx=0x7f4de000c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.746+0000 7f4def400700 1 -- 172.21.15.45:0/1535853848 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0018020 con 0x7f4df806b0f0 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.747+0000 7f4def400700 1 -- 172.21.15.45:0/1535853848 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4de0003a00 con 0x7f4df806b0f0 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.747+0000 7f4def400700 1 -- 172.21.15.45:0/1535853848 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4de0007c40 con 0x7f4df806b0f0 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.747+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4df819e470 con 0x7f4df806b0f0 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.747+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4df81a5cf0 con 0x7f4df806b0f0 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.748+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4df80631d0 con 0x7f4df806b0f0 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.750+0000 7f4def400700 1 -- 172.21.15.45:0/1535853848 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4de0012050 con 0x7f4df806b0f0 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.751+0000 7f4def400700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4dd80a3d50 0x7f4dd80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:11.006 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.751+0000 7f4def400700 1 -- 172.21.15.45:0/1535853848 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f4de00d0cd0 con 0x7f4df806b0f0 2024-04-25T18:57:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.752+0000 7f4df6800700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4dd80a3d50 0x7f4dd80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.752+0000 7f4def400700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] conn(0x7f4dd80ad990 0x7f4dd80afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.752+0000 7f4def400700 1 -- 172.21.15.45:0/1535853848 --> [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4de001ce50 con 0x7f4dd80ad990 2024-04-25T18:57:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.752+0000 7f4def400700 1 -- 172.21.15.45:0/1535853848 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f4de00d1080 con 0x7f4df806b0f0 2024-04-25T18:57:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.752+0000 7f4df7200700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] conn(0x7f4dd80ad990 0x7f4dd80afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.752+0000 7f4df6800700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4dd80a3d50 0x7f4dd80a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4de8011550 tx=0x7f4de8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.753+0000 7f4df7200700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] conn(0x7f4dd80ad990 0x7f4dd80afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.754+0000 7f4def400700 1 -- 172.21.15.45:0/1535853848 <== osd.4 v2:172.21.15.62:6808/2453949627 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4de001ce50 con 0x7f4dd80ad990 2024-04-25T18:57:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.773+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 --> [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4df80636a0 con 0x7f4dd80ad990 2024-04-25T18:57:11.007 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.773+0000 7f4def400700 1 -- 172.21.15.45:0/1535853848 <== osd.4 v2:172.21.15.62:6808/2453949627 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4df80636a0 con 0x7f4dd80ad990 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.774+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 >> [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] conn(0x7f4dd80ad990 msgr2=0x7f4dd80afe30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.774+0000 7f4dfd400700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] conn(0x7f4dd80ad990 0x7f4dd80afe30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.774+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4dd80a3d50 msgr2=0x7f4dd80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.774+0000 7f4dfd400700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f4dd80a3d50 0x7f4dd80a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4de8011550 tx=0x7f4de8009260 comp rx=0 tx=0).stop 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.774+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4df806b0f0 msgr2=0x7f4df81996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.774+0000 7f4dfd400700 1 --2- 172.21.15.45:0/1535853848 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4df806b0f0 0x7f4df81996b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009fd0 tx=0x7f4de000c620 comp rx=0 tx=0).stop 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.774+0000 7f4df7200700 1 -- 172.21.15.45:0/1535853848 reap_dead start 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.774+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 shutdown_connections 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.774+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 >> 172.21.15.45:0/1535853848 conn(0x7f4df80788f0 msgr2=0x7f4df80ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.774+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 shutdown_connections 2024-04-25T18:57:11.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:05.775+0000 7f4dfd400700 1 -- 172.21.15.45:0/1535853848 wait complete. 2024-04-25T18:57:11.685 INFO:teuthology.orchestra.run.smithi045.stdout:150323855436 2024-04-25T18:57:11.686 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T18:57:11.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:11 smithi045 ceph-mon[27776]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:57:12.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:11 smithi062 ceph-mon[32844]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:57:12.133 INFO:teuthology.orchestra.run.smithi045.stdout:231928234045 2024-04-25T18:57:12.134 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T18:57:12.154 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:12.172 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:11 smithi079 ceph-mon[33002]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:57:12.191 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:12.193 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:12.204 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:13.961 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:14.014 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:13 smithi045 ceph-mon[27776]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:57:14.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:13 smithi062 ceph-mon[32844]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:57:14.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:13 smithi079 ceph-mon[33002]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:57:15.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:15 smithi045 ceph-mon[27776]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T18:57:15.992 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:15.999 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:16.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:15 smithi062 ceph-mon[32844]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T18:57:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:16.157 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:15 smithi079 ceph-mon[33002]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T18:57:18.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:17 smithi062 ceph-mon[32844]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-25T18:57:18.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:17 smithi062 ceph-mon[32844]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-25T18:57:18.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:17 smithi045 ceph-mon[27776]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-25T18:57:18.164 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:17 smithi045 ceph-mon[27776]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-25T18:57:18.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:17 smithi079 ceph-mon[33002]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-04-25T18:57:18.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:17 smithi079 ceph-mon[33002]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-25T18:57:20.060 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:19 smithi045 ceph-mon[27776]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T18:57:20.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:19 smithi062 ceph-mon[32844]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T18:57:20.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:19 smithi079 ceph-mon[33002]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T18:57:21.937 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:21 smithi045 ceph-mon[27776]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T18:57:22.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:21 smithi062 ceph-mon[32844]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T18:57:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:21 smithi079 ceph-mon[33002]: pgmap v386: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T18:57:22.919 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.780+0000 7f5f03400700 1 -- 172.21.15.45:0/3422469009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5efc102a90 msgr2=0x7f5efc102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.780+0000 7f5f03400700 1 --2- 172.21.15.45:0/3422469009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5efc102a90 0x7f5efc102f10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009990 tx=0x7f5ee8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.780+0000 7f5f03400700 1 -- 172.21.15.45:0/3422469009 shutdown_connections 2024-04-25T18:57:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.780+0000 7f5f03400700 1 --2- 172.21.15.45:0/3422469009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc108a50 0x7f5efc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.780+0000 7f5f03400700 1 --2- 172.21.15.45:0/3422469009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5efc103450 0x7f5efc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.780+0000 7f5f03400700 1 --2- 172.21.15.45:0/3422469009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5efc102a90 0x7f5efc102f10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.780+0000 7f5f03400700 1 -- 172.21.15.45:0/3422469009 >> 172.21.15.45:0/3422469009 conn(0x7f5efc079ba0 msgr2=0x7f5efc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.780+0000 7f5f03400700 1 -- 172.21.15.45:0/3422469009 shutdown_connections 2024-04-25T18:57:23.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.780+0000 7f5f03400700 1 -- 172.21.15.45:0/3422469009 wait complete. 2024-04-25T18:57:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.782+0000 7f5f03400700 1 Processor -- start 2024-04-25T18:57:23.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.782+0000 7f5f03400700 1 -- start start 2024-04-25T18:57:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5f03400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc102a90 0x7f5efc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5f03400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5efc103450 0x7f5efc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5f03400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5efc108a50 0x7f5efc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5f03400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc112c50 con 0x7f5efc108a50 2024-04-25T18:57:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5f03400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc112ad0 con 0x7f5efc102a90 2024-04-25T18:57:23.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5f03400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc112dd0 con 0x7f5efc103450 2024-04-25T18:57:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5efbe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5efc103450 0x7f5efc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5efbe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5efc103450 0x7f5efc075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:59802/0 (socket says 172.21.15.45:59802) 2024-04-25T18:57:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5efbe00700 1 -- 172.21.15.45:0/2106993080 learned_addr learned my addr 172.21.15.45:0/2106993080 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5f00a00700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc102a90 0x7f5efc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5f01400700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5efc108a50 0x7f5efc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5efbe00700 1 -- 172.21.15.45:0/2106993080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc102a90 msgr2=0x7f5efc074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.783+0000 7f5efbe00700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc102a90 0x7f5efc074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.784+0000 7f5efbe00700 1 -- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5efc108a50 msgr2=0x7f5efc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.784+0000 7f5efbe00700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5efc108a50 0x7f5efc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.784+0000 7f5efbe00700 1 -- 172.21.15.45:0/2106993080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc1a76d0 con 0x7f5efc103450 2024-04-25T18:57:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.784+0000 7f5f01400700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5efc108a50 0x7f5efc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:57:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.784+0000 7f5efbe00700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5efc103450 0x7f5efc075280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8000c00 tx=0x7f5ee8009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.784+0000 7f5ef9600700 1 -- 172.21.15.45:0/2106993080 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8027070 con 0x7f5efc103450 2024-04-25T18:57:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.784+0000 7f5ef9600700 1 -- 172.21.15.45:0/2106993080 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee8022470 con 0x7f5efc103450 2024-04-25T18:57:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.784+0000 7f5ef9600700 1 -- 172.21.15.45:0/2106993080 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8017650 con 0x7f5efc103450 2024-04-25T18:57:23.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.784+0000 7f5f03400700 1 -- 172.21.15.45:0/2106993080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5efc1a7960 con 0x7f5efc103450 2024-04-25T18:57:23.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.785+0000 7f5f03400700 1 -- 172.21.15.45:0/2106993080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5efc1a7e40 con 0x7f5efc103450 2024-04-25T18:57:23.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.786+0000 7f5f03400700 1 -- 172.21.15.45:0/2106993080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5efc069730 con 0x7f5efc103450 2024-04-25T18:57:23.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.788+0000 7f5ef9600700 1 -- 172.21.15.45:0/2106993080 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5ee802a070 con 0x7f5efc103450 2024-04-25T18:57:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.789+0000 7f5ef9600700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5ee40a3c40 0x7f5ee40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:23.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.789+0000 7f5ef9600700 1 -- 172.21.15.45:0/2106993080 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f5ee80d1370 con 0x7f5efc103450 2024-04-25T18:57:23.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.790+0000 7f5f00a00700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5ee40a3c40 0x7f5ee40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:23.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.790+0000 7f5f00a00700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5ee40a3c40 0x7f5ee40a6100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009680 tx=0x7f5ef0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:23.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.793+0000 7f5ef9600700 1 -- 172.21.15.45:0/2106993080 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ee805eff0 con 0x7f5efc103450 2024-04-25T18:57:23.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.478+0000 7fa569c00700 1 -- 172.21.15.45:0/2686614669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa564102a90 msgr2=0x7fa564102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:23.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.478+0000 7fa569c00700 1 --2- 172.21.15.45:0/2686614669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa564102a90 0x7fa564102f10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa5500098f0 tx=0x7fa550009c00 comp rx=0 tx=0).stop 2024-04-25T18:57:23.943 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.478+0000 7fa569c00700 1 -- 172.21.15.45:0/2686614669 shutdown_connections 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.478+0000 7fa569c00700 1 --2- 172.21.15.45:0/2686614669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa564108a50 0x7fa564108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.478+0000 7fa569c00700 1 --2- 172.21.15.45:0/2686614669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564103450 0x7fa56410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.478+0000 7fa569c00700 1 --2- 172.21.15.45:0/2686614669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa564102a90 0x7fa564102f10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.478+0000 7fa569c00700 1 -- 172.21.15.45:0/2686614669 >> 172.21.15.45:0/2686614669 conn(0x7fa564079ba0 msgr2=0x7fa564079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.479+0000 7fa569c00700 1 -- 172.21.15.45:0/2686614669 shutdown_connections 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.479+0000 7fa569c00700 1 -- 172.21.15.45:0/2686614669 wait complete. 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.480+0000 7fa569c00700 1 Processor -- start 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.480+0000 7fa569c00700 1 -- start start 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa569c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa564102a90 0x7fa564199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa569c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564103450 0x7fa564199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa569c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa564108a50 0x7fa56419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa569c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa564112c50 con 0x7fa564103450 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa569c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa564112ad0 con 0x7fa564102a90 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa569c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa564112dd0 con 0x7fa564108a50 2024-04-25T18:57:23.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa562a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa564102a90 0x7fa564199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa562a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa564102a90 0x7fa564199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:40530/0 (socket says 172.21.15.45:40530) 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa562a00700 1 -- 172.21.15.45:0/2206742020 learned_addr learned my addr 172.21.15.45:0/2206742020 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa562000700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564103450 0x7fa564199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa563400700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa564108a50 0x7fa56419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa562a00700 1 -- 172.21.15.45:0/2206742020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa564108a50 msgr2=0x7fa56419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa562a00700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa564108a50 0x7fa56419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa562a00700 1 -- 172.21.15.45:0/2206742020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564103450 msgr2=0x7fa564199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa562a00700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564103450 0x7fa564199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.481+0000 7fa562a00700 1 -- 172.21.15.45:0/2206742020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56419e230 con 0x7fa564102a90 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.482+0000 7fa562a00700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa564102a90 0x7fa564199220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa55400ec10 tx=0x7fa55400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.482+0000 7fa55b400700 1 -- 172.21.15.45:0/2206742020 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55400cc90 con 0x7fa564102a90 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.482+0000 7fa569c00700 1 -- 172.21.15.45:0/2206742020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56419e520 con 0x7fa564102a90 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.482+0000 7fa55b400700 1 -- 172.21.15.45:0/2206742020 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa55400ce30 con 0x7fa564102a90 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.482+0000 7fa55b400700 1 -- 172.21.15.45:0/2206742020 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554010430 con 0x7fa564102a90 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.482+0000 7fa569c00700 1 -- 172.21.15.45:0/2206742020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5641a5da0 con 0x7fa564102a90 2024-04-25T18:57:23.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.483+0000 7fa569c00700 1 -- 172.21.15.45:0/2206742020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa564069730 con 0x7fa564102a90 2024-04-25T18:57:23.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.485+0000 7fa55b400700 1 -- 172.21.15.45:0/2206742020 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa554004000 con 0x7fa564102a90 2024-04-25T18:57:23.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.487+0000 7fa55b400700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5480a3c40 0x7fa5480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:23.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.487+0000 7fa55b400700 1 -- 172.21.15.45:0/2206742020 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa5540d0670 con 0x7fa564102a90 2024-04-25T18:57:23.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.487+0000 7fa562000700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5480a3c40 0x7fa5480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:23.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.488+0000 7fa562000700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5480a3c40 0x7fa5480a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa5500098c0 tx=0x7fa550019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:23.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.491+0000 7fa55b400700 1 -- 172.21.15.45:0/2206742020 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa55409cfd0 con 0x7fa564102a90 2024-04-25T18:57:23.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.738+0000 7fa569c00700 1 -- 172.21.15.45:0/2206742020 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa56419a580 con 0x7fa564102a90 2024-04-25T18:57:23.946 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.739+0000 7fa55b400700 1 -- 172.21.15.45:0/2206742020 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa55409c8d0 con 0x7fa564102a90 2024-04-25T18:57:23.946 INFO:teuthology.orchestra.run.smithi045.stdout:115964117076 2024-04-25T18:57:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 -- 172.21.15.45:0/2206742020 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5480a3c40 msgr2=0x7fa5480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5480a3c40 0x7fa5480a6100 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa5500098c0 tx=0x7fa550019040 comp rx=0 tx=0).stop 2024-04-25T18:57:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 -- 172.21.15.45:0/2206742020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa564102a90 msgr2=0x7fa564199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa564102a90 0x7fa564199220 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa55400ec10 tx=0x7fa55400ef20 comp rx=0 tx=0).stop 2024-04-25T18:57:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 -- 172.21.15.45:0/2206742020 shutdown_connections 2024-04-25T18:57:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa5480a3c40 0x7fa5480a6100 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa564102a90 0x7fa564199220 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa564108a50 0x7fa56419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 --2- 172.21.15.45:0/2206742020 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa564103450 0x7fa564199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 -- 172.21.15.45:0/2206742020 >> 172.21.15.45:0/2206742020 conn(0x7fa564079ba0 msgr2=0x7fa5641019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:23.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 -- 172.21.15.45:0/2206742020 shutdown_connections 2024-04-25T18:57:23.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.743+0000 7fa569c00700 1 -- 172.21.15.45:0/2206742020 wait complete. 2024-04-25T18:57:23.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:23 smithi045 ceph-mon[27776]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T18:57:23.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:23 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:23.993 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:23 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2206742020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T18:57:24.039 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.039+0000 7f5f03400700 1 -- 172.21.15.45:0/2106993080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5efc02df80 con 0x7f5efc103450 2024-04-25T18:57:24.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.039+0000 7f5ef9600700 1 -- 172.21.15.45:0/2106993080 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5ee8013070 con 0x7f5efc103450 2024-04-25T18:57:24.040 INFO:teuthology.orchestra.run.smithi045.stdout:55834574944 2024-04-25T18:57:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.043+0000 7f5f03400700 1 -- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5ee40a3c40 msgr2=0x7f5ee40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.043+0000 7f5f03400700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5ee40a3c40 0x7f5ee40a6100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009680 tx=0x7f5ef0009260 comp rx=0 tx=0).stop 2024-04-25T18:57:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.043+0000 7f5f03400700 1 -- 172.21.15.45:0/2106993080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5efc103450 msgr2=0x7f5efc075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.043+0000 7f5f03400700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5efc103450 0x7f5efc075280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8000c00 tx=0x7f5ee8009510 comp rx=0 tx=0).stop 2024-04-25T18:57:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.043+0000 7f5f03400700 1 -- 172.21.15.45:0/2106993080 shutdown_connections 2024-04-25T18:57:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.043+0000 7f5f03400700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5ee40a3c40 0x7f5ee40a6100 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.043+0000 7f5f03400700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5efc102a90 0x7f5efc074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.044 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.043+0000 7f5f03400700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5efc108a50 0x7f5efc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.043+0000 7f5f03400700 1 --2- 172.21.15.45:0/2106993080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5efc103450 0x7f5efc075280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.043+0000 7f5f03400700 1 -- 172.21.15.45:0/2106993080 >> 172.21.15.45:0/2106993080 conn(0x7f5efc079ba0 msgr2=0x7f5efc101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:24.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.044+0000 7f5f03400700 1 -- 172.21.15.45:0/2106993080 shutdown_connections 2024-04-25T18:57:24.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.044+0000 7f5f03400700 1 -- 172.21.15.45:0/2106993080 wait complete. 2024-04-25T18:57:24.109 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:23 smithi079 ceph-mon[33002]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T18:57:24.109 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:23 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:24.109 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:23 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2206742020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T18:57:24.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:23 smithi062 ceph-mon[32844]: pgmap v387: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T18:57:24.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:23 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:57:24.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:23 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2206742020' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.671+0000 7fa8bea00700 1 -- 172.21.15.45:0/3221687736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b806a750 msgr2=0x7fa8b806ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.671+0000 7fa8bea00700 1 --2- 172.21.15.45:0/3221687736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b806a750 0x7fa8b806ab30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8009990 tx=0x7fa8a8009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.671+0000 7fa8bea00700 1 -- 172.21.15.45:0/3221687736 shutdown_connections 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.671+0000 7fa8bea00700 1 --2- 172.21.15.45:0/3221687736 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b806a750 0x7fa8b806ab30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.671+0000 7fa8bea00700 1 --2- 172.21.15.45:0/3221687736 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b810f4f0 0x7fa8b81118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.671+0000 7fa8bea00700 1 --2- 172.21.15.45:0/3221687736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8b806b070 0x7fa8b810efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.671+0000 7fa8bea00700 1 -- 172.21.15.45:0/3221687736 >> 172.21.15.45:0/3221687736 conn(0x7fa8b80788f0 msgr2=0x7fa8b8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.671+0000 7fa8bea00700 1 -- 172.21.15.45:0/3221687736 shutdown_connections 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.671+0000 7fa8bea00700 1 -- 172.21.15.45:0/3221687736 wait complete. 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.672+0000 7fa8bea00700 1 Processor -- start 2024-04-25T18:57:24.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.672+0000 7fa8bea00700 1 -- start start 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b806a750 0x7fa8b819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8b806b070 0x7fa8b819b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b810f4f0 0x7fa8b819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bea00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b8114e50 con 0x7fa8b806a750 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bea00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b8114cd0 con 0x7fa8b810f4f0 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bea00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b8114fd0 con 0x7fa8b806b070 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bca00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b810f4f0 0x7fa8b819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bca00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b810f4f0 0x7fa8b819fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:40546/0 (socket says 172.21.15.45:40546) 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bca00700 1 -- 172.21.15.45:0/2166698196 learned_addr learned my addr 172.21.15.45:0/2166698196 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8b7400700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8b806b070 0x7fa8b819b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8b7e00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b806a750 0x7fa8b819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bca00700 1 -- 172.21.15.45:0/2166698196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8b806b070 msgr2=0x7fa8b819b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bca00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8b806b070 0x7fa8b819b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bca00700 1 -- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b806a750 msgr2=0x7fa8b819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:24.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bca00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b806a750 0x7fa8b819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.673+0000 7fa8bca00700 1 -- 172.21.15.45:0/2166698196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b81a0340 con 0x7fa8b810f4f0 2024-04-25T18:57:24.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.674+0000 7fa8b7e00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b806a750 0x7fa8b819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:57:24.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.674+0000 7fa8bca00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b810f4f0 0x7fa8b819fc00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000ef90 tx=0x7fa8b000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:24.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.674+0000 7fa8b4c00700 1 -- 172.21.15.45:0/2166698196 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b00092e0 con 0x7fa8b810f4f0 2024-04-25T18:57:24.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.674+0000 7fa8b4c00700 1 -- 172.21.15.45:0/2166698196 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8b0004bb0 con 0x7fa8b810f4f0 2024-04-25T18:57:24.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.674+0000 7fa8bea00700 1 -- 172.21.15.45:0/2166698196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b81a05d0 con 0x7fa8b810f4f0 2024-04-25T18:57:24.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.674+0000 7fa8bea00700 1 -- 172.21.15.45:0/2166698196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b81a7eb0 con 0x7fa8b810f4f0 2024-04-25T18:57:24.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.675+0000 7fa8b4c00700 1 -- 172.21.15.45:0/2166698196 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0004280 con 0x7fa8b810f4f0 2024-04-25T18:57:24.794 INFO:teuthology.orchestra.run.smithi045.stdout:206158430275 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.676+0000 7fa8bea00700 1 -- 172.21.15.45:0/2166698196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa898005330 con 0x7fa8b810f4f0 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.678+0000 7fa8b4c00700 1 -- 172.21.15.45:0/2166698196 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa8b001b030 con 0x7fa8b810f4f0 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.679+0000 7fa8b4c00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa89c0a3cc0 0x7fa89c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.679+0000 7fa8b4c00700 1 -- 172.21.15.45:0/2166698196 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa8b00cfe20 con 0x7fa8b810f4f0 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.680+0000 7fa8b7e00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa89c0a3cc0 0x7fa89c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.681+0000 7fa8b7e00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa89c0a3cc0 0x7fa89c0a6180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8006010 tx=0x7fa8a801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.684+0000 7fa8b4c00700 1 -- 172.21.15.45:0/2166698196 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8b0022020 con 0x7fa8b810f4f0 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.990+0000 7fa8bea00700 1 -- 172.21.15.45:0/2166698196 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa898005620 con 0x7fa8b810f4f0 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.991+0000 7fa8b4c00700 1 -- 172.21.15.45:0/2166698196 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa8b0010040 con 0x7fa8b810f4f0 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.995+0000 7fa8bea00700 1 -- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa89c0a3cc0 msgr2=0x7fa89c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:24.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.995+0000 7fa8bea00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa89c0a3cc0 0x7fa89c0a6180 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8006010 tx=0x7fa8a801a040 comp rx=0 tx=0).stop 2024-04-25T18:57:24.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.995+0000 7fa8bea00700 1 -- 172.21.15.45:0/2166698196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b810f4f0 msgr2=0x7fa8b819fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:24.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.995+0000 7fa8bea00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b810f4f0 0x7fa8b819fc00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000ef90 tx=0x7fa8b000c3a0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.996+0000 7fa8bea00700 1 -- 172.21.15.45:0/2166698196 shutdown_connections 2024-04-25T18:57:24.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.996+0000 7fa8bea00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa89c0a3cc0 0x7fa89c0a6180 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.996+0000 7fa8bea00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa8b810f4f0 0x7fa8b819fc00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.996+0000 7fa8bea00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa8b806a750 0x7fa8b819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.996+0000 7fa8bea00700 1 --2- 172.21.15.45:0/2166698196 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa8b806b070 0x7fa8b819b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.996+0000 7fa8bea00700 1 -- 172.21.15.45:0/2166698196 >> 172.21.15.45:0/2166698196 conn(0x7fa8b80788f0 msgr2=0x7fa8b81109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:24.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.996+0000 7fa8bea00700 1 -- 172.21.15.45:0/2166698196 shutdown_connections 2024-04-25T18:57:24.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:23.996+0000 7fa8bea00700 1 -- 172.21.15.45:0/2166698196 wait complete. 2024-04-25T18:57:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.814+0000 7feee7200700 1 -- 172.21.15.45:0/745993620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8103330 msgr2=0x7feee810f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:24.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.814+0000 7feee7200700 1 --2- 172.21.15.45:0/745993620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8103330 0x7feee810f840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feee000b0a0 tx=0x7feee000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.814+0000 7feee7200700 1 -- 172.21.15.45:0/745993620 shutdown_connections 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.814+0000 7feee7200700 1 --2- 172.21.15.45:0/745993620 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8103330 0x7feee810f840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.814+0000 7feee7200700 1 --2- 172.21.15.45:0/745993620 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee8108970 0x7feee8108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.814+0000 7feee7200700 1 --2- 172.21.15.45:0/745993620 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8102970 0x7feee8102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.814+0000 7feee7200700 1 -- 172.21.15.45:0/745993620 >> 172.21.15.45:0/745993620 conn(0x7feee80fe4c0 msgr2=0x7feee81008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.814+0000 7feee7200700 1 -- 172.21.15.45:0/745993620 shutdown_connections 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.814+0000 7feee7200700 1 -- 172.21.15.45:0/745993620 wait complete. 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.816+0000 7feee7200700 1 Processor -- start 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.816+0000 7feee7200700 1 -- start start 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.816+0000 7feee7200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8102970 0x7feee819b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee7200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee8103330 0x7feee819b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:24.951 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee7200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8108970 0x7feee819fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee7200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feee8112b80 con 0x7feee8103330 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee7200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feee8112a00 con 0x7feee8102970 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee7200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feee8112d00 con 0x7feee8108970 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee5e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8102970 0x7feee819b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee5e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8102970 0x7feee819b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:51432/0 (socket says 172.21.15.45:51432) 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee5e00700 1 -- 172.21.15.45:0/609629032 learned_addr learned my addr 172.21.15.45:0/609629032 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee6800700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8108970 0x7feee819fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee5400700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee8103330 0x7feee819b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee5e00700 1 -- 172.21.15.45:0/609629032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8108970 msgr2=0x7feee819fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee5e00700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8108970 0x7feee819fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee5e00700 1 -- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee8103330 msgr2=0x7feee819b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee5e00700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee8103330 0x7feee819b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee5e00700 1 -- 172.21.15.45:0/609629032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee81a0180 con 0x7feee8102970 2024-04-25T18:57:24.952 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee6800700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8108970 0x7feee819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.817+0000 7feee5400700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee8103330 0x7feee819b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:57:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.818+0000 7feee5e00700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8102970 0x7feee819b170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feed800e9a0 tx=0x7feed800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.818+0000 7feedea00700 1 -- 172.21.15.45:0/609629032 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed800cbd0 con 0x7feee8102970 2024-04-25T18:57:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.818+0000 7feedea00700 1 -- 172.21.15.45:0/609629032 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed800cd70 con 0x7feee8102970 2024-04-25T18:57:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.818+0000 7feedea00700 1 -- 172.21.15.45:0/609629032 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed8018680 con 0x7feee8102970 2024-04-25T18:57:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.818+0000 7feee7200700 1 -- 172.21.15.45:0/609629032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee81a0470 con 0x7feee8102970 2024-04-25T18:57:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.818+0000 7feee7200700 1 -- 172.21.15.45:0/609629032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee81a7cf0 con 0x7feee8102970 2024-04-25T18:57:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.819+0000 7feee7200700 1 -- 172.21.15.45:0/609629032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee8069730 con 0x7feee8102970 2024-04-25T18:57:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.821+0000 7feedea00700 1 -- 172.21.15.45:0/609629032 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feed8010430 con 0x7feee8102970 2024-04-25T18:57:24.953 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.822+0000 7feedea00700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7feecc0a3cc0 0x7feecc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.822+0000 7feedea00700 1 -- 172.21.15.45:0/609629032 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7feed80d0600 con 0x7feee8102970 2024-04-25T18:57:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.823+0000 7feee5400700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7feecc0a3cc0 0x7feecc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.823+0000 7feee5400700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7feecc0a3cc0 0x7feecc0a6180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feee810f4e0 tx=0x7feed4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:24.954 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:24.827+0000 7feedea00700 1 -- 172.21.15.45:0/609629032 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feed809cf60 con 0x7feee8102970 2024-04-25T18:57:24.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:24 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2166698196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T18:57:24.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:24 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2106993080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T18:57:24.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:24.954 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:24.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:24.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:24.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:24.955 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:24 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.058+0000 7feee7200700 1 -- 172.21.15.45:0/609629032 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7feee81a7ec0 con 0x7feee8102970 2024-04-25T18:57:25.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.059+0000 7feedea00700 1 -- 172.21.15.45:0/609629032 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feed809c860 con 0x7feee8102970 2024-04-25T18:57:25.060 INFO:teuthology.orchestra.run.smithi045.stdout:300647710771 2024-04-25T18:57:25.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 -- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7feecc0a3cc0 msgr2=0x7feecc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7feecc0a3cc0 0x7feecc0a6180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feee810f4e0 tx=0x7feed4009260 comp rx=0 tx=0).stop 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 -- 172.21.15.45:0/609629032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8102970 msgr2=0x7feee819b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8102970 0x7feee819b170 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feed800e9a0 tx=0x7feed800ecb0 comp rx=0 tx=0).stop 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 -- 172.21.15.45:0/609629032 shutdown_connections 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7feecc0a3cc0 0x7feecc0a6180 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feee8102970 0x7feee819b170 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feee8108970 0x7feee819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 --2- 172.21.15.45:0/609629032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feee8103330 0x7feee819b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 -- 172.21.15.45:0/609629032 >> 172.21.15.45:0/609629032 conn(0x7feee80fe4c0 msgr2=0x7feee810e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.062+0000 7feee7200700 1 -- 172.21.15.45:0/609629032 shutdown_connections 2024-04-25T18:57:25.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:25.063+0000 7feee7200700 1 -- 172.21.15.45:0/609629032 wait complete. 2024-04-25T18:57:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:24 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2166698196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T18:57:25.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:24 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2106993080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T18:57:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:24 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:24 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:24 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:24 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:24 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.114 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:24 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:24 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2166698196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T18:57:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:24 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2106993080' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T18:57:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:25.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:24 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:26.039 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574940 got 55834574944 for osd.1 2024-04-25T18:57:26.039 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:26.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:26 smithi062 ceph-mon[32844]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T18:57:26.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:26 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/609629032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T18:57:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:26 smithi079 ceph-mon[33002]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T18:57:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:26 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/609629032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T18:57:26.459 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:26 smithi045 ceph-mon[27776]: pgmap v388: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T18:57:26.459 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:26 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/609629032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T18:57:26.714 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:27.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.161+0000 7f3879000700 1 -- 172.21.15.45:0/2345340657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406a750 msgr2=0x7f387406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:27.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.161+0000 7f3879000700 1 --2- 172.21.15.45:0/2345340657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406a750 0x7f387406abb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3860009990 tx=0x7f3860009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:27.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.161+0000 7f3879000700 1 -- 172.21.15.45:0/2345340657 shutdown_connections 2024-04-25T18:57:27.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.161+0000 7f3879000700 1 --2- 172.21.15.45:0/2345340657 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f387406b0f0 0x7f387410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:27.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.161+0000 7f3879000700 1 --2- 172.21.15.45:0/2345340657 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3874108a30 0x7f3874108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.161+0000 7f3879000700 1 --2- 172.21.15.45:0/2345340657 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406a750 0x7f387406abb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.161+0000 7f3879000700 1 -- 172.21.15.45:0/2345340657 >> 172.21.15.45:0/2345340657 conn(0x7f38740788f0 msgr2=0x7f3874078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.161+0000 7f3879000700 1 -- 172.21.15.45:0/2345340657 shutdown_connections 2024-04-25T18:57:27.164 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.161+0000 7f3879000700 1 -- 172.21.15.45:0/2345340657 wait complete. 2024-04-25T18:57:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.162+0000 7f3879000700 1 Processor -- start 2024-04-25T18:57:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.162+0000 7f3879000700 1 -- start start 2024-04-25T18:57:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3879000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f387406a750 0x7f387419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3879000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406b0f0 0x7f387419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3879000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3874108a30 0x7f387419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3879000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3874112c40 con 0x7f3874108a30 2024-04-25T18:57:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3879000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3874112ac0 con 0x7f387406b0f0 2024-04-25T18:57:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3879000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3874112dc0 con 0x7f387406a750 2024-04-25T18:57:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3872800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3874108a30 0x7f387419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:27.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3872800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3874108a30 0x7f387419fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56054/0 (socket says 172.21.15.45:56054) 2024-04-25T18:57:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3872800700 1 -- 172.21.15.45:0/549678462 learned_addr learned my addr 172.21.15.45:0/549678462 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3871e00700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f387406a750 0x7f387419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:27.166 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3871400700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406b0f0 0x7f387419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3871e00700 1 -- 172.21.15.45:0/549678462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406b0f0 msgr2=0x7f387419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3871e00700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406b0f0 0x7f387419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3871e00700 1 -- 172.21.15.45:0/549678462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3874108a30 msgr2=0x7f387419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.163+0000 7f3871e00700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3874108a30 0x7f387419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.164+0000 7f3871e00700 1 -- 172.21.15.45:0/549678462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38741a0370 con 0x7f387406a750 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.164+0000 7f3872800700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3874108a30 0x7f387419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.164+0000 7f3871400700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406b0f0 0x7f387419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.164+0000 7f3871e00700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f387406a750 0x7f387419b3b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f386400e9f0 tx=0x7f386400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.164+0000 7f386aa00700 1 -- 172.21.15.45:0/549678462 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386400c9c0 con 0x7f387406a750 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.164+0000 7f3879000700 1 -- 172.21.15.45:0/549678462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38741a0660 con 0x7f387406a750 2024-04-25T18:57:27.167 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.164+0000 7f386aa00700 1 -- 172.21.15.45:0/549678462 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f386400cb60 con 0x7f387406a750 2024-04-25T18:57:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.164+0000 7f386aa00700 1 -- 172.21.15.45:0/549678462 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864018680 con 0x7f387406a750 2024-04-25T18:57:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.164+0000 7f3879000700 1 -- 172.21.15.45:0/549678462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38741a7ee0 con 0x7f387406a750 2024-04-25T18:57:27.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.165+0000 7f3879000700 1 -- 172.21.15.45:0/549678462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3874069730 con 0x7f387406a750 2024-04-25T18:57:27.170 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.168+0000 7f386aa00700 1 -- 172.21.15.45:0/549678462 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3864010430 con 0x7f387406a750 2024-04-25T18:57:27.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.169+0000 7f386aa00700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f38580a3c40 0x7f38580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:27.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.170+0000 7f386aa00700 1 -- 172.21.15.45:0/549678462 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f38640d08c0 con 0x7f387406a750 2024-04-25T18:57:27.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.170+0000 7f3871400700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f38580a3c40 0x7f38580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:27.172 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.171+0000 7f3871400700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f38580a3c40 0x7f38580a6100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3874078ed0 tx=0x7f386000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:27.173 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.172+0000 7f386aa00700 1 -- 172.21.15.45:0/549678462 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f386409d220 con 0x7f387406a750 2024-04-25T18:57:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.422+0000 7f3879000700 1 -- 172.21.15.45:0/549678462 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f387402df80 con 0x7f387406a750 2024-04-25T18:57:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.422+0000 7f386aa00700 1 -- 172.21.15.45:0/549678462 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f386409cb20 con 0x7f387406a750 2024-04-25T18:57:27.424 INFO:teuthology.orchestra.run.smithi045.stdout:360777252906 2024-04-25T18:57:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.425+0000 7f3879000700 1 -- 172.21.15.45:0/549678462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f38580a3c40 msgr2=0x7f38580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.425+0000 7f3879000700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f38580a3c40 0x7f38580a6100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3874078ed0 tx=0x7f386000b260 comp rx=0 tx=0).stop 2024-04-25T18:57:27.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.425+0000 7f3879000700 1 -- 172.21.15.45:0/549678462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f387406a750 msgr2=0x7f387419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.425+0000 7f3879000700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f387406a750 0x7f387419b3b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f386400e9f0 tx=0x7f386400ed00 comp rx=0 tx=0).stop 2024-04-25T18:57:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.425+0000 7f3879000700 1 -- 172.21.15.45:0/549678462 shutdown_connections 2024-04-25T18:57:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.425+0000 7f3879000700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f38580a3c40 0x7f38580a6100 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.425+0000 7f3879000700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3874108a30 0x7f387419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.425+0000 7f3879000700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f387406a750 0x7f387419b3b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.425+0000 7f3879000700 1 --2- 172.21.15.45:0/549678462 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f387406b0f0 0x7f387419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.425+0000 7f3879000700 1 -- 172.21.15.45:0/549678462 >> 172.21.15.45:0/549678462 conn(0x7f38740788f0 msgr2=0x7f387410df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.426+0000 7f3879000700 1 -- 172.21.15.45:0/549678462 shutdown_connections 2024-04-25T18:57:27.428 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.426+0000 7f3879000700 1 -- 172.21.15.45:0/549678462 wait complete. 2024-04-25T18:57:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.407+0000 7f5b6cc00700 1 -- 172.21.15.45:0/3422273996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6806a750 msgr2=0x7f5b6806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.407+0000 7f5b6cc00700 1 --2- 172.21.15.45:0/3422273996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6806a750 0x7f5b6806abb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5b540098f0 tx=0x7f5b54009c00 comp rx=0 tx=0).stop 2024-04-25T18:57:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.407+0000 7f5b6cc00700 1 -- 172.21.15.45:0/3422273996 shutdown_connections 2024-04-25T18:57:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.407+0000 7f5b6cc00700 1 --2- 172.21.15.45:0/3422273996 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b6806b0f0 0x7f5b6810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.407+0000 7f5b6cc00700 1 --2- 172.21.15.45:0/3422273996 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b68108a30 0x7f5b68108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.407+0000 7f5b6cc00700 1 --2- 172.21.15.45:0/3422273996 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6806a750 0x7f5b6806abb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.407+0000 7f5b6cc00700 1 -- 172.21.15.45:0/3422273996 >> 172.21.15.45:0/3422273996 conn(0x7f5b680788f0 msgr2=0x7f5b68078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:28.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.407+0000 7f5b6cc00700 1 -- 172.21.15.45:0/3422273996 shutdown_connections 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.407+0000 7f5b6cc00700 1 -- 172.21.15.45:0/3422273996 wait complete. 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.408+0000 7f5b6cc00700 1 Processor -- start 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b6cc00700 1 -- start start 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b6cc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b6806a750 0x7f5b6819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b6cc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6806b0f0 0x7f5b6819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b6cc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b68108a30 0x7f5b6819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b6cc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68112c60 con 0x7f5b6806a750 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b6cc00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68112ae0 con 0x7f5b6806b0f0 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b6cc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5b68112de0 con 0x7f5b68108a30 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b65e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b6806a750 0x7f5b6819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b66800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b68108a30 0x7f5b6819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b65e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b6806a750 0x7f5b6819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56060/0 (socket says 172.21.15.45:56060) 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b65e00700 1 -- 172.21.15.45:0/162361180 learned_addr learned my addr 172.21.15.45:0/162361180 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:28.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.409+0000 7f5b65400700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6806b0f0 0x7f5b6819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b66800700 1 -- 172.21.15.45:0/162361180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6806b0f0 msgr2=0x7f5b6819b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b66800700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6806b0f0 0x7f5b6819b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b66800700 1 -- 172.21.15.45:0/162361180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b6806a750 msgr2=0x7f5b6819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.158 INFO:teuthology.orchestra.run.smithi045.stdout:34359738470 2024-04-25T18:57:28.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b66800700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b6806a750 0x7f5b6819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b66800700 1 -- 172.21.15.45:0/162361180 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b681a03c0 con 0x7f5b68108a30 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b65e00700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b6806a750 0x7f5b6819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b66800700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b68108a30 0x7f5b6819fc80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000efc0 tx=0x7f5b6000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b5ea00700 1 -- 172.21.15.45:0/162361180 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b600092e0 con 0x7f5b68108a30 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b6cc00700 1 -- 172.21.15.45:0/162361180 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b681a0650 con 0x7f5b68108a30 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b5ea00700 1 -- 172.21.15.45:0/162361180 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b60004bb0 con 0x7f5b68108a30 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.410+0000 7f5b5ea00700 1 -- 172.21.15.45:0/162361180 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60004020 con 0x7f5b68108a30 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.411+0000 7f5b6cc00700 1 -- 172.21.15.45:0/162361180 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681a7f30 con 0x7f5b68108a30 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.412+0000 7f5b6cc00700 1 -- 172.21.15.45:0/162361180 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b68069730 con 0x7f5b68108a30 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.414+0000 7f5b5ea00700 1 -- 172.21.15.45:0/162361180 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5b6001b030 con 0x7f5b68108a30 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.415+0000 7f5b5ea00700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5b4c0a3c40 0x7f5b4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.415+0000 7f5b5ea00700 1 -- 172.21.15.45:0/162361180 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f5b600cff00 con 0x7f5b68108a30 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.415+0000 7f5b65e00700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5b4c0a3c40 0x7f5b4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:28.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.416+0000 7f5b65e00700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5b4c0a3c40 0x7f5b4c0a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b58005950 tx=0x7f5b58009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.419+0000 7f5b5ea00700 1 -- 172.21.15.45:0/162361180 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5b60022020 con 0x7f5b68108a30 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.667+0000 7f5b6cc00700 1 -- 172.21.15.45:0/162361180 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5b6802e140 con 0x7f5b68108a30 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.668+0000 7f5b5ea00700 1 -- 172.21.15.45:0/162361180 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5b60010040 con 0x7f5b68108a30 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 -- 172.21.15.45:0/162361180 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5b4c0a3c40 msgr2=0x7f5b4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5b4c0a3c40 0x7f5b4c0a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5b58005950 tx=0x7f5b58009310 comp rx=0 tx=0).stop 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 -- 172.21.15.45:0/162361180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b68108a30 msgr2=0x7f5b6819fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b68108a30 0x7f5b6819fc80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000efc0 tx=0x7f5b6000c3a0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 -- 172.21.15.45:0/162361180 shutdown_connections 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f5b4c0a3c40 0x7f5b4c0a6100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b68108a30 0x7f5b6819fc80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5b6806a750 0x7f5b6819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 --2- 172.21.15.45:0/162361180 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5b6806b0f0 0x7f5b6819b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 -- 172.21.15.45:0/162361180 >> 172.21.15.45:0/162361180 conn(0x7f5b680788f0 msgr2=0x7f5b6810df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.671+0000 7f5b6cc00700 1 -- 172.21.15.45:0/162361180 shutdown_connections 2024-04-25T18:57:28.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:27.672+0000 7f5b6cc00700 1 -- 172.21.15.45:0/162361180 wait complete. 2024-04-25T18:57:28.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:28 smithi062 ceph-mon[32844]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T18:57:28.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:28 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/549678462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T18:57:28.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:28 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/162361180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T18:57:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:28 smithi079 ceph-mon[33002]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T18:57:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:28 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/549678462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T18:57:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:28 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/162361180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T18:57:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:28 smithi045 ceph-mon[27776]: pgmap v389: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-04-25T18:57:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:28 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/549678462' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T18:57:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:28 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/162361180' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T18:57:28.653 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117072 got 115964117076 for osd.3 2024-04-25T18:57:28.654 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:30.313 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:30 smithi045 ceph-mon[27776]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T18:57:30.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:30 smithi062 ceph-mon[32844]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T18:57:30.374 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.372+0000 7fd23f000700 1 -- 172.21.15.45:0/902181154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd238076f70 msgr2=0x7fd238077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.372+0000 7fd23f000700 1 --2- 172.21.15.45:0/902181154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd238076f70 0x7fd238077350 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd2280098f0 tx=0x7fd228009c00 comp rx=0 tx=0).stop 2024-04-25T18:57:30.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.372+0000 7fd23f000700 1 -- 172.21.15.45:0/902181154 shutdown_connections 2024-04-25T18:57:30.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.372+0000 7fd23f000700 1 --2- 172.21.15.45:0/902181154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238113900 0x7fd238115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.372+0000 7fd23f000700 1 --2- 172.21.15.45:0/902181154 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd238076f70 0x7fd238077350 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.372+0000 7fd23f000700 1 --2- 172.21.15.45:0/902181154 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238077890 0x7fd2381133c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.372+0000 7fd23f000700 1 -- 172.21.15.45:0/902181154 >> 172.21.15.45:0/902181154 conn(0x7fd238100750 msgr2=0x7fd238102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:30.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.372+0000 7fd23f000700 1 -- 172.21.15.45:0/902181154 shutdown_connections 2024-04-25T18:57:30.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.372+0000 7fd23f000700 1 -- 172.21.15.45:0/902181154 wait complete. 2024-04-25T18:57:30.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.373+0000 7fd23f000700 1 Processor -- start 2024-04-25T18:57:30.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd23f000700 1 -- start start 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd23f000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd238076f70 0x7fd23819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd23f000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238077890 0x7fd23819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd23f000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238113900 0x7fd2381a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd23f000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd238119270 con 0x7fd238113900 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd23f000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd2381190f0 con 0x7fd238077890 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd23f000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd2381193f0 con 0x7fd238076f70 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd237e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd238076f70 0x7fd23819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd237e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd238076f70 0x7fd23819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41432/0 (socket says 172.21.15.45:41432) 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd237e00700 1 -- 172.21.15.45:0/3845824467 learned_addr learned my addr 172.21.15.45:0/3845824467 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.374+0000 7fd237400700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238077890 0x7fd23819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.375+0000 7fd23d000700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238113900 0x7fd2381a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.375+0000 7fd237e00700 1 -- 172.21.15.45:0/3845824467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238077890 msgr2=0x7fd23819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.375+0000 7fd237e00700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238077890 0x7fd23819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.375+0000 7fd237e00700 1 -- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238113900 msgr2=0x7fd2381a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.375+0000 7fd237e00700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238113900 0x7fd2381a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.375+0000 7fd237e00700 1 -- 172.21.15.45:0/3845824467 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2381a4760 con 0x7fd238076f70 2024-04-25T18:57:30.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.375+0000 7fd23d000700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238113900 0x7fd2381a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:57:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.375+0000 7fd237e00700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd238076f70 0x7fd23819f7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd22800f990 tx=0x7fd22800f9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.375+0000 7fd234c00700 1 -- 172.21.15.45:0/3845824467 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228027070 con 0x7fd238076f70 2024-04-25T18:57:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.376+0000 7fd234c00700 1 -- 172.21.15.45:0/3845824467 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2280055f0 con 0x7fd238076f70 2024-04-25T18:57:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.376+0000 7fd23f000700 1 -- 172.21.15.45:0/3845824467 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2381a49f0 con 0x7fd238076f70 2024-04-25T18:57:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.376+0000 7fd234c00700 1 -- 172.21.15.45:0/3845824467 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228004930 con 0x7fd238076f70 2024-04-25T18:57:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.376+0000 7fd23f000700 1 -- 172.21.15.45:0/3845824467 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2381ac2d0 con 0x7fd238076f70 2024-04-25T18:57:30.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.377+0000 7fd23f000700 1 -- 172.21.15.45:0/3845824467 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd238069730 con 0x7fd238076f70 2024-04-25T18:57:30.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.379+0000 7fd234c00700 1 -- 172.21.15.45:0/3845824467 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd22802a070 con 0x7fd238076f70 2024-04-25T18:57:30.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.380+0000 7fd234c00700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd21c0a3cc0 0x7fd21c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:30.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.380+0000 7fd234c00700 1 -- 172.21.15.45:0/3845824467 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fd2280d0ff0 con 0x7fd238076f70 2024-04-25T18:57:30.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.381+0000 7fd237400700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd21c0a3cc0 0x7fd21c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:30.383 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.381+0000 7fd237400700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd21c0a3cc0 0x7fd21c0a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd224009750 tx=0x7fd224009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:30.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.383+0000 7fd234c00700 1 -- 172.21.15.45:0/3845824467 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd228013030 con 0x7fd238076f70 2024-04-25T18:57:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:30 smithi079 ceph-mon[33002]: pgmap v390: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T18:57:30.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.629+0000 7fd23f000700 1 -- 172.21.15.45:0/3845824467 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd2381a0ab0 con 0x7fd238076f70 2024-04-25T18:57:30.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.629+0000 7fd234c00700 1 -- 172.21.15.45:0/3845824467 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd22809db60 con 0x7fd238076f70 2024-04-25T18:57:30.632 INFO:teuthology.orchestra.run.smithi045.stdout:171798691915 2024-04-25T18:57:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 -- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd21c0a3cc0 msgr2=0x7fd21c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd21c0a3cc0 0x7fd21c0a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd224009750 tx=0x7fd224009310 comp rx=0 tx=0).stop 2024-04-25T18:57:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 -- 172.21.15.45:0/3845824467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd238076f70 msgr2=0x7fd23819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd238076f70 0x7fd23819f7a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd22800f990 tx=0x7fd22800f9c0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 -- 172.21.15.45:0/3845824467 shutdown_connections 2024-04-25T18:57:30.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd21c0a3cc0 0x7fd21c0a6180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd238113900 0x7fd2381a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd238076f70 0x7fd23819f7a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 --2- 172.21.15.45:0/3845824467 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd238077890 0x7fd23819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 -- 172.21.15.45:0/3845824467 >> 172.21.15.45:0/3845824467 conn(0x7fd238100750 msgr2=0x7fd238102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 -- 172.21.15.45:0/3845824467 shutdown_connections 2024-04-25T18:57:30.636 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:30.633+0000 7fd23f000700 1 -- 172.21.15.45:0/3845824467 wait complete. 2024-04-25T18:57:31.178 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:31 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3845824467' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T18:57:31.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:31 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3845824467' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T18:57:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:31 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3845824467' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T18:57:32.287 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:32 smithi045 ceph-mon[27776]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:57:32.288 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:32 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2690220276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T18:57:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:32 smithi062 ceph-mon[32844]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:57:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:32 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2690220276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T18:57:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:32 smithi079 ceph-mon[33002]: pgmap v391: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T18:57:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:32 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2690220276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T18:57:32.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.357+0000 7f9908e00700 1 -- 172.21.15.45:0/3632836274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f990410f4f0 msgr2=0x7f99041118e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:32.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.357+0000 7f9908e00700 1 --2- 172.21.15.45:0/3632836274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f990410f4f0 0x7f99041118e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f98fc00b0a0 tx=0x7f98fc00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.357+0000 7f9908e00700 1 -- 172.21.15.45:0/3632836274 shutdown_connections 2024-04-25T18:57:32.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.357+0000 7f9908e00700 1 --2- 172.21.15.45:0/3632836274 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f990410f4f0 0x7f99041118e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.357+0000 7f9908e00700 1 --2- 172.21.15.45:0/3632836274 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f990406a750 0x7f990406ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.357+0000 7f9908e00700 1 --2- 172.21.15.45:0/3632836274 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990406b070 0x7f990410efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.357+0000 7f9908e00700 1 -- 172.21.15.45:0/3632836274 >> 172.21.15.45:0/3632836274 conn(0x7f99040788f0 msgr2=0x7f9904078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:32.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.357+0000 7f9908e00700 1 -- 172.21.15.45:0/3632836274 shutdown_connections 2024-04-25T18:57:32.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.357+0000 7f9908e00700 1 -- 172.21.15.45:0/3632836274 wait complete. 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.358+0000 7f9908e00700 1 Processor -- start 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.358+0000 7f9908e00700 1 -- start start 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f9908e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990406a750 0x7f990419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f9908e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f990406b070 0x7f990419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f9908e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f990410f4f0 0x7f990419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f9908e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9904114e10 con 0x7f990410f4f0 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f9908e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9904114c90 con 0x7f990406b070 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f9908e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9904114f90 con 0x7f990406a750 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f9901e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990406a750 0x7f990419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f9902800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f990410f4f0 0x7f990419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f9901e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990406a750 0x7f990419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41456/0 (socket says 172.21.15.45:41456) 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f9901e00700 1 -- 172.21.15.45:0/2690220276 learned_addr learned my addr 172.21.15.45:0/2690220276 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.359+0000 7f98f9400700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f990406b070 0x7f990419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:32.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.360+0000 7f9901e00700 1 -- 172.21.15.45:0/2690220276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f990406b070 msgr2=0x7f990419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.360+0000 7f9901e00700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f990406b070 0x7f990419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.360+0000 7f9901e00700 1 -- 172.21.15.45:0/2690220276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f990410f4f0 msgr2=0x7f990419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.360+0000 7f9901e00700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f990410f4f0 0x7f990419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.360+0000 7f9901e00700 1 -- 172.21.15.45:0/2690220276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99041a0300 con 0x7f990406a750 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.360+0000 7f9901e00700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990406a750 0x7f990419b340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e9a0 tx=0x7f98f400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.360+0000 7f98fb400700 1 -- 172.21.15.45:0/2690220276 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f400cbd0 con 0x7f990406a750 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.360+0000 7f9908e00700 1 -- 172.21.15.45:0/2690220276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99041a05f0 con 0x7f990406a750 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.360+0000 7f98fb400700 1 -- 172.21.15.45:0/2690220276 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98f400cd70 con 0x7f990406a750 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.360+0000 7f98fb400700 1 -- 172.21.15.45:0/2690220276 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4018680 con 0x7f990406a750 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.361+0000 7f9908e00700 1 -- 172.21.15.45:0/2690220276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99041a7e70 con 0x7f990406a750 2024-04-25T18:57:32.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.362+0000 7f9908e00700 1 -- 172.21.15.45:0/2690220276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9904069730 con 0x7f990406a750 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stdout:326417514545 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.364+0000 7f98fb400700 1 -- 172.21.15.45:0/2690220276 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f98f4010430 con 0x7f990406a750 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.365+0000 7f98fb400700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f98e80a3cc0 0x7f98e80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.365+0000 7f98fb400700 1 -- 172.21.15.45:0/2690220276 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f98f40d0640 con 0x7f990406a750 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.365+0000 7f98f9400700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f98e80a3cc0 0x7f98e80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.366+0000 7f98f9400700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f98e80a3cc0 0x7f98e80a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9904102f00 tx=0x7f98ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.370+0000 7f98fb400700 1 -- 172.21.15.45:0/2690220276 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98f409cfa0 con 0x7f990406a750 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.610+0000 7f9908e00700 1 -- 172.21.15.45:0/2690220276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f990402df80 con 0x7f990406a750 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.610+0000 7f98fb400700 1 -- 172.21.15.45:0/2690220276 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f98f409c8a0 con 0x7f990406a750 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 -- 172.21.15.45:0/2690220276 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f98e80a3cc0 msgr2=0x7f98e80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:32.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f98e80a3cc0 0x7f98e80a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9904102f00 tx=0x7f98ec009260 comp rx=0 tx=0).stop 2024-04-25T18:57:32.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 -- 172.21.15.45:0/2690220276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990406a750 msgr2=0x7f990419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:32.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990406a750 0x7f990419b340 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f98f400e9a0 tx=0x7f98f400ecb0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 -- 172.21.15.45:0/2690220276 shutdown_connections 2024-04-25T18:57:32.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f98e80a3cc0 0x7f98e80a6180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f990410f4f0 0x7f990419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f990406a750 0x7f990419b340 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 --2- 172.21.15.45:0/2690220276 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f990406b070 0x7f990419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:32.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 -- 172.21.15.45:0/2690220276 >> 172.21.15.45:0/2690220276 conn(0x7f99040788f0 msgr2=0x7f99041109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:32.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 -- 172.21.15.45:0/2690220276 shutdown_connections 2024-04-25T18:57:32.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:31.613+0000 7f9908e00700 1 -- 172.21.15.45:0/2690220276 wait complete. 2024-04-25T18:57:33.336 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:33 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3111607360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T18:57:33.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:33 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3111607360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T18:57:33.457 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:33 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3111607360' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T18:57:33.546 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691909 got 171798691915 for osd.5 2024-04-25T18:57:33.547 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.462+0000 7ff1f6000700 1 -- 172.21.15.45:0/4141124036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f0104450 msgr2=0x7ff1f010c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.462+0000 7ff1f6000700 1 --2- 172.21.15.45:0/4141124036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f0104450 0x7ff1f010c940 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0098f0 tx=0x7ff1dc009c00 comp rx=0 tx=0).stop 2024-04-25T18:57:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.462+0000 7ff1f6000700 1 -- 172.21.15.45:0/4141124036 shutdown_connections 2024-04-25T18:57:33.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.462+0000 7ff1f6000700 1 --2- 172.21.15.45:0/4141124036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f010cf60 0x7ff1f010f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.462+0000 7ff1f6000700 1 --2- 172.21.15.45:0/4141124036 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103b30 0x7ff1f0103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.462+0000 7ff1f6000700 1 --2- 172.21.15.45:0/4141124036 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f0104450 0x7ff1f010c940 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.462+0000 7ff1f6000700 1 -- 172.21.15.45:0/4141124036 >> 172.21.15.45:0/4141124036 conn(0x7ff1f00fd360 msgr2=0x7ff1f00ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.462+0000 7ff1f6000700 1 -- 172.21.15.45:0/4141124036 shutdown_connections 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.462+0000 7ff1f6000700 1 -- 172.21.15.45:0/4141124036 wait complete. 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.463+0000 7ff1f6000700 1 Processor -- start 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f6000700 1 -- start start 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f6000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103b30 0x7ff1f019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f6000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f010cf60 0x7ff1f019bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f6000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f019feb0 0x7ff1f01a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f6000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0112b90 con 0x7ff1f0103b30 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f6000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0112a10 con 0x7ff1f019feb0 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f6000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0112d10 con 0x7ff1f010cf60 2024-04-25T18:57:33.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f4c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103b30 0x7ff1f019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f5600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f019feb0 0x7ff1f01a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1efe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f010cf60 0x7ff1f019bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f5600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f019feb0 0x7ff1f01a0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:51530/0 (socket says 172.21.15.45:51530) 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f4c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103b30 0x7ff1f019b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56104/0 (socket says 172.21.15.45:56104) 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.464+0000 7ff1f5600700 1 -- 172.21.15.45:0/3111607360 learned_addr learned my addr 172.21.15.45:0/3111607360 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.465+0000 7ff1efe00700 1 -- 172.21.15.45:0/3111607360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f019feb0 msgr2=0x7ff1f01a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.465+0000 7ff1efe00700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f019feb0 0x7ff1f01a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.465+0000 7ff1efe00700 1 -- 172.21.15.45:0/3111607360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103b30 msgr2=0x7ff1f019b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.465+0000 7ff1efe00700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103b30 0x7ff1f019b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.465+0000 7ff1efe00700 1 -- 172.21.15.45:0/3111607360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f01a09d0 con 0x7ff1f010cf60 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.465+0000 7ff1efe00700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f010cf60 0x7ff1f019bb20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1f006aa30 tx=0x7ff1dc00f950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.465+0000 7ff1ed600700 1 -- 172.21.15.45:0/3111607360 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc005040 con 0x7ff1f010cf60 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.465+0000 7ff1f6000700 1 -- 172.21.15.45:0/3111607360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f01a0c60 con 0x7ff1f010cf60 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.466+0000 7ff1ed600700 1 -- 172.21.15.45:0/3111607360 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1dc005690 con 0x7ff1f010cf60 2024-04-25T18:57:33.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.466+0000 7ff1ed600700 1 -- 172.21.15.45:0/3111607360 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc0038e0 con 0x7ff1f010cf60 2024-04-25T18:57:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.466+0000 7ff1f6000700 1 -- 172.21.15.45:0/3111607360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f01a8390 con 0x7ff1f010cf60 2024-04-25T18:57:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.467+0000 7ff1f6000700 1 -- 172.21.15.45:0/3111607360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f0069730 con 0x7ff1f010cf60 2024-04-25T18:57:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.469+0000 7ff1ed600700 1 -- 172.21.15.45:0/3111607360 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff1dc00b560 con 0x7ff1f010cf60 2024-04-25T18:57:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.470+0000 7ff1ed600700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1d80a3c40 0x7ff1d80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.471+0000 7ff1ed600700 1 -- 172.21.15.45:0/3111607360 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7ff1dc0d1340 con 0x7ff1f010cf60 2024-04-25T18:57:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.471+0000 7ff1f4c00700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1d80a3c40 0x7ff1d80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.471+0000 7ff1f4c00700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1d80a3c40 0x7ff1d80a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400f540 tx=0x7ff1e400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:33.888 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.472+0000 7ff1ed600700 1 -- 172.21.15.45:0/3111607360 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1dc013030 con 0x7ff1f010cf60 2024-04-25T18:57:33.888 INFO:teuthology.orchestra.run.smithi045.stdout:266287972412 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.711+0000 7ff1f6000700 1 -- 172.21.15.45:0/3111607360 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff1f002df80 con 0x7ff1f010cf60 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.712+0000 7ff1ed600700 1 -- 172.21.15.45:0/3111607360 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff1dc09dd30 con 0x7ff1f010cf60 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.714+0000 7ff1f6000700 1 -- 172.21.15.45:0/3111607360 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1d80a3c40 msgr2=0x7ff1d80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.714+0000 7ff1f6000700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1d80a3c40 0x7ff1d80a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff1e400f540 tx=0x7ff1e400b040 comp rx=0 tx=0).stop 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.715+0000 7ff1f6000700 1 -- 172.21.15.45:0/3111607360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f010cf60 msgr2=0x7ff1f019bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.715+0000 7ff1f6000700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f010cf60 0x7ff1f019bb20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff1f006aa30 tx=0x7ff1dc00f950 comp rx=0 tx=0).stop 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.715+0000 7ff1f6000700 1 -- 172.21.15.45:0/3111607360 shutdown_connections 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.715+0000 7ff1f6000700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff1d80a3c40 0x7ff1d80a6100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.715+0000 7ff1f6000700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff1f019feb0 0x7ff1f01a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.715+0000 7ff1f6000700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103b30 0x7ff1f019b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.715+0000 7ff1f6000700 1 --2- 172.21.15.45:0/3111607360 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff1f010cf60 0x7ff1f019bb20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:33.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.715+0000 7ff1f6000700 1 -- 172.21.15.45:0/3111607360 >> 172.21.15.45:0/3111607360 conn(0x7ff1f00fd360 msgr2=0x7ff1f00ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.715+0000 7ff1f6000700 1 -- 172.21.15.45:0/3111607360 shutdown_connections 2024-04-25T18:57:33.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:32.715+0000 7ff1f6000700 1 -- 172.21.15.45:0/3111607360 wait complete. 2024-04-25T18:57:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:34 smithi062 ceph-mon[32844]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T18:57:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:34 smithi079 ceph-mon[33002]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T18:57:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:34 smithi045 ceph-mon[27776]: pgmap v392: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 6 op/s 2024-04-25T18:57:34.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972405 got 266287972412 for osd.8 2024-04-25T18:57:34.592 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:34.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.817+0000 7fd614c00700 1 -- 172.21.15.45:0/1578231387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6101092d0 msgr2=0x7fd6101096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:34.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.817+0000 7fd614c00700 1 --2- 172.21.15.45:0/1578231387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6101092d0 0x7fd6101096b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd600009990 tx=0x7fd600009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:34.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.817+0000 7fd614c00700 1 -- 172.21.15.45:0/1578231387 shutdown_connections 2024-04-25T18:57:34.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.817+0000 7fd614c00700 1 --2- 172.21.15.45:0/1578231387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd6101092d0 0x7fd6101096b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:34.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.817+0000 7fd614c00700 1 --2- 172.21.15.45:0/1578231387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd61010e3b0 0x7fd610115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:34.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.817+0000 7fd614c00700 1 --2- 172.21.15.45:0/1578231387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd610109c80 0x7fd61010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:34.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.817+0000 7fd614c00700 1 -- 172.21.15.45:0/1578231387 >> 172.21.15.45:0/1578231387 conn(0x7fd610076210 msgr2=0x7fd610107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.817+0000 7fd614c00700 1 -- 172.21.15.45:0/1578231387 shutdown_connections 2024-04-25T18:57:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.817+0000 7fd614c00700 1 -- 172.21.15.45:0/1578231387 wait complete. 2024-04-25T18:57:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.818+0000 7fd614c00700 1 Processor -- start 2024-04-25T18:57:34.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.818+0000 7fd614c00700 1 -- start start 2024-04-25T18:57:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd614c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6101092d0 0x7fd6101a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd614c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd610109c80 0x7fd6101a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd614c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61010e3b0 0x7fd61019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd614c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd610119290 con 0x7fd6101092d0 2024-04-25T18:57:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd614c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd610119110 con 0x7fd61010e3b0 2024-04-25T18:57:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd614c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd610119410 con 0x7fd610109c80 2024-04-25T18:57:34.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd60de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6101092d0 0x7fd6101a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd60e800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61010e3b0 0x7fd61019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd60de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6101092d0 0x7fd6101a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43440/0 (socket says 172.21.15.45:43440) 2024-04-25T18:57:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd60de00700 1 -- 172.21.15.45:0/774791985 learned_addr learned my addr 172.21.15.45:0/774791985 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:34.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.819+0000 7fd60d400700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd610109c80 0x7fd6101a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd60e800700 1 -- 172.21.15.45:0/774791985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd610109c80 msgr2=0x7fd6101a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd60e800700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd610109c80 0x7fd6101a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd60e800700 1 -- 172.21.15.45:0/774791985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6101092d0 msgr2=0x7fd6101a53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd60e800700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6101092d0 0x7fd6101a53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd60e800700 1 -- 172.21.15.45:0/774791985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61019fd50 con 0x7fd61010e3b0 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd60e800700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61010e3b0 0x7fd61019f5f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd60800efb0 tx=0x7fd60800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd606a00700 1 -- 172.21.15.45:0/774791985 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608003a00 con 0x7fd61010e3b0 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd614c00700 1 -- 172.21.15.45:0/774791985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6101a0040 con 0x7fd61010e3b0 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd606a00700 1 -- 172.21.15.45:0/774791985 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd60800ce50 con 0x7fd61010e3b0 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd606a00700 1 -- 172.21.15.45:0/774791985 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608004710 con 0x7fd61010e3b0 2024-04-25T18:57:34.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.820+0000 7fd614c00700 1 -- 172.21.15.45:0/774791985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6101ac2b0 con 0x7fd61010e3b0 2024-04-25T18:57:34.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.822+0000 7fd614c00700 1 -- 172.21.15.45:0/774791985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd610069730 con 0x7fd61010e3b0 2024-04-25T18:57:34.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.823+0000 7fd606a00700 1 -- 172.21.15.45:0/774791985 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd608012050 con 0x7fd61010e3b0 2024-04-25T18:57:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.825+0000 7fd606a00700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd5f40a3c40 0x7fd5f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.825+0000 7fd606a00700 1 -- 172.21.15.45:0/774791985 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fd6080cfc50 con 0x7fd61010e3b0 2024-04-25T18:57:34.827 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.825+0000 7fd60de00700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd5f40a3c40 0x7fd5f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:34.828 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.826+0000 7fd60de00700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd5f40a3c40 0x7fd5f40a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b790 tx=0x7fd60000b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:34.831 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:34.830+0000 7fd606a00700 1 -- 172.21.15.45:0/774791985 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd608022060 con 0x7fd61010e3b0 2024-04-25T18:57:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.066+0000 7fd614c00700 1 -- 172.21.15.45:0/774791985 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd61002df80 con 0x7fd61010e3b0 2024-04-25T18:57:35.068 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.066+0000 7fd606a00700 1 -- 172.21.15.45:0/774791985 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd608010040 con 0x7fd61010e3b0 2024-04-25T18:57:35.069 INFO:teuthology.orchestra.run.smithi045.stdout:150323855441 2024-04-25T18:57:35.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.069+0000 7fd614c00700 1 -- 172.21.15.45:0/774791985 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd5f40a3c40 msgr2=0x7fd5f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:35.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.069+0000 7fd614c00700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd5f40a3c40 0x7fd5f40a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd60000b790 tx=0x7fd60000b280 comp rx=0 tx=0).stop 2024-04-25T18:57:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.069+0000 7fd614c00700 1 -- 172.21.15.45:0/774791985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61010e3b0 msgr2=0x7fd61019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.069+0000 7fd614c00700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61010e3b0 0x7fd61019f5f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd60800efb0 tx=0x7fd60800c3a0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.069+0000 7fd614c00700 1 -- 172.21.15.45:0/774791985 shutdown_connections 2024-04-25T18:57:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.069+0000 7fd614c00700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd5f40a3c40 0x7fd5f40a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.069+0000 7fd614c00700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd61010e3b0 0x7fd61019f5f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.069+0000 7fd614c00700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd6101092d0 0x7fd6101a53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.069+0000 7fd614c00700 1 --2- 172.21.15.45:0/774791985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd610109c80 0x7fd6101a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.069+0000 7fd614c00700 1 -- 172.21.15.45:0/774791985 >> 172.21.15.45:0/774791985 conn(0x7fd610076210 msgr2=0x7fd610077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:35.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.070+0000 7fd614c00700 1 -- 172.21.15.45:0/774791985 shutdown_connections 2024-04-25T18:57:35.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:35.070+0000 7fd614c00700 1 -- 172.21.15.45:0/774791985 wait complete. 2024-04-25T18:57:35.324 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:35 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/774791985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T18:57:35.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:35 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/774791985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T18:57:35.405 INFO:tasks.cephadm.ceph_manager.ceph:need seq 360777252902 got 360777252906 for osd.11 2024-04-25T18:57:35.405 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:35 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/774791985' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T18:57:35.485 INFO:tasks.cephadm.ceph_manager.ceph:need seq 300647710765 got 300647710771 for osd.9 2024-04-25T18:57:35.485 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:36.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:36 smithi045 ceph-mon[27776]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T18:57:36.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:36 smithi062 ceph-mon[32844]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T18:57:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:36 smithi079 ceph-mon[33002]: pgmap v393: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T18:57:37.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514540 got 326417514545 for osd.10 2024-04-25T18:57:37.566 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.135+0000 7f7d02c00700 1 -- 172.21.15.45:0/439155169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cfc108a80 msgr2=0x7f7cfc108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.135+0000 7f7d02c00700 1 --2- 172.21.15.45:0/439155169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cfc108a80 0x7f7cfc108e60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009990 tx=0x7f7cec009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.135+0000 7f7d02c00700 1 -- 172.21.15.45:0/439155169 shutdown_connections 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.135+0000 7f7d02c00700 1 --2- 172.21.15.45:0/439155169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc103450 0x7f7cfc10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.135+0000 7f7d02c00700 1 --2- 172.21.15.45:0/439155169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cfc108a80 0x7f7cfc108e60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.135+0000 7f7d02c00700 1 --2- 172.21.15.45:0/439155169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cfc102a90 0x7f7cfc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.135+0000 7f7d02c00700 1 -- 172.21.15.45:0/439155169 >> 172.21.15.45:0/439155169 conn(0x7f7cfc079bf0 msgr2=0x7f7cfc07a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.135+0000 7f7d02c00700 1 -- 172.21.15.45:0/439155169 shutdown_connections 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.135+0000 7f7d02c00700 1 -- 172.21.15.45:0/439155169 wait complete. 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.136+0000 7f7d02c00700 1 Processor -- start 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.136+0000 7f7d02c00700 1 -- start start 2024-04-25T18:57:37.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7d02c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cfc102a90 0x7f7cfc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7d02c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cfc103450 0x7f7cfc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7d02c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc108a80 0x7f7cfc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7d02c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc112c30 con 0x7f7cfc102a90 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7d02c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc112ab0 con 0x7f7cfc108a80 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7d02c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7cfc112db0 con 0x7f7cfc103450 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7cfbe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cfc102a90 0x7f7cfc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7cfb400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cfc103450 0x7f7cfc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7cfbe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cfc102a90 0x7f7cfc074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43456/0 (socket says 172.21.15.45:43456) 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7cfbe00700 1 -- 172.21.15.45:0/3123834146 learned_addr learned my addr 172.21.15.45:0/3123834146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7d00c00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc108a80 0x7f7cfc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7cfb400700 1 -- 172.21.15.45:0/3123834146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc108a80 msgr2=0x7f7cfc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7cfb400700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc108a80 0x7f7cfc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7cfb400700 1 -- 172.21.15.45:0/3123834146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cfc102a90 msgr2=0x7f7cfc074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:37.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7cfb400700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cfc102a90 0x7f7cfc074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7cfb400700 1 -- 172.21.15.45:0/3123834146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cfc1a7690 con 0x7f7cfc103450 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.137+0000 7f7d00c00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc108a80 0x7f7cfc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.138+0000 7f7cfb400700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cfc103450 0x7f7cfc075280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800e9f0 tx=0x7f7ce800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.138+0000 7f7cf8c00700 1 -- 172.21.15.45:0/3123834146 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800cbd0 con 0x7f7cfc103450 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.138+0000 7f7cf8c00700 1 -- 172.21.15.45:0/3123834146 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce80044d0 con 0x7f7cfc103450 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.138+0000 7f7cf8c00700 1 -- 172.21.15.45:0/3123834146 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8005020 con 0x7f7cfc103450 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.138+0000 7f7d02c00700 1 -- 172.21.15.45:0/3123834146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cfc1a7980 con 0x7f7cfc103450 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.138+0000 7f7d02c00700 1 -- 172.21.15.45:0/3123834146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cfc1a7ec0 con 0x7f7cfc103450 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.139+0000 7f7d02c00700 1 -- 172.21.15.45:0/3123834146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cfc069730 con 0x7f7cfc103450 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.141+0000 7f7cf8c00700 1 -- 172.21.15.45:0/3123834146 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ce8004750 con 0x7f7cfc103450 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.142+0000 7f7cf8c00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ce00a3cc0 0x7f7ce00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.143+0000 7f7cf8c00700 1 -- 172.21.15.45:0/3123834146 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f7ce80d0380 con 0x7f7cfc103450 2024-04-25T18:57:37.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.143+0000 7f7cfbe00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ce00a3cc0 0x7f7ce00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.143+0000 7f7cfbe00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ce00a3cc0 0x7f7ce00a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7cec005cd0 tx=0x7f7cec005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:37.727 INFO:teuthology.orchestra.run.smithi045.stdout:77309411422 2024-04-25T18:57:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.147+0000 7f7cf8c00700 1 -- 172.21.15.45:0/3123834146 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ce809cce0 con 0x7f7cfc103450 2024-04-25T18:57:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.381+0000 7f7d02c00700 1 -- 172.21.15.45:0/3123834146 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7cfc02df80 con 0x7f7cfc103450 2024-04-25T18:57:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.381+0000 7f7cf8c00700 1 -- 172.21.15.45:0/3123834146 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7ce809c5e0 con 0x7f7cfc103450 2024-04-25T18:57:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.384+0000 7f7d02c00700 1 -- 172.21.15.45:0/3123834146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ce00a3cc0 msgr2=0x7f7ce00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.384+0000 7f7d02c00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ce00a3cc0 0x7f7ce00a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7cec005cd0 tx=0x7f7cec005c60 comp rx=0 tx=0).stop 2024-04-25T18:57:37.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.384+0000 7f7d02c00700 1 -- 172.21.15.45:0/3123834146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cfc103450 msgr2=0x7f7cfc075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.384+0000 7f7d02c00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cfc103450 0x7f7cfc075280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800e9f0 tx=0x7f7ce800ed00 comp rx=0 tx=0).stop 2024-04-25T18:57:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.385+0000 7f7d02c00700 1 -- 172.21.15.45:0/3123834146 shutdown_connections 2024-04-25T18:57:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.385+0000 7f7d02c00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ce00a3cc0 0x7f7ce00a6180 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.385+0000 7f7d02c00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7cfc108a80 0x7f7cfc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.385+0000 7f7d02c00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7cfc102a90 0x7f7cfc074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.385+0000 7f7d02c00700 1 --2- 172.21.15.45:0/3123834146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7cfc103450 0x7f7cfc075280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.385+0000 7f7d02c00700 1 -- 172.21.15.45:0/3123834146 >> 172.21.15.45:0/3123834146 conn(0x7f7cfc079bf0 msgr2=0x7f7cfc101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.385+0000 7f7d02c00700 1 -- 172.21.15.45:0/3123834146 shutdown_connections 2024-04-25T18:57:37.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:37.385+0000 7f7d02c00700 1 -- 172.21.15.45:0/3123834146 wait complete. 2024-04-25T18:57:38.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:38 smithi062 ceph-mon[32844]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T18:57:38.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:38 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3123834146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T18:57:38.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:38 smithi045 ceph-mon[27776]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T18:57:38.394 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:38 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3123834146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T18:57:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:38 smithi079 ceph-mon[33002]: pgmap v394: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T18:57:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:38 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3123834146' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T18:57:38.456 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738465 got 34359738470 for osd.0 2024-04-25T18:57:38.457 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:39.186 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855436 got 150323855441 for osd.4 2024-04-25T18:57:39.186 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:39.273 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430271 got 206158430275 for osd.6 2024-04-25T18:57:39.273 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.559+0000 7fb952000700 1 -- 172.21.15.45:0/3868685688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c108a30 msgr2=0x7fb94c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.559+0000 7fb952000700 1 --2- 172.21.15.45:0/3868685688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c108a30 0x7fb94c108e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb93c0098f0 tx=0x7fb93c009c00 comp rx=0 tx=0).stop 2024-04-25T18:57:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.559+0000 7fb952000700 1 -- 172.21.15.45:0/3868685688 shutdown_connections 2024-04-25T18:57:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.559+0000 7fb952000700 1 --2- 172.21.15.45:0/3868685688 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c108a30 0x7fb94c108e10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.559+0000 7fb952000700 1 --2- 172.21.15.45:0/3868685688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb94c06b0f0 0x7fb94c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.559+0000 7fb952000700 1 --2- 172.21.15.45:0/3868685688 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94c06a750 0x7fb94c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.559+0000 7fb952000700 1 -- 172.21.15.45:0/3868685688 >> 172.21.15.45:0/3868685688 conn(0x7fb94c0788f0 msgr2=0x7fb94c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.560+0000 7fb952000700 1 -- 172.21.15.45:0/3868685688 shutdown_connections 2024-04-25T18:57:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.560+0000 7fb952000700 1 -- 172.21.15.45:0/3868685688 wait complete. 2024-04-25T18:57:39.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.561+0000 7fb952000700 1 Processor -- start 2024-04-25T18:57:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.561+0000 7fb952000700 1 -- start start 2024-04-25T18:57:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb952000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c06a750 0x7fb94c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb952000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94c06b0f0 0x7fb94c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb952000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb94c108a30 0x7fb94c19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb952000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c112c20 con 0x7fb94c06b0f0 2024-04-25T18:57:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb952000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c112aa0 con 0x7fb94c06a750 2024-04-25T18:57:39.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb952000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c112da0 con 0x7fb94c108a30 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb94aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94c06b0f0 0x7fb94c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb94b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c06a750 0x7fb94c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb94aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94c06b0f0 0x7fb94c199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43470/0 (socket says 172.21.15.45:43470) 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb94aa00700 1 -- 172.21.15.45:0/946815338 learned_addr learned my addr 172.21.15.45:0/946815338 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.562+0000 7fb94be00700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb94c108a30 0x7fb94c19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb94b400700 1 -- 172.21.15.45:0/946815338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb94c108a30 msgr2=0x7fb94c19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb94b400700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb94c108a30 0x7fb94c19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb94b400700 1 -- 172.21.15.45:0/946815338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94c06b0f0 msgr2=0x7fb94c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb94b400700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94c06b0f0 0x7fb94c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb94b400700 1 -- 172.21.15.45:0/946815338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c19e160 con 0x7fb94c06a750 2024-04-25T18:57:39.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb94be00700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb94c108a30 0x7fb94c19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb94b400700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c06a750 0x7fb94c199150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00c010 tx=0x7fb93c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb943e00700 1 -- 172.21.15.45:0/946815338 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0050c0 con 0x7fb94c06a750 2024-04-25T18:57:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb943e00700 1 -- 172.21.15.45:0/946815338 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb93c00b560 con 0x7fb94c06a750 2024-04-25T18:57:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb952000700 1 -- 172.21.15.45:0/946815338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c19e3f0 con 0x7fb94c06a750 2024-04-25T18:57:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.563+0000 7fb943e00700 1 -- 172.21.15.45:0/946815338 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c003a00 con 0x7fb94c06a750 2024-04-25T18:57:39.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.564+0000 7fb952000700 1 -- 172.21.15.45:0/946815338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c1a5cd0 con 0x7fb94c06a750 2024-04-25T18:57:39.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.565+0000 7fb952000700 1 -- 172.21.15.45:0/946815338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c069730 con 0x7fb94c06a750 2024-04-25T18:57:39.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.567+0000 7fb943e00700 1 -- 172.21.15.45:0/946815338 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb93c00b710 con 0x7fb94c06a750 2024-04-25T18:57:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.569+0000 7fb943e00700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb9300a3cc0 0x7fb9300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.569+0000 7fb943e00700 1 -- 172.21.15.45:0/946815338 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fb93c0d1190 con 0x7fb94c06a750 2024-04-25T18:57:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.569+0000 7fb94aa00700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb9300a3cc0 0x7fb9300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:39.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.570+0000 7fb94aa00700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb9300a3cc0 0x7fb9300a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb94c078fd0 tx=0x7fb938009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:39.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.574+0000 7fb943e00700 1 -- 172.21.15.45:0/946815338 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb93c013030 con 0x7fb94c06a750 2024-04-25T18:57:39.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.810+0000 7fb952000700 1 -- 172.21.15.45:0/946815338 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb94c02df80 con 0x7fb94c06a750 2024-04-25T18:57:39.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.811+0000 7fb943e00700 1 -- 172.21.15.45:0/946815338 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb93c021380 con 0x7fb94c06a750 2024-04-25T18:57:39.813 INFO:teuthology.orchestra.run.smithi045.stdout:231928234051 2024-04-25T18:57:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 -- 172.21.15.45:0/946815338 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb9300a3cc0 msgr2=0x7fb9300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb9300a3cc0 0x7fb9300a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb94c078fd0 tx=0x7fb938009310 comp rx=0 tx=0).stop 2024-04-25T18:57:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 -- 172.21.15.45:0/946815338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c06a750 msgr2=0x7fb94c199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:39.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c06a750 0x7fb94c199150 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00c010 tx=0x7fb93c00fa00 comp rx=0 tx=0).stop 2024-04-25T18:57:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 -- 172.21.15.45:0/946815338 shutdown_connections 2024-04-25T18:57:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb9300a3cc0 0x7fb9300a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb94c06a750 0x7fb94c199150 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb94c108a30 0x7fb94c19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 --2- 172.21.15.45:0/946815338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb94c06b0f0 0x7fb94c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 -- 172.21.15.45:0/946815338 >> 172.21.15.45:0/946815338 conn(0x7fb94c0788f0 msgr2=0x7fb94c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.814+0000 7fb952000700 1 -- 172.21.15.45:0/946815338 shutdown_connections 2024-04-25T18:57:39.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:39.815+0000 7fb952000700 1 -- 172.21.15.45:0/946815338 wait complete. 2024-04-25T18:57:40.034 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411416 got 77309411422 for osd.2 2024-04-25T18:57:40.034 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:40.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:40 smithi045 ceph-mon[27776]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-25T18:57:40.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:40 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/946815338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T18:57:40.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:40 smithi062 ceph-mon[32844]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-25T18:57:40.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:40 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/946815338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T18:57:40.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:40 smithi079 ceph-mon[33002]: pgmap v395: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s 2024-04-25T18:57:40.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:40 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/946815338' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T18:57:40.795 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234045 got 231928234051 for osd.7 2024-04-25T18:57:40.795 DEBUG:teuthology.parallel:result is None 2024-04-25T18:57:40.795 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T18:57:40.795 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:57:41.648 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:42.422 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:42 smithi062 ceph-mon[32844]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:57:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:42 smithi079 ceph-mon[33002]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:57:42.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:42 smithi045 ceph-mon[27776]: pgmap v396: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T18:57:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.933+0000 7fd43cc00700 1 -- 172.21.15.45:0/3434530614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd438103450 msgr2=0x7fd43810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.933+0000 7fd43cc00700 1 --2- 172.21.15.45:0/3434530614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd438103450 0x7fd43810f8b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd43000b0a0 tx=0x7fd43000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:57:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.933+0000 7fd43cc00700 1 -- 172.21.15.45:0/3434530614 shutdown_connections 2024-04-25T18:57:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.933+0000 7fd43cc00700 1 --2- 172.21.15.45:0/3434530614 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd438103450 0x7fd43810f8b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.933+0000 7fd43cc00700 1 --2- 172.21.15.45:0/3434530614 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd438108a50 0x7fd438108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.933+0000 7fd43cc00700 1 --2- 172.21.15.45:0/3434530614 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd438102a90 0x7fd438102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.933+0000 7fd43cc00700 1 -- 172.21.15.45:0/3434530614 >> 172.21.15.45:0/3434530614 conn(0x7fd438079ba0 msgr2=0x7fd438079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.933+0000 7fd43cc00700 1 -- 172.21.15.45:0/3434530614 shutdown_connections 2024-04-25T18:57:43.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.933+0000 7fd43cc00700 1 -- 172.21.15.45:0/3434530614 wait complete. 2024-04-25T18:57:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.934+0000 7fd43cc00700 1 Processor -- start 2024-04-25T18:57:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd43cc00700 1 -- start start 2024-04-25T18:57:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd43cc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd438102a90 0x7fd43819b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd43cc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd438103450 0x7fd43819b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd43cc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd438108a50 0x7fd43819fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd43cc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd438112be0 con 0x7fd438103450 2024-04-25T18:57:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd43cc00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd438112a60 con 0x7fd438108a50 2024-04-25T18:57:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd43cc00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd438112d60 con 0x7fd438102a90 2024-04-25T18:57:43.938 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd435400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd438103450 0x7fd43819b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd435400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd438103450 0x7fd43819b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43488/0 (socket says 172.21.15.45:43488) 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd435400700 1 -- 172.21.15.45:0/1961424238 learned_addr learned my addr 172.21.15.45:0/1961424238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.935+0000 7fd435e00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd438102a90 0x7fd43819b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.936+0000 7fd436800700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd438108a50 0x7fd43819fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.936+0000 7fd435400700 1 -- 172.21.15.45:0/1961424238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd438102a90 msgr2=0x7fd43819b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.936+0000 7fd435400700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd438102a90 0x7fd43819b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.936+0000 7fd435400700 1 -- 172.21.15.45:0/1961424238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd438108a50 msgr2=0x7fd43819fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.936+0000 7fd435400700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd438108a50 0x7fd43819fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.936+0000 7fd435400700 1 -- 172.21.15.45:0/1961424238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4381a02e0 con 0x7fd438103450 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.936+0000 7fd435e00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd438102a90 0x7fd43819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:43.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.936+0000 7fd435400700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd438103450 0x7fd43819b810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd42400eb30 tx=0x7fd42400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.937+0000 7fd42ea00700 1 -- 172.21.15.45:0/1961424238 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42400cc90 con 0x7fd438103450 2024-04-25T18:57:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.937+0000 7fd42ea00700 1 -- 172.21.15.45:0/1961424238 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4240044d0 con 0x7fd438103450 2024-04-25T18:57:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.937+0000 7fd42ea00700 1 -- 172.21.15.45:0/1961424238 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd424010430 con 0x7fd438103450 2024-04-25T18:57:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.937+0000 7fd43cc00700 1 -- 172.21.15.45:0/1961424238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4381a05d0 con 0x7fd438103450 2024-04-25T18:57:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.937+0000 7fd43cc00700 1 -- 172.21.15.45:0/1961424238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4381a7e50 con 0x7fd438103450 2024-04-25T18:57:43.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.938+0000 7fd43cc00700 1 -- 172.21.15.45:0/1961424238 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd438069730 con 0x7fd438103450 2024-04-25T18:57:43.942 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.940+0000 7fd42ea00700 1 -- 172.21.15.45:0/1961424238 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd424003ea0 con 0x7fd438103450 2024-04-25T18:57:43.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.942+0000 7fd42ea00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd41c0a3c40 0x7fd41c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:43.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.942+0000 7fd42ea00700 1 -- 172.21.15.45:0/1961424238 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fd4240d0760 con 0x7fd438103450 2024-04-25T18:57:43.944 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.942+0000 7fd435e00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd41c0a3c40 0x7fd41c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:43.945 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.943+0000 7fd435e00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd41c0a3c40 0x7fd41c0a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd428009680 tx=0x7fd428009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:43.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:43.947+0000 7fd42ea00700 1 -- 172.21.15.45:0/1961424238 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd42409d0c0 con 0x7fd438103450 2024-04-25T18:57:44.185 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.183+0000 7fd43cc00700 1 -- 172.21.15.45:0/1961424238 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd43810f5c0 con 0x7fd41c0a3c40 2024-04-25T18:57:44.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.201+0000 7fd42ea00700 1 -- 172.21.15.45:0/1961424238 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258850 (secure 0 0 0) 0x7fd43810f5c0 con 0x7fd41c0a3c40 2024-04-25T18:57:44.209 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:57:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.207+0000 7fd43cc00700 1 -- 172.21.15.45:0/1961424238 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd41c0a3c40 msgr2=0x7fd41c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:44.211 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.207+0000 7fd43cc00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd41c0a3c40 0x7fd41c0a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd428009680 tx=0x7fd428009260 comp rx=0 tx=0).stop 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.207+0000 7fd43cc00700 1 -- 172.21.15.45:0/1961424238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd438103450 msgr2=0x7fd43819b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.207+0000 7fd43cc00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd438103450 0x7fd43819b810 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd42400eb30 tx=0x7fd42400ee40 comp rx=0 tx=0).stop 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.208+0000 7fd43cc00700 1 -- 172.21.15.45:0/1961424238 shutdown_connections 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.208+0000 7fd43cc00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fd41c0a3c40 0x7fd41c0a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.208+0000 7fd43cc00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd438108a50 0x7fd43819fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.208+0000 7fd43cc00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd438102a90 0x7fd43819b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.208+0000 7fd43cc00700 1 --2- 172.21.15.45:0/1961424238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd438103450 0x7fd43819b810 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.208+0000 7fd43cc00700 1 -- 172.21.15.45:0/1961424238 >> 172.21.15.45:0/1961424238 conn(0x7fd438079ba0 msgr2=0x7fd43810dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.208+0000 7fd43cc00700 1 -- 172.21.15.45:0/1961424238 shutdown_connections 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:44.208+0000 7fd43cc00700 1 -- 172.21.15.45:0/1961424238 wait complete. 2024-04-25T18:57:44.212 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T18:57:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:44 smithi079 ceph-mon[33002]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:57:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:44 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:44.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:44 smithi045 ceph-mon[27776]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:57:44.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:44 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:44.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:44 smithi062 ceph-mon[32844]: pgmap v397: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T18:57:44.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:44.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:44 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:45.019 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-04-25T18:57:42.296344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660249,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":854,"num_read_kb":552,"num_write":361,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":791,"ondisk_log_size":791,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392456,"kb_used_data":14184,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549368,"statfs":{"total":1151940427776,"available":1151538552832,"internally_reserved":0,"allocated":14524416,"data_stored":7958847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":39,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005066"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.263126+0000","last_change":"2024-04-25T18:55:23.241030+0000","last_active":"2024-04-25T18:57:16.263126+0000","last_peered":"2024-04-25T18:57:16.263126+0000","last_clean":"2024-04-25T18:57:16.263126+0000","last_became_active":"2024-04-25T18:55:23.240724+0000","last_became_peered":"2024-04-25T18:55:23.240724+0000","last_unstale":"2024-04-25T18:57:16.263126+0000","last_undegraded":"2024-04-25T18:57:16.263126+0000","last_fullsized":"2024-04-25T18:57:16.263126+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:37.995475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.272664+0000","last_change":"2024-04-25T18:55:23.229875+0000","last_active":"2024-04-25T18:57:16.272664+0000","last_peered":"2024-04-25T18:57:16.272664+0000","last_clean":"2024-04-25T18:57:16.272664+0000","last_became_active":"2024-04-25T18:55:23.229640+0000","last_became_peered":"2024-04-25T18:55:23.229640+0000","last_unstale":"2024-04-25T18:57:16.272664+0000","last_undegraded":"2024-04-25T18:57:16.272664+0000","last_fullsized":"2024-04-25T18:57:16.272664+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:10.262376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.273728+0000","last_change":"2024-04-25T18:55:23.229821+0000","last_active":"2024-04-25T18:57:16.273728+0000","last_peered":"2024-04-25T18:57:16.273728+0000","last_clean":"2024-04-25T18:57:16.273728+0000","last_became_active":"2024-04-25T18:55:23.229414+0000","last_became_peered":"2024-04-25T18:55:23.229414+0000","last_unstale":"2024-04-25T18:57:16.273728+0000","last_undegraded":"2024-04-25T18:57:16.273728+0000","last_fullsized":"2024-04-25T18:57:16.273728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:56.063786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.259553+0000","last_change":"2024-04-25T18:56:17.560167+0000","last_active":"2024-04-25T18:57:16.259553+0000","last_peered":"2024-04-25T18:57:16.259553+0000","last_clean":"2024-04-25T18:57:16.259553+0000","last_became_active":"2024-04-25T18:56:17.558466+0000","last_became_peered":"2024-04-25T18:56:17.558466+0000","last_unstale":"2024-04-25T18:57:16.259553+0000","last_undegraded":"2024-04-25T18:57:16.259553+0000","last_fullsized":"2024-04-25T18:57:16.259553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:22.371076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.244648+0000","last_change":"2024-04-25T18:55:23.233512+0000","last_active":"2024-04-25T18:57:16.244648+0000","last_peered":"2024-04-25T18:57:16.244648+0000","last_clean":"2024-04-25T18:57:16.244648+0000","last_became_active":"2024-04-25T18:55:23.233158+0000","last_became_peered":"2024-04-25T18:55:23.233158+0000","last_unstale":"2024-04-25T18:57:16.244648+0000","last_undegraded":"2024-04-25T18:57:16.244648+0000","last_fullsized":"2024-04-25T18:57:16.244648+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:44.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.264597+0000","last_change":"2024-04-25T18:55:23.225921+0000","last_active":"2024-04-25T18:57:16.264597+0000","last_peered":"2024-04-25T18:57:16.264597+0000","last_clean":"2024-04-25T18:57:16.264597+0000","last_became_active":"2024-04-25T18:55:23.225547+0000","last_became_peered":"2024-04-25T18:55:23.225547+0000","last_unstale":"2024-04-25T18:57:16.264597+0000","last_undegraded":"2024-04-25T18:57:16.264597+0000","last_fullsized":"2024-04-25T18:57:16.264597+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:29.828394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.262596+0000","last_change":"2024-04-25T18:55:23.217111+0000","last_active":"2024-04-25T18:57:16.262596+0000","last_peered":"2024-04-25T18:57:16.262596+0000","last_clean":"2024-04-25T18:57:16.262596+0000","last_became_active":"2024-04-25T18:55:23.216731+0000","last_became_peered":"2024-04-25T18:55:23.216731+0000","last_unstale":"2024-04-25T18:57:16.262596+0000","last_undegraded":"2024-04-25T18:57:16.262596+0000","last_fullsized":"2024-04-25T18:57:16.262596+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:17.931928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.259545+0000","last_change":"2024-04-25T18:56:17.555014+0000","last_active":"2024-04-25T18:57:16.259545+0000","last_peered":"2024-04-25T18:57:16.259545+0000","last_clean":"2024-04-25T18:57:16.259545+0000","last_became_active":"2024-04-25T18:56:17.553863+0000","last_became_peered":"2024-04-25T18:56:17.553863+0000","last_unstale":"2024-04-25T18:57:16.259545+0000","last_undegraded":"2024-04-25T18:57:16.259545+0000","last_fullsized":"2024-04-25T18:57:16.259545+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.240754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.250991+0000","last_change":"2024-04-25T18:55:23.218197+0000","last_active":"2024-04-25T18:57:16.250991+0000","last_peered":"2024-04-25T18:57:16.250991+0000","last_clean":"2024-04-25T18:57:16.250991+0000","last_became_active":"2024-04-25T18:55:23.217665+0000","last_became_peered":"2024-04-25T18:55:23.217665+0000","last_unstale":"2024-04-25T18:57:16.250991+0000","last_undegraded":"2024-04-25T18:57:16.250991+0000","last_fullsized":"2024-04-25T18:57:16.250991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:11.829991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.240858+0000","last_change":"2024-04-25T18:55:23.221533+0000","last_active":"2024-04-25T18:57:16.240858+0000","last_peered":"2024-04-25T18:57:16.240858+0000","last_clean":"2024-04-25T18:57:16.240858+0000","last_became_active":"2024-04-25T18:55:23.220711+0000","last_became_peered":"2024-04-25T18:55:23.220711+0000","last_unstale":"2024-04-25T18:57:16.240858+0000","last_undegraded":"2024-04-25T18:57:16.240858+0000","last_fullsized":"2024-04-25T18:57:16.240858+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:45.185166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.250298+0000","last_change":"2024-04-25T18:55:23.221329+0000","last_active":"2024-04-25T18:57:16.250298+0000","last_peered":"2024-04-25T18:57:16.250298+0000","last_clean":"2024-04-25T18:57:16.250298+0000","last_became_active":"2024-04-25T18:55:23.220156+0000","last_became_peered":"2024-04-25T18:55:23.220156+0000","last_unstale":"2024-04-25T18:57:16.250298+0000","last_undegraded":"2024-04-25T18:57:16.250298+0000","last_fullsized":"2024-04-25T18:57:16.250298+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:46.981199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.260612+0000","last_change":"2024-04-25T18:56:17.558018+0000","last_active":"2024-04-25T18:57:16.260612+0000","last_peered":"2024-04-25T18:57:16.260612+0000","last_clean":"2024-04-25T18:57:16.260612+0000","last_became_active":"2024-04-25T18:56:17.556229+0000","last_became_peered":"2024-04-25T18:56:17.556229+0000","last_unstale":"2024-04-25T18:57:16.260612+0000","last_undegraded":"2024-04-25T18:57:16.260612+0000","last_fullsized":"2024-04-25T18:57:16.260612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:50.949895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.248414+0000","last_change":"2024-04-25T18:55:23.227474+0000","last_active":"2024-04-25T18:57:16.248414+0000","last_peered":"2024-04-25T18:57:16.248414+0000","last_clean":"2024-04-25T18:57:16.248414+0000","last_became_active":"2024-04-25T18:55:23.227068+0000","last_became_peered":"2024-04-25T18:55:23.227068+0000","last_unstale":"2024-04-25T18:57:16.248414+0000","last_undegraded":"2024-04-25T18:57:16.248414+0000","last_fullsized":"2024-04-25T18:57:16.248414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:13.884833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.271596+0000","last_change":"2024-04-25T18:55:23.230572+0000","last_active":"2024-04-25T18:57:16.271596+0000","last_peered":"2024-04-25T18:57:16.271596+0000","last_clean":"2024-04-25T18:57:16.271596+0000","last_became_active":"2024-04-25T18:55:23.229756+0000","last_became_peered":"2024-04-25T18:55:23.229756+0000","last_unstale":"2024-04-25T18:57:16.271596+0000","last_undegraded":"2024-04-25T18:57:16.271596+0000","last_fullsized":"2024-04-25T18:57:16.271596+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:15.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.271036+0000","last_change":"2024-04-25T18:55:23.227913+0000","last_active":"2024-04-25T18:57:16.271036+0000","last_peered":"2024-04-25T18:57:16.271036+0000","last_clean":"2024-04-25T18:57:16.271036+0000","last_became_active":"2024-04-25T18:55:23.227598+0000","last_became_peered":"2024-04-25T18:55:23.227598+0000","last_unstale":"2024-04-25T18:57:16.271036+0000","last_undegraded":"2024-04-25T18:57:16.271036+0000","last_fullsized":"2024-04-25T18:57:16.271036+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:33.555741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.273547+0000","last_change":"2024-04-25T18:55:23.230241+0000","last_active":"2024-04-25T18:57:16.273547+0000","last_peered":"2024-04-25T18:57:16.273547+0000","last_clean":"2024-04-25T18:57:16.273547+0000","last_became_active":"2024-04-25T18:55:23.229898+0000","last_became_peered":"2024-04-25T18:55:23.229898+0000","last_unstale":"2024-04-25T18:57:16.273547+0000","last_undegraded":"2024-04-25T18:57:16.273547+0000","last_fullsized":"2024-04-25T18:57:16.273547+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:27.278475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.263221+0000","last_change":"2024-04-25T18:55:23.240978+0000","last_active":"2024-04-25T18:57:16.263221+0000","last_peered":"2024-04-25T18:57:16.263221+0000","last_clean":"2024-04-25T18:57:16.263221+0000","last_became_active":"2024-04-25T18:55:23.240702+0000","last_became_peered":"2024-04-25T18:55:23.240702+0000","last_unstale":"2024-04-25T18:57:16.263221+0000","last_undegraded":"2024-04-25T18:57:16.263221+0000","last_fullsized":"2024-04-25T18:57:16.263221+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:06.053688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.262547+0000","last_change":"2024-04-25T18:55:23.232116+0000","last_active":"2024-04-25T18:57:16.262547+0000","last_peered":"2024-04-25T18:57:16.262547+0000","last_clean":"2024-04-25T18:57:16.262547+0000","last_became_active":"2024-04-25T18:55:23.231586+0000","last_became_peered":"2024-04-25T18:55:23.231586+0000","last_unstale":"2024-04-25T18:57:16.262547+0000","last_undegraded":"2024-04-25T18:57:16.262547+0000","last_fullsized":"2024-04-25T18:57:16.262547+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:32.640823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.272600+0000","last_change":"2024-04-25T18:55:23.224519+0000","last_active":"2024-04-25T18:57:16.272600+0000","last_peered":"2024-04-25T18:57:16.272600+0000","last_clean":"2024-04-25T18:57:16.272600+0000","last_became_active":"2024-04-25T18:55:23.224145+0000","last_became_peered":"2024-04-25T18:55:23.224145+0000","last_unstale":"2024-04-25T18:57:16.272600+0000","last_undegraded":"2024-04-25T18:57:16.272600+0000","last_fullsized":"2024-04-25T18:57:16.272600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:03.340162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.548403+0000","last_change":"2024-04-25T18:55:19.164977+0000","last_active":"2024-04-25T18:56:17.548403+0000","last_peered":"2024-04-25T18:56:17.548403+0000","last_clean":"2024-04-25T18:56:17.548403+0000","last_became_active":"2024-04-25T18:55:19.164316+0000","last_became_peered":"2024-04-25T18:55:19.164316+0000","last_unstale":"2024-04-25T18:56:17.548403+0000","last_undegraded":"2024-04-25T18:56:17.548403+0000","last_fullsized":"2024-04-25T18:56:17.548403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:08.031680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.274666+0000","last_change":"2024-04-25T18:56:17.559746+0000","last_active":"2024-04-25T18:57:16.274666+0000","last_peered":"2024-04-25T18:57:16.274666+0000","last_clean":"2024-04-25T18:57:16.274666+0000","last_became_active":"2024-04-25T18:56:17.558761+0000","last_became_peered":"2024-04-25T18:56:17.558761+0000","last_unstale":"2024-04-25T18:57:16.274666+0000","last_undegraded":"2024-04-25T18:57:16.274666+0000","last_fullsized":"2024-04-25T18:57:16.274666+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:43.518760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.547499+0000","last_change":"2024-04-25T18:55:19.179943+0000","last_active":"2024-04-25T18:56:17.547499+0000","last_peered":"2024-04-25T18:56:17.547499+0000","last_clean":"2024-04-25T18:56:17.547499+0000","last_became_active":"2024-04-25T18:55:19.179519+0000","last_became_peered":"2024-04-25T18:55:19.179519+0000","last_unstale":"2024-04-25T18:56:17.547499+0000","last_undegraded":"2024-04-25T18:56:17.547499+0000","last_fullsized":"2024-04-25T18:56:17.547499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:46.365187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.273246+0000","last_change":"2024-04-25T18:55:23.237965+0000","last_active":"2024-04-25T18:57:16.273246+0000","last_peered":"2024-04-25T18:57:16.273246+0000","last_clean":"2024-04-25T18:57:16.273246+0000","last_became_active":"2024-04-25T18:55:23.237520+0000","last_became_peered":"2024-04-25T18:55:23.237520+0000","last_unstale":"2024-04-25T18:57:16.273246+0000","last_undegraded":"2024-04-25T18:57:16.273246+0000","last_fullsized":"2024-04-25T18:57:16.273246+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:03.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540660+0000","last_change":"2024-04-25T18:55:19.163965+0000","last_active":"2024-04-25T18:56:17.540660+0000","last_peered":"2024-04-25T18:56:17.540660+0000","last_clean":"2024-04-25T18:56:17.540660+0000","last_became_active":"2024-04-25T18:55:19.163390+0000","last_became_peered":"2024-04-25T18:55:19.163390+0000","last_unstale":"2024-04-25T18:56:17.540660+0000","last_undegraded":"2024-04-25T18:56:17.540660+0000","last_fullsized":"2024-04-25T18:56:17.540660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:54.183730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.265552+0000","last_change":"2024-04-25T18:55:23.227124+0000","last_active":"2024-04-25T18:57:16.265552+0000","last_peered":"2024-04-25T18:57:16.265552+0000","last_clean":"2024-04-25T18:57:16.265552+0000","last_became_active":"2024-04-25T18:55:23.226383+0000","last_became_peered":"2024-04-25T18:55:23.226383+0000","last_unstale":"2024-04-25T18:57:16.265552+0000","last_undegraded":"2024-04-25T18:57:16.265552+0000","last_fullsized":"2024-04-25T18:57:16.265552+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:22.011212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.544679+0000","last_change":"2024-04-25T18:56:17.544303+0000","last_active":"2024-04-25T18:56:17.544679+0000","last_peered":"2024-04-25T18:56:17.544679+0000","last_clean":"2024-04-25T18:56:17.544679+0000","last_became_active":"2024-04-25T18:56:17.542002+0000","last_became_peered":"2024-04-25T18:56:17.542002+0000","last_unstale":"2024-04-25T18:56:17.544679+0000","last_undegraded":"2024-04-25T18:56:17.544679+0000","last_fullsized":"2024-04-25T18:56:17.544679+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:23.848398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.242121+0000","last_change":"2024-04-25T18:55:23.217703+0000","last_active":"2024-04-25T18:57:16.242121+0000","last_peered":"2024-04-25T18:57:16.242121+0000","last_clean":"2024-04-25T18:57:16.242121+0000","last_became_active":"2024-04-25T18:55:23.216767+0000","last_became_peered":"2024-04-25T18:55:23.216767+0000","last_unstale":"2024-04-25T18:57:16.242121+0000","last_undegraded":"2024-04-25T18:57:16.242121+0000","last_fullsized":"2024-04-25T18:57:16.242121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:17.779356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":150,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:37.587614+0000","last_change":"2024-04-25T18:55:19.164199+0000","last_active":"2024-04-25T18:57:37.587614+0000","last_peered":"2024-04-25T18:57:37.587614+0000","last_clean":"2024-04-25T18:57:37.587614+0000","last_became_active":"2024-04-25T18:55:19.163849+0000","last_became_peered":"2024-04-25T18:55:19.163849+0000","last_unstale":"2024-04-25T18:57:37.587614+0000","last_undegraded":"2024-04-25T18:57:37.587614+0000","last_fullsized":"2024-04-25T18:57:37.587614+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:25.234229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":117,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.244875+0000","last_change":"2024-04-25T18:55:23.216916+0000","last_active":"2024-04-25T18:57:16.244875+0000","last_peered":"2024-04-25T18:57:16.244875+0000","last_clean":"2024-04-25T18:57:16.244875+0000","last_became_active":"2024-04-25T18:55:23.216249+0000","last_became_peered":"2024-04-25T18:55:23.216249+0000","last_unstale":"2024-04-25T18:57:16.244875+0000","last_undegraded":"2024-04-25T18:57:16.244875+0000","last_fullsized":"2024-04-25T18:57:16.244875+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:21.522318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540439+0000","last_change":"2024-04-25T18:55:19.171004+0000","last_active":"2024-04-25T18:56:17.540439+0000","last_peered":"2024-04-25T18:56:17.540439+0000","last_clean":"2024-04-25T18:56:17.540439+0000","last_became_active":"2024-04-25T18:55:19.170570+0000","last_became_peered":"2024-04-25T18:55:19.170570+0000","last_unstale":"2024-04-25T18:56:17.540439+0000","last_undegraded":"2024-04-25T18:56:17.540439+0000","last_fullsized":"2024-04-25T18:56:17.540439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:15:24.954033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.266668+0000","last_change":"2024-04-25T18:56:17.552615+0000","last_active":"2024-04-25T18:57:16.266668+0000","last_peered":"2024-04-25T18:57:16.266668+0000","last_clean":"2024-04-25T18:57:16.266668+0000","last_became_active":"2024-04-25T18:56:17.552154+0000","last_became_peered":"2024-04-25T18:56:17.552154+0000","last_unstale":"2024-04-25T18:57:16.266668+0000","last_undegraded":"2024-04-25T18:57:16.266668+0000","last_fullsized":"2024-04-25T18:57:16.266668+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:01.199859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.549445+0000","last_change":"2024-04-25T18:55:19.173398+0000","last_active":"2024-04-25T18:56:17.549445+0000","last_peered":"2024-04-25T18:56:17.549445+0000","last_clean":"2024-04-25T18:56:17.549445+0000","last_became_active":"2024-04-25T18:55:19.171477+0000","last_became_peered":"2024-04-25T18:55:19.171477+0000","last_unstale":"2024-04-25T18:56:17.549445+0000","last_undegraded":"2024-04-25T18:56:17.549445+0000","last_fullsized":"2024-04-25T18:56:17.549445+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:35.230596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.274477+0000","last_change":"2024-04-25T18:55:23.235209+0000","last_active":"2024-04-25T18:57:16.274477+0000","last_peered":"2024-04-25T18:57:16.274477+0000","last_clean":"2024-04-25T18:57:16.274477+0000","last_became_active":"2024-04-25T18:55:23.231898+0000","last_became_peered":"2024-04-25T18:55:23.231898+0000","last_unstale":"2024-04-25T18:57:16.274477+0000","last_undegraded":"2024-04-25T18:57:16.274477+0000","last_fullsized":"2024-04-25T18:57:16.274477+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.521282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539932+0000","last_change":"2024-04-25T18:55:19.163780+0000","last_active":"2024-04-25T18:56:17.539932+0000","last_peered":"2024-04-25T18:56:17.539932+0000","last_clean":"2024-04-25T18:56:17.539932+0000","last_became_active":"2024-04-25T18:55:19.163197+0000","last_became_peered":"2024-04-25T18:55:19.163197+0000","last_unstale":"2024-04-25T18:56:17.539932+0000","last_undegraded":"2024-04-25T18:56:17.539932+0000","last_fullsized":"2024-04-25T18:56:17.539932+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:02:26.519160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.264232+0000","last_change":"2024-04-25T18:56:17.556804+0000","last_active":"2024-04-25T18:57:16.264232+0000","last_peered":"2024-04-25T18:57:16.264232+0000","last_clean":"2024-04-25T18:57:16.264232+0000","last_became_active":"2024-04-25T18:56:17.554885+0000","last_became_peered":"2024-04-25T18:56:17.554885+0000","last_unstale":"2024-04-25T18:57:16.264232+0000","last_undegraded":"2024-04-25T18:57:16.264232+0000","last_fullsized":"2024-04-25T18:57:16.264232+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:36.316156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":145,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:39.590311+0000","last_change":"2024-04-25T18:55:19.170565+0000","last_active":"2024-04-25T18:57:39.590311+0000","last_peered":"2024-04-25T18:57:39.590311+0000","last_clean":"2024-04-25T18:57:39.590311+0000","last_became_active":"2024-04-25T18:55:19.170162+0000","last_became_peered":"2024-04-25T18:55:19.170162+0000","last_unstale":"2024-04-25T18:57:39.590311+0000","last_undegraded":"2024-04-25T18:57:39.590311+0000","last_fullsized":"2024-04-25T18:57:39.590311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:36.606661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.263589+0000","last_change":"2024-04-25T18:55:23.227436+0000","last_active":"2024-04-25T18:57:16.263589+0000","last_peered":"2024-04-25T18:57:16.263589+0000","last_clean":"2024-04-25T18:57:16.263589+0000","last_became_active":"2024-04-25T18:55:23.227044+0000","last_became_peered":"2024-04-25T18:55:23.227044+0000","last_unstale":"2024-04-25T18:57:16.263589+0000","last_undegraded":"2024-04-25T18:57:16.263589+0000","last_fullsized":"2024-04-25T18:57:16.263589+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:34.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583525+0000","last_change":"2024-04-25T18:55:19.167514+0000","last_active":"2024-04-25T18:56:17.583525+0000","last_peered":"2024-04-25T18:56:17.583525+0000","last_clean":"2024-04-25T18:56:17.583525+0000","last_became_active":"2024-04-25T18:55:19.167102+0000","last_became_peered":"2024-04-25T18:55:19.167102+0000","last_unstale":"2024-04-25T18:56:17.583525+0000","last_undegraded":"2024-04-25T18:56:17.583525+0000","last_fullsized":"2024-04-25T18:56:17.583525+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:41.472274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.270537+0000","last_change":"2024-04-25T18:55:23.237012+0000","last_active":"2024-04-25T18:57:16.270537+0000","last_peered":"2024-04-25T18:57:16.270537+0000","last_clean":"2024-04-25T18:57:16.270537+0000","last_became_active":"2024-04-25T18:55:23.230263+0000","last_became_peered":"2024-04-25T18:55:23.230263+0000","last_unstale":"2024-04-25T18:57:16.270537+0000","last_undegraded":"2024-04-25T18:57:16.270537+0000","last_fullsized":"2024-04-25T18:57:16.270537+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:30.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540250+0000","last_change":"2024-04-25T18:55:19.179555+0000","last_active":"2024-04-25T18:56:17.540250+0000","last_peered":"2024-04-25T18:56:17.540250+0000","last_clean":"2024-04-25T18:56:17.540250+0000","last_became_active":"2024-04-25T18:55:19.179190+0000","last_became_peered":"2024-04-25T18:55:19.179190+0000","last_unstale":"2024-04-25T18:56:17.540250+0000","last_undegraded":"2024-04-25T18:56:17.540250+0000","last_fullsized":"2024-04-25T18:56:17.540250+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:57.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.254893+0000","last_change":"2024-04-25T18:55:23.219978+0000","last_active":"2024-04-25T18:57:16.254893+0000","last_peered":"2024-04-25T18:57:16.254893+0000","last_clean":"2024-04-25T18:57:16.254893+0000","last_became_active":"2024-04-25T18:55:23.219630+0000","last_became_peered":"2024-04-25T18:55:23.219630+0000","last_unstale":"2024-04-25T18:57:16.254893+0000","last_undegraded":"2024-04-25T18:57:16.254893+0000","last_fullsized":"2024-04-25T18:57:16.254893+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:31.294500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":303,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:40.592308+0000","last_change":"2024-04-25T18:55:19.164042+0000","last_active":"2024-04-25T18:57:40.592308+0000","last_peered":"2024-04-25T18:57:40.592308+0000","last_clean":"2024-04-25T18:57:40.592308+0000","last_became_active":"2024-04-25T18:55:19.163604+0000","last_became_peered":"2024-04-25T18:55:19.163604+0000","last_unstale":"2024-04-25T18:57:40.592308+0000","last_undegraded":"2024-04-25T18:57:40.592308+0000","last_fullsized":"2024-04-25T18:57:40.592308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:55.030715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64973,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":183,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.239140+0000","last_change":"2024-04-25T18:55:23.222702+0000","last_active":"2024-04-25T18:57:16.239140+0000","last_peered":"2024-04-25T18:57:16.239140+0000","last_clean":"2024-04-25T18:57:16.239140+0000","last_became_active":"2024-04-25T18:55:23.221849+0000","last_became_peered":"2024-04-25T18:55:23.221849+0000","last_unstale":"2024-04-25T18:57:16.239140+0000","last_undegraded":"2024-04-25T18:57:16.239140+0000","last_fullsized":"2024-04-25T18:57:16.239140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.957027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.562287+0000","last_change":"2024-04-25T18:56:17.561912+0000","last_active":"2024-04-25T18:56:17.562287+0000","last_peered":"2024-04-25T18:56:17.562287+0000","last_clean":"2024-04-25T18:56:17.562287+0000","last_became_active":"2024-04-25T18:56:17.552120+0000","last_became_peered":"2024-04-25T18:56:17.552120+0000","last_unstale":"2024-04-25T18:56:17.562287+0000","last_undegraded":"2024-04-25T18:56:17.562287+0000","last_fullsized":"2024-04-25T18:56:17.562287+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:30.209802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.276009+0000","last_change":"2024-04-25T18:55:23.233074+0000","last_active":"2024-04-25T18:57:16.276009+0000","last_peered":"2024-04-25T18:57:16.276009+0000","last_clean":"2024-04-25T18:57:16.276009+0000","last_became_active":"2024-04-25T18:55:23.232751+0000","last_became_peered":"2024-04-25T18:55:23.232751+0000","last_unstale":"2024-04-25T18:57:16.276009+0000","last_undegraded":"2024-04-25T18:57:16.276009+0000","last_fullsized":"2024-04-25T18:57:16.276009+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.584929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540218+0000","last_change":"2024-04-25T18:55:19.166093+0000","last_active":"2024-04-25T18:56:17.540218+0000","last_peered":"2024-04-25T18:56:17.540218+0000","last_clean":"2024-04-25T18:56:17.540218+0000","last_became_active":"2024-04-25T18:55:19.165715+0000","last_became_peered":"2024-04-25T18:55:19.165715+0000","last_unstale":"2024-04-25T18:56:17.540218+0000","last_undegraded":"2024-04-25T18:56:17.540218+0000","last_fullsized":"2024-04-25T18:56:17.540218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:32.177943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.262364+0000","last_change":"2024-04-25T18:55:23.224617+0000","last_active":"2024-04-25T18:57:16.262364+0000","last_peered":"2024-04-25T18:57:16.262364+0000","last_clean":"2024-04-25T18:57:16.262364+0000","last_became_active":"2024-04-25T18:55:23.224269+0000","last_became_peered":"2024-04-25T18:55:23.224269+0000","last_unstale":"2024-04-25T18:57:16.262364+0000","last_undegraded":"2024-04-25T18:57:16.262364+0000","last_fullsized":"2024-04-25T18:57:16.262364+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:19.079910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":150,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:39.674410+0000","last_change":"2024-04-25T18:55:19.164971+0000","last_active":"2024-04-25T18:57:39.674410+0000","last_peered":"2024-04-25T18:57:39.674410+0000","last_clean":"2024-04-25T18:57:39.674410+0000","last_became_active":"2024-04-25T18:55:19.164579+0000","last_became_peered":"2024-04-25T18:55:19.164579+0000","last_unstale":"2024-04-25T18:57:39.674410+0000","last_undegraded":"2024-04-25T18:57:39.674410+0000","last_fullsized":"2024-04-25T18:57:39.674410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:37.419379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.236788+0000","last_change":"2024-04-25T18:55:23.222160+0000","last_active":"2024-04-25T18:57:16.236788+0000","last_peered":"2024-04-25T18:57:16.236788+0000","last_clean":"2024-04-25T18:57:16.236788+0000","last_became_active":"2024-04-25T18:55:23.221812+0000","last_became_peered":"2024-04-25T18:55:23.221812+0000","last_unstale":"2024-04-25T18:57:16.236788+0000","last_undegraded":"2024-04-25T18:57:16.236788+0000","last_fullsized":"2024-04-25T18:57:16.236788+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:53:53.388764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540123+0000","last_change":"2024-04-25T18:55:19.179970+0000","last_active":"2024-04-25T18:56:17.540123+0000","last_peered":"2024-04-25T18:56:17.540123+0000","last_clean":"2024-04-25T18:56:17.540123+0000","last_became_active":"2024-04-25T18:55:19.179545+0000","last_became_peered":"2024-04-25T18:55:19.179545+0000","last_unstale":"2024-04-25T18:56:17.540123+0000","last_undegraded":"2024-04-25T18:56:17.540123+0000","last_fullsized":"2024-04-25T18:56:17.540123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:42.841661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.241910+0000","last_change":"2024-04-25T18:55:23.216134+0000","last_active":"2024-04-25T18:57:16.241910+0000","last_peered":"2024-04-25T18:57:16.241910+0000","last_clean":"2024-04-25T18:57:16.241910+0000","last_became_active":"2024-04-25T18:55:23.215535+0000","last_became_peered":"2024-04-25T18:55:23.215535+0000","last_unstale":"2024-04-25T18:57:16.241910+0000","last_undegraded":"2024-04-25T18:57:16.241910+0000","last_fullsized":"2024-04-25T18:57:16.241910+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:08.042095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540722+0000","last_change":"2024-04-25T18:55:19.162437+0000","last_active":"2024-04-25T18:56:17.540722+0000","last_peered":"2024-04-25T18:56:17.540722+0000","last_clean":"2024-04-25T18:56:17.540722+0000","last_became_active":"2024-04-25T18:55:19.162005+0000","last_became_peered":"2024-04-25T18:55:19.162005+0000","last_unstale":"2024-04-25T18:56:17.540722+0000","last_undegraded":"2024-04-25T18:56:17.540722+0000","last_fullsized":"2024-04-25T18:56:17.540722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:50:53.967081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.238362+0000","last_change":"2024-04-25T18:55:23.222345+0000","last_active":"2024-04-25T18:57:16.238362+0000","last_peered":"2024-04-25T18:57:16.238362+0000","last_clean":"2024-04-25T18:57:16.238362+0000","last_became_active":"2024-04-25T18:55:23.221142+0000","last_became_peered":"2024-04-25T18:55:23.221142+0000","last_unstale":"2024-04-25T18:57:16.238362+0000","last_undegraded":"2024-04-25T18:57:16.238362+0000","last_fullsized":"2024-04-25T18:57:16.238362+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:53.823585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550654+0000","last_change":"2024-04-25T18:55:19.181153+0000","last_active":"2024-04-25T18:56:17.550654+0000","last_peered":"2024-04-25T18:56:17.550654+0000","last_clean":"2024-04-25T18:56:17.550654+0000","last_became_active":"2024-04-25T18:55:19.180553+0000","last_became_peered":"2024-04-25T18:55:19.180553+0000","last_unstale":"2024-04-25T18:56:17.550654+0000","last_undegraded":"2024-04-25T18:56:17.550654+0000","last_fullsized":"2024-04-25T18:56:17.550654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:07.829617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.265847+0000","last_change":"2024-04-25T18:55:23.238824+0000","last_active":"2024-04-25T18:57:16.265847+0000","last_peered":"2024-04-25T18:57:16.265847+0000","last_clean":"2024-04-25T18:57:16.265847+0000","last_became_active":"2024-04-25T18:55:23.238228+0000","last_became_peered":"2024-04-25T18:55:23.238228+0000","last_unstale":"2024-04-25T18:57:16.265847+0000","last_undegraded":"2024-04-25T18:57:16.265847+0000","last_fullsized":"2024-04-25T18:57:16.265847+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:16.279254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583366+0000","last_change":"2024-04-25T18:55:19.171186+0000","last_active":"2024-04-25T18:56:17.583366+0000","last_peered":"2024-04-25T18:56:17.583366+0000","last_clean":"2024-04-25T18:56:17.583366+0000","last_became_active":"2024-04-25T18:55:19.170800+0000","last_became_peered":"2024-04-25T18:55:19.170800+0000","last_unstale":"2024-04-25T18:56:17.583366+0000","last_undegraded":"2024-04-25T18:56:17.583366+0000","last_fullsized":"2024-04-25T18:56:17.583366+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.599119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.261311+0000","last_change":"2024-04-25T18:55:23.236951+0000","last_active":"2024-04-25T18:57:16.261311+0000","last_peered":"2024-04-25T18:57:16.261311+0000","last_clean":"2024-04-25T18:57:16.261311+0000","last_became_active":"2024-04-25T18:55:23.236596+0000","last_became_peered":"2024-04-25T18:55:23.236596+0000","last_unstale":"2024-04-25T18:57:16.261311+0000","last_undegraded":"2024-04-25T18:57:16.261311+0000","last_fullsized":"2024-04-25T18:57:16.261311+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:14.634653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.240085+0000","last_change":"2024-04-25T18:55:23.211984+0000","last_active":"2024-04-25T18:57:16.240085+0000","last_peered":"2024-04-25T18:57:16.240085+0000","last_clean":"2024-04-25T18:57:16.240085+0000","last_became_active":"2024-04-25T18:55:23.211591+0000","last_became_peered":"2024-04-25T18:55:23.211591+0000","last_unstale":"2024-04-25T18:57:16.240085+0000","last_undegraded":"2024-04-25T18:57:16.240085+0000","last_fullsized":"2024-04-25T18:57:16.240085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:09.024891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":145,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:37.667914+0000","last_change":"2024-04-25T18:55:19.180378+0000","last_active":"2024-04-25T18:57:37.667914+0000","last_peered":"2024-04-25T18:57:37.667914+0000","last_clean":"2024-04-25T18:57:37.667914+0000","last_became_active":"2024-04-25T18:55:19.179905+0000","last_became_peered":"2024-04-25T18:55:19.179905+0000","last_unstale":"2024-04-25T18:57:37.667914+0000","last_undegraded":"2024-04-25T18:57:37.667914+0000","last_fullsized":"2024-04-25T18:57:37.667914+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:36.434477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":113,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.275037+0000","last_change":"2024-04-25T18:55:23.231489+0000","last_active":"2024-04-25T18:57:16.275037+0000","last_peered":"2024-04-25T18:57:16.275037+0000","last_clean":"2024-04-25T18:57:16.275037+0000","last_became_active":"2024-04-25T18:55:23.231049+0000","last_became_peered":"2024-04-25T18:55:23.231049+0000","last_unstale":"2024-04-25T18:57:16.275037+0000","last_undegraded":"2024-04-25T18:57:16.275037+0000","last_fullsized":"2024-04-25T18:57:16.275037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:25.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539314+0000","last_change":"2024-04-25T18:55:19.171694+0000","last_active":"2024-04-25T18:56:17.539314+0000","last_peered":"2024-04-25T18:56:17.539314+0000","last_clean":"2024-04-25T18:56:17.539314+0000","last_became_active":"2024-04-25T18:55:19.171296+0000","last_became_peered":"2024-04-25T18:55:19.171296+0000","last_unstale":"2024-04-25T18:56:17.539314+0000","last_undegraded":"2024-04-25T18:56:17.539314+0000","last_fullsized":"2024-04-25T18:56:17.539314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:33.216980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583253+0000","last_change":"2024-04-25T18:53:49.127506+0000","last_active":"2024-04-25T18:56:17.583253+0000","last_peered":"2024-04-25T18:56:17.583253+0000","last_clean":"2024-04-25T18:56:17.583253+0000","last_became_active":"2024-04-25T18:53:49.126974+0000","last_became_peered":"2024-04-25T18:53:49.126974+0000","last_unstale":"2024-04-25T18:56:17.583253+0000","last_undegraded":"2024-04-25T18:56:17.583253+0000","last_fullsized":"2024-04-25T18:56:17.583253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_clean_scrub_stamp":"2024-04-25T18:50:00.765468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.488525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.245945+0000","last_change":"2024-04-25T18:55:23.211697+0000","last_active":"2024-04-25T18:57:16.245945+0000","last_peered":"2024-04-25T18:57:16.245945+0000","last_clean":"2024-04-25T18:57:16.245945+0000","last_became_active":"2024-04-25T18:55:23.211160+0000","last_became_peered":"2024-04-25T18:55:23.211160+0000","last_unstale":"2024-04-25T18:57:16.245945+0000","last_undegraded":"2024-04-25T18:57:16.245945+0000","last_fullsized":"2024-04-25T18:57:16.245945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:03.452578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583335+0000","last_change":"2024-04-25T18:55:19.165955+0000","last_active":"2024-04-25T18:56:17.583335+0000","last_peered":"2024-04-25T18:56:17.583335+0000","last_clean":"2024-04-25T18:56:17.583335+0000","last_became_active":"2024-04-25T18:55:19.165594+0000","last_became_peered":"2024-04-25T18:55:19.165594+0000","last_unstale":"2024-04-25T18:56:17.583335+0000","last_undegraded":"2024-04-25T18:56:17.583335+0000","last_fullsized":"2024-04-25T18:56:17.583335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:11.692374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540598+0000","last_change":"2024-04-25T18:55:19.182698+0000","last_active":"2024-04-25T18:56:17.540598+0000","last_peered":"2024-04-25T18:56:17.540598+0000","last_clean":"2024-04-25T18:56:17.540598+0000","last_became_active":"2024-04-25T18:55:19.182393+0000","last_became_peered":"2024-04-25T18:55:19.182393+0000","last_unstale":"2024-04-25T18:56:17.540598+0000","last_undegraded":"2024-04-25T18:56:17.540598+0000","last_fullsized":"2024-04-25T18:56:17.540598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.513613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.265788+0000","last_change":"2024-04-25T18:55:23.223240+0000","last_active":"2024-04-25T18:57:16.265788+0000","last_peered":"2024-04-25T18:57:16.265788+0000","last_clean":"2024-04-25T18:57:16.265788+0000","last_became_active":"2024-04-25T18:55:23.218253+0000","last_became_peered":"2024-04-25T18:55:23.218253+0000","last_unstale":"2024-04-25T18:57:16.265788+0000","last_undegraded":"2024-04-25T18:57:16.265788+0000","last_fullsized":"2024-04-25T18:57:16.265788+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:40.435633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'13","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.277105+0000","last_change":"2024-04-25T18:55:19.168535+0000","last_active":"2024-04-25T18:57:16.277105+0000","last_peered":"2024-04-25T18:57:16.277105+0000","last_clean":"2024-04-25T18:57:16.277105+0000","last_became_active":"2024-04-25T18:55:19.166735+0000","last_became_peered":"2024-04-25T18:55:19.166735+0000","last_unstale":"2024-04-25T18:57:16.277105+0000","last_undegraded":"2024-04-25T18:57:16.277105+0000","last_fullsized":"2024-04-25T18:57:16.277105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:50:14.008290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.275975+0000","last_change":"2024-04-25T18:55:23.233989+0000","last_active":"2024-04-25T18:57:16.275975+0000","last_peered":"2024-04-25T18:57:16.275975+0000","last_clean":"2024-04-25T18:57:16.275975+0000","last_became_active":"2024-04-25T18:55:23.226867+0000","last_became_peered":"2024-04-25T18:55:23.226867+0000","last_unstale":"2024-04-25T18:57:16.275975+0000","last_undegraded":"2024-04-25T18:57:16.275975+0000","last_fullsized":"2024-04-25T18:57:16.275975+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:02.314368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539538+0000","last_change":"2024-04-25T18:55:19.870412+0000","last_active":"2024-04-25T18:56:17.539538+0000","last_peered":"2024-04-25T18:56:17.539538+0000","last_clean":"2024-04-25T18:56:17.539538+0000","last_became_active":"2024-04-25T18:55:19.869970+0000","last_became_peered":"2024-04-25T18:55:19.869970+0000","last_unstale":"2024-04-25T18:56:17.539538+0000","last_undegraded":"2024-04-25T18:56:17.539538+0000","last_fullsized":"2024-04-25T18:56:17.539538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:22.236417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.277236+0000","last_change":"2024-04-25T18:55:23.239172+0000","last_active":"2024-04-25T18:57:16.277236+0000","last_peered":"2024-04-25T18:57:16.277236+0000","last_clean":"2024-04-25T18:57:16.277236+0000","last_became_active":"2024-04-25T18:55:23.238732+0000","last_became_peered":"2024-04-25T18:55:23.238732+0000","last_unstale":"2024-04-25T18:57:16.277236+0000","last_undegraded":"2024-04-25T18:57:16.277236+0000","last_fullsized":"2024-04-25T18:57:16.277236+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:27.459659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540801+0000","last_change":"2024-04-25T18:55:19.179687+0000","last_active":"2024-04-25T18:56:17.540801+0000","last_peered":"2024-04-25T18:56:17.540801+0000","last_clean":"2024-04-25T18:56:17.540801+0000","last_became_active":"2024-04-25T18:55:19.179225+0000","last_became_peered":"2024-04-25T18:55:19.179225+0000","last_unstale":"2024-04-25T18:56:17.540801+0000","last_undegraded":"2024-04-25T18:56:17.540801+0000","last_fullsized":"2024-04-25T18:56:17.540801+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:16:16.266021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.230530+0000","last_change":"2024-04-25T18:55:23.221379+0000","last_active":"2024-04-25T18:57:16.230530+0000","last_peered":"2024-04-25T18:57:16.230530+0000","last_clean":"2024-04-25T18:57:16.230530+0000","last_became_active":"2024-04-25T18:55:23.221037+0000","last_became_peered":"2024-04-25T18:55:23.221037+0000","last_unstale":"2024-04-25T18:57:16.230530+0000","last_undegraded":"2024-04-25T18:57:16.230530+0000","last_fullsized":"2024-04-25T18:57:16.230530+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:27.581084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.546313+0000","last_change":"2024-04-25T18:55:19.171298+0000","last_active":"2024-04-25T18:56:17.546313+0000","last_peered":"2024-04-25T18:56:17.546313+0000","last_clean":"2024-04-25T18:56:17.546313+0000","last_became_active":"2024-04-25T18:55:19.170738+0000","last_became_peered":"2024-04-25T18:55:19.170738+0000","last_unstale":"2024-04-25T18:56:17.546313+0000","last_undegraded":"2024-04-25T18:56:17.546313+0000","last_fullsized":"2024-04-25T18:56:17.546313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.236331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.274256+0000","last_change":"2024-04-25T18:55:23.226577+0000","last_active":"2024-04-25T18:57:16.274256+0000","last_peered":"2024-04-25T18:57:16.274256+0000","last_clean":"2024-04-25T18:57:16.274256+0000","last_became_active":"2024-04-25T18:55:23.226198+0000","last_became_peered":"2024-04-25T18:55:23.226198+0000","last_unstale":"2024-04-25T18:57:16.274256+0000","last_undegraded":"2024-04-25T18:57:16.274256+0000","last_fullsized":"2024-04-25T18:57:16.274256+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:34:59.124523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536406+0000","last_change":"2024-04-25T18:55:19.171917+0000","last_active":"2024-04-25T18:56:17.536406+0000","last_peered":"2024-04-25T18:56:17.536406+0000","last_clean":"2024-04-25T18:56:17.536406+0000","last_became_active":"2024-04-25T18:55:19.171506+0000","last_became_peered":"2024-04-25T18:55:19.171506+0000","last_unstale":"2024-04-25T18:56:17.536406+0000","last_undegraded":"2024-04-25T18:56:17.536406+0000","last_fullsized":"2024-04-25T18:56:17.536406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:53.446504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.275128+0000","last_change":"2024-04-25T18:55:23.237828+0000","last_active":"2024-04-25T18:57:16.275128+0000","last_peered":"2024-04-25T18:57:16.275128+0000","last_clean":"2024-04-25T18:57:16.275128+0000","last_became_active":"2024-04-25T18:55:23.237310+0000","last_became_peered":"2024-04-25T18:55:23.237310+0000","last_unstale":"2024-04-25T18:57:16.275128+0000","last_undegraded":"2024-04-25T18:57:16.275128+0000","last_fullsized":"2024-04-25T18:57:16.275128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:10.987032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540312+0000","last_change":"2024-04-25T18:55:19.163824+0000","last_active":"2024-04-25T18:56:17.540312+0000","last_peered":"2024-04-25T18:56:17.540312+0000","last_clean":"2024-04-25T18:56:17.540312+0000","last_became_active":"2024-04-25T18:55:19.163465+0000","last_became_peered":"2024-04-25T18:55:19.163465+0000","last_unstale":"2024-04-25T18:56:17.540312+0000","last_undegraded":"2024-04-25T18:56:17.540312+0000","last_fullsized":"2024-04-25T18:56:17.540312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:38.347100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.262441+0000","last_change":"2024-04-25T18:55:23.224648+0000","last_active":"2024-04-25T18:57:16.262441+0000","last_peered":"2024-04-25T18:57:16.262441+0000","last_clean":"2024-04-25T18:57:16.262441+0000","last_became_active":"2024-04-25T18:55:23.224323+0000","last_became_peered":"2024-04-25T18:55:23.224323+0000","last_unstale":"2024-04-25T18:57:16.262441+0000","last_undegraded":"2024-04-25T18:57:16.262441+0000","last_fullsized":"2024-04-25T18:57:16.262441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.117792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536728+0000","last_change":"2024-04-25T18:55:19.870213+0000","last_active":"2024-04-25T18:56:17.536728+0000","last_peered":"2024-04-25T18:56:17.536728+0000","last_clean":"2024-04-25T18:56:17.536728+0000","last_became_active":"2024-04-25T18:55:19.869756+0000","last_became_peered":"2024-04-25T18:55:19.869756+0000","last_unstale":"2024-04-25T18:56:17.536728+0000","last_undegraded":"2024-04-25T18:56:17.536728+0000","last_fullsized":"2024-04-25T18:56:17.536728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:44.695876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.250586+0000","last_change":"2024-04-25T18:55:23.217094+0000","last_active":"2024-04-25T18:57:16.250586+0000","last_peered":"2024-04-25T18:57:16.250586+0000","last_clean":"2024-04-25T18:57:16.250586+0000","last_became_active":"2024-04-25T18:55:23.216696+0000","last_became_peered":"2024-04-25T18:55:23.216696+0000","last_unstale":"2024-04-25T18:57:16.250586+0000","last_undegraded":"2024-04-25T18:57:16.250586+0000","last_fullsized":"2024-04-25T18:57:16.250586+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:07.049349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540128+0000","last_change":"2024-04-25T18:55:19.166377+0000","last_active":"2024-04-25T18:56:17.540128+0000","last_peered":"2024-04-25T18:56:17.540128+0000","last_clean":"2024-04-25T18:56:17.540128+0000","last_became_active":"2024-04-25T18:55:19.166036+0000","last_became_peered":"2024-04-25T18:55:19.166036+0000","last_unstale":"2024-04-25T18:56:17.540128+0000","last_undegraded":"2024-04-25T18:56:17.540128+0000","last_fullsized":"2024-04-25T18:56:17.540128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:04.761919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.273051+0000","last_change":"2024-04-25T18:55:23.236893+0000","last_active":"2024-04-25T18:57:16.273051+0000","last_peered":"2024-04-25T18:57:16.273051+0000","last_clean":"2024-04-25T18:57:16.273051+0000","last_became_active":"2024-04-25T18:55:23.236488+0000","last_became_peered":"2024-04-25T18:55:23.236488+0000","last_unstale":"2024-04-25T18:57:16.273051+0000","last_undegraded":"2024-04-25T18:57:16.273051+0000","last_fullsized":"2024-04-25T18:57:16.273051+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:44.605041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.551204+0000","last_change":"2024-04-25T18:55:19.175076+0000","last_active":"2024-04-25T18:56:17.551204+0000","last_peered":"2024-04-25T18:56:17.551204+0000","last_clean":"2024-04-25T18:56:17.551204+0000","last_became_active":"2024-04-25T18:55:19.174547+0000","last_became_peered":"2024-04-25T18:55:19.174547+0000","last_unstale":"2024-04-25T18:56:17.551204+0000","last_undegraded":"2024-04-25T18:56:17.551204+0000","last_fullsized":"2024-04-25T18:56:17.551204+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:43.791068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.247558+0000","last_change":"2024-04-25T18:55:23.219665+0000","last_active":"2024-04-25T18:57:16.247558+0000","last_peered":"2024-04-25T18:57:16.247558+0000","last_clean":"2024-04-25T18:57:16.247558+0000","last_became_active":"2024-04-25T18:55:23.219264+0000","last_became_peered":"2024-04-25T18:55:23.219264+0000","last_unstale":"2024-04-25T18:57:16.247558+0000","last_undegraded":"2024-04-25T18:57:16.247558+0000","last_fullsized":"2024-04-25T18:57:16.247558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:19.008120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.239322+0000","last_change":"2024-04-25T18:55:23.220290+0000","last_active":"2024-04-25T18:57:16.239322+0000","last_peered":"2024-04-25T18:57:16.239322+0000","last_clean":"2024-04-25T18:57:16.239322+0000","last_became_active":"2024-04-25T18:55:23.219131+0000","last_became_peered":"2024-04-25T18:55:23.219131+0000","last_unstale":"2024-04-25T18:57:16.239322+0000","last_undegraded":"2024-04-25T18:57:16.239322+0000","last_fullsized":"2024-04-25T18:57:16.239322+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:25:25.678525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.242421+0000","last_change":"2024-04-25T18:55:23.216415+0000","last_active":"2024-04-25T18:57:16.242421+0000","last_peered":"2024-04-25T18:57:16.242421+0000","last_clean":"2024-04-25T18:57:16.242421+0000","last_became_active":"2024-04-25T18:55:23.216066+0000","last_became_peered":"2024-04-25T18:55:23.216066+0000","last_unstale":"2024-04-25T18:57:16.242421+0000","last_undegraded":"2024-04-25T18:57:16.242421+0000","last_fullsized":"2024-04-25T18:57:16.242421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:54.633216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.275888+0000","last_change":"2024-04-25T18:55:23.234034+0000","last_active":"2024-04-25T18:57:16.275888+0000","last_peered":"2024-04-25T18:57:16.275888+0000","last_clean":"2024-04-25T18:57:16.275888+0000","last_became_active":"2024-04-25T18:55:23.233639+0000","last_became_peered":"2024-04-25T18:55:23.233639+0000","last_unstale":"2024-04-25T18:57:16.275888+0000","last_undegraded":"2024-04-25T18:57:16.275888+0000","last_fullsized":"2024-04-25T18:57:16.275888+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:57.117958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.272622+0000","last_change":"2024-04-25T18:56:17.584511+0000","last_active":"2024-04-25T18:57:16.272622+0000","last_peered":"2024-04-25T18:57:16.272622+0000","last_clean":"2024-04-25T18:57:16.272622+0000","last_became_active":"2024-04-25T18:56:17.583959+0000","last_became_peered":"2024-04-25T18:56:17.583959+0000","last_unstale":"2024-04-25T18:57:16.272622+0000","last_undegraded":"2024-04-25T18:57:16.272622+0000","last_fullsized":"2024-04-25T18:57:16.272622+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.622823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.268811+0000","last_change":"2024-04-25T18:55:23.238068+0000","last_active":"2024-04-25T18:57:16.268811+0000","last_peered":"2024-04-25T18:57:16.268811+0000","last_clean":"2024-04-25T18:57:16.268811+0000","last_became_active":"2024-04-25T18:55:23.237773+0000","last_became_peered":"2024-04-25T18:55:23.237773+0000","last_unstale":"2024-04-25T18:57:16.268811+0000","last_undegraded":"2024-04-25T18:57:16.268811+0000","last_fullsized":"2024-04-25T18:57:16.268811+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.688576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.236776+0000","last_change":"2024-04-25T18:55:23.215463+0000","last_active":"2024-04-25T18:57:16.236776+0000","last_peered":"2024-04-25T18:57:16.236776+0000","last_clean":"2024-04-25T18:57:16.236776+0000","last_became_active":"2024-04-25T18:55:23.215068+0000","last_became_peered":"2024-04-25T18:55:23.215068+0000","last_unstale":"2024-04-25T18:57:16.236776+0000","last_undegraded":"2024-04-25T18:57:16.236776+0000","last_fullsized":"2024-04-25T18:57:16.236776+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.450828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.244816+0000","last_change":"2024-04-25T18:55:23.214362+0000","last_active":"2024-04-25T18:57:16.244816+0000","last_peered":"2024-04-25T18:57:16.244816+0000","last_clean":"2024-04-25T18:57:16.244816+0000","last_became_active":"2024-04-25T18:55:23.214035+0000","last_became_peered":"2024-04-25T18:55:23.214035+0000","last_unstale":"2024-04-25T18:57:16.244816+0000","last_undegraded":"2024-04-25T18:57:16.244816+0000","last_fullsized":"2024-04-25T18:57:16.244816+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:33.386859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.266075+0000","last_change":"2024-04-25T18:55:23.235709+0000","last_active":"2024-04-25T18:57:16.266075+0000","last_peered":"2024-04-25T18:57:16.266075+0000","last_clean":"2024-04-25T18:57:16.266075+0000","last_became_active":"2024-04-25T18:55:23.226089+0000","last_became_peered":"2024-04-25T18:55:23.226089+0000","last_unstale":"2024-04-25T18:57:16.266075+0000","last_undegraded":"2024-04-25T18:57:16.266075+0000","last_fullsized":"2024-04-25T18:57:16.266075+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:57.161365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.267325+0000","last_change":"2024-04-25T18:56:17.561546+0000","last_active":"2024-04-25T18:57:16.267325+0000","last_peered":"2024-04-25T18:57:16.267325+0000","last_clean":"2024-04-25T18:57:16.267325+0000","last_became_active":"2024-04-25T18:56:17.559767+0000","last_became_peered":"2024-04-25T18:56:17.559767+0000","last_unstale":"2024-04-25T18:57:16.267325+0000","last_undegraded":"2024-04-25T18:57:16.267325+0000","last_fullsized":"2024-04-25T18:57:16.267325+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:59.512008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.275457+0000","last_change":"2024-04-25T18:55:23.227620+0000","last_active":"2024-04-25T18:57:16.275457+0000","last_peered":"2024-04-25T18:57:16.275457+0000","last_clean":"2024-04-25T18:57:16.275457+0000","last_became_active":"2024-04-25T18:55:23.227236+0000","last_became_peered":"2024-04-25T18:55:23.227236+0000","last_unstale":"2024-04-25T18:57:16.275457+0000","last_undegraded":"2024-04-25T18:57:16.275457+0000","last_fullsized":"2024-04-25T18:57:16.275457+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:32.707619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.276796+0000","last_change":"2024-04-25T18:55:23.238393+0000","last_active":"2024-04-25T18:57:16.276796+0000","last_peered":"2024-04-25T18:57:16.276796+0000","last_clean":"2024-04-25T18:57:16.276796+0000","last_became_active":"2024-04-25T18:55:23.237726+0000","last_became_peered":"2024-04-25T18:55:23.237726+0000","last_unstale":"2024-04-25T18:57:16.276796+0000","last_undegraded":"2024-04-25T18:57:16.276796+0000","last_fullsized":"2024-04-25T18:57:16.276796+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:48.543014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.268811+0000","last_change":"2024-04-25T18:55:23.231374+0000","last_active":"2024-04-25T18:57:16.268811+0000","last_peered":"2024-04-25T18:57:16.268811+0000","last_clean":"2024-04-25T18:57:16.268811+0000","last_became_active":"2024-04-25T18:55:23.230976+0000","last_became_peered":"2024-04-25T18:55:23.230976+0000","last_unstale":"2024-04-25T18:57:16.268811+0000","last_undegraded":"2024-04-25T18:57:16.268811+0000","last_fullsized":"2024-04-25T18:57:16.268811+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:06.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.239368+0000","last_change":"2024-04-25T18:55:23.217332+0000","last_active":"2024-04-25T18:57:16.239368+0000","last_peered":"2024-04-25T18:57:16.239368+0000","last_clean":"2024-04-25T18:57:16.239368+0000","last_became_active":"2024-04-25T18:55:23.216968+0000","last_became_peered":"2024-04-25T18:55:23.216968+0000","last_unstale":"2024-04-25T18:57:16.239368+0000","last_undegraded":"2024-04-25T18:57:16.239368+0000","last_fullsized":"2024-04-25T18:57:16.239368+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:31.000083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69647,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":718,"num_read_kb":433,"num_write":178,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":208941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252909,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":498791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514548,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1728512,"data_stored":1155743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710774,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":498993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972414,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":498913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234052,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1638400,"data_stored":1091516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430279,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":498382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":563148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":499653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":564996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117080,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":498693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855443,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":499351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691918,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1090668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:57:45.025 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:57:45.524 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:46.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:46 smithi045 ceph-mon[27776]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:57:46.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:46 smithi045 ceph-mon[27776]: pgmap v398: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:46.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:46 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:46.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:46 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:46.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:46 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:46 smithi079 ceph-mon[33002]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:57:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:46 smithi079 ceph-mon[33002]: pgmap v398: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:46 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:46 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:46 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:46.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:46 smithi062 ceph-mon[32844]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:57:46.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:46 smithi062 ceph-mon[32844]: pgmap v398: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:46.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:46 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:57:46.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:46 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:57:46.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:46 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:57:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.768+0000 7fa36b400700 1 -- 172.21.15.45:0/1096892488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364105870 msgr2=0x7fa3641098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.768+0000 7fa36b400700 1 --2- 172.21.15.45:0/1096892488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364105870 0x7fa3641098c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa34c0099c0 tx=0x7fa34c009cd0 comp rx=0 tx=0).stop 2024-04-25T18:57:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.768+0000 7fa36b400700 1 -- 172.21.15.45:0/1096892488 shutdown_connections 2024-04-25T18:57:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.768+0000 7fa36b400700 1 --2- 172.21.15.45:0/1096892488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa364104ec0 0x7fa3641052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.768+0000 7fa36b400700 1 --2- 172.21.15.45:0/1096892488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364109fa0 0x7fa364111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.768+0000 7fa36b400700 1 --2- 172.21.15.45:0/1096892488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364105870 0x7fa3641098c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.768+0000 7fa36b400700 1 -- 172.21.15.45:0/1096892488 >> 172.21.15.45:0/1096892488 conn(0x7fa364100750 msgr2=0x7fa364102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.768+0000 7fa36b400700 1 -- 172.21.15.45:0/1096892488 shutdown_connections 2024-04-25T18:57:47.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.768+0000 7fa36b400700 1 -- 172.21.15.45:0/1096892488 wait complete. 2024-04-25T18:57:47.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.769+0000 7fa36b400700 1 Processor -- start 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.769+0000 7fa36b400700 1 -- start start 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa36b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364104ec0 0x7fa36419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa36b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364105870 0x7fa36419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa36b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa364109fa0 0x7fa36419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa36b400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa364114e30 con 0x7fa364105870 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa36b400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa364114cb0 con 0x7fa364109fa0 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa36b400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa364114fb0 con 0x7fa364104ec0 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa368a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364104ec0 0x7fa36419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa35be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364105870 0x7fa36419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa369400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa364109fa0 0x7fa36419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa368a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364104ec0 0x7fa36419b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:51800/0 (socket says 172.21.15.45:51800) 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.770+0000 7fa368a00700 1 -- 172.21.15.45:0/494086838 learned_addr learned my addr 172.21.15.45:0/494086838 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.771+0000 7fa368a00700 1 -- 172.21.15.45:0/494086838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa364109fa0 msgr2=0x7fa36419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.771+0000 7fa368a00700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa364109fa0 0x7fa36419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:47.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.771+0000 7fa368a00700 1 -- 172.21.15.45:0/494086838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364105870 msgr2=0x7fa36419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.771+0000 7fa368a00700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364105870 0x7fa36419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.771+0000 7fa368a00700 1 -- 172.21.15.45:0/494086838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3641a0300 con 0x7fa364104ec0 2024-04-25T18:57:47.774 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.771+0000 7fa368a00700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364104ec0 0x7fa36419b340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa35400e9a0 tx=0x7fa35400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:47.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.771+0000 7fa362000700 1 -- 172.21.15.45:0/494086838 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa35400ca10 con 0x7fa364104ec0 2024-04-25T18:57:47.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.771+0000 7fa362000700 1 -- 172.21.15.45:0/494086838 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa35400cbb0 con 0x7fa364104ec0 2024-04-25T18:57:47.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.771+0000 7fa36b400700 1 -- 172.21.15.45:0/494086838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3641a05f0 con 0x7fa364104ec0 2024-04-25T18:57:47.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.771+0000 7fa362000700 1 -- 172.21.15.45:0/494086838 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa354018680 con 0x7fa364104ec0 2024-04-25T18:57:47.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.772+0000 7fa36b400700 1 -- 172.21.15.45:0/494086838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3641a7e70 con 0x7fa364104ec0 2024-04-25T18:57:47.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.773+0000 7fa36b400700 1 -- 172.21.15.45:0/494086838 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa364069730 con 0x7fa364104ec0 2024-04-25T18:57:47.776 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.775+0000 7fa362000700 1 -- 172.21.15.45:0/494086838 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa354010430 con 0x7fa364104ec0 2024-04-25T18:57:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.776+0000 7fa362000700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa3440a3c40 0x7fa3440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.776+0000 7fa362000700 1 -- 172.21.15.45:0/494086838 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa3540d08d0 con 0x7fa364104ec0 2024-04-25T18:57:47.778 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.776+0000 7fa35be00700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa3440a3c40 0x7fa3440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:47.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.777+0000 7fa35be00700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa3440a3c40 0x7fa3440a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa34c000c00 tx=0x7fa34c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:47.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:47.778+0000 7fa362000700 1 -- 172.21.15.45:0/494086838 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa35409d230 con 0x7fa364104ec0 2024-04-25T18:57:48.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.006+0000 7fa36b400700 1 -- 172.21.15.45:0/494086838 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa364062e80 con 0x7fa3440a3c40 2024-04-25T18:57:48.017 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.015+0000 7fa362000700 1 -- 172.21.15.45:0/494086838 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258850 (secure 0 0 0) 0x7fa364062e80 con 0x7fa3440a3c40 2024-04-25T18:57:48.024 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.022+0000 7fa36b400700 1 -- 172.21.15.45:0/494086838 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa3440a3c40 msgr2=0x7fa3440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.022+0000 7fa36b400700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa3440a3c40 0x7fa3440a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa34c000c00 tx=0x7fa34c005fb0 comp rx=0 tx=0).stop 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.022+0000 7fa36b400700 1 -- 172.21.15.45:0/494086838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364104ec0 msgr2=0x7fa36419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.022+0000 7fa36b400700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364104ec0 0x7fa36419b340 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa35400e9a0 tx=0x7fa35400ecb0 comp rx=0 tx=0).stop 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.022+0000 7fa36b400700 1 -- 172.21.15.45:0/494086838 shutdown_connections 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.022+0000 7fa36b400700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa3440a3c40 0x7fa3440a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.022+0000 7fa36b400700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa364109fa0 0x7fa36419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.022+0000 7fa36b400700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa364104ec0 0x7fa36419b340 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.022+0000 7fa36b400700 1 --2- 172.21.15.45:0/494086838 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa364105870 0x7fa36419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.022+0000 7fa36b400700 1 -- 172.21.15.45:0/494086838 >> 172.21.15.45:0/494086838 conn(0x7fa364100750 msgr2=0x7fa364102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.023+0000 7fa36b400700 1 -- 172.21.15.45:0/494086838 shutdown_connections 2024-04-25T18:57:48.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:48.023+0000 7fa36b400700 1 -- 172.21.15.45:0/494086838 wait complete. 2024-04-25T18:57:48.027 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T18:57:48.400 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:48 smithi045 ceph-mon[27776]: pgmap v399: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:57:48.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:48 smithi079 ceph-mon[33002]: pgmap v399: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:57:48.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:48 smithi062 ceph-mon[32844]: pgmap v399: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:57:48.813 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-04-25T18:57:46.297909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660249,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":869,"num_read_kb":557,"num_write":361,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":791,"ondisk_log_size":791,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":392456,"kb_used_data":14184,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124549368,"statfs":{"total":1151940427776,"available":1151538552832,"internally_reserved":0,"allocated":14524416,"data_stored":7958847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004740"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.263126+0000","last_change":"2024-04-25T18:55:23.241030+0000","last_active":"2024-04-25T18:57:16.263126+0000","last_peered":"2024-04-25T18:57:16.263126+0000","last_clean":"2024-04-25T18:57:16.263126+0000","last_became_active":"2024-04-25T18:55:23.240724+0000","last_became_peered":"2024-04-25T18:55:23.240724+0000","last_unstale":"2024-04-25T18:57:16.263126+0000","last_undegraded":"2024-04-25T18:57:16.263126+0000","last_fullsized":"2024-04-25T18:57:16.263126+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:37.995475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.272664+0000","last_change":"2024-04-25T18:55:23.229875+0000","last_active":"2024-04-25T18:57:16.272664+0000","last_peered":"2024-04-25T18:57:16.272664+0000","last_clean":"2024-04-25T18:57:16.272664+0000","last_became_active":"2024-04-25T18:55:23.229640+0000","last_became_peered":"2024-04-25T18:55:23.229640+0000","last_unstale":"2024-04-25T18:57:16.272664+0000","last_undegraded":"2024-04-25T18:57:16.272664+0000","last_fullsized":"2024-04-25T18:57:16.272664+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:10.262376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.273728+0000","last_change":"2024-04-25T18:55:23.229821+0000","last_active":"2024-04-25T18:57:16.273728+0000","last_peered":"2024-04-25T18:57:16.273728+0000","last_clean":"2024-04-25T18:57:16.273728+0000","last_became_active":"2024-04-25T18:55:23.229414+0000","last_became_peered":"2024-04-25T18:55:23.229414+0000","last_unstale":"2024-04-25T18:57:16.273728+0000","last_undegraded":"2024-04-25T18:57:16.273728+0000","last_fullsized":"2024-04-25T18:57:16.273728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:56.063786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.259553+0000","last_change":"2024-04-25T18:56:17.560167+0000","last_active":"2024-04-25T18:57:16.259553+0000","last_peered":"2024-04-25T18:57:16.259553+0000","last_clean":"2024-04-25T18:57:16.259553+0000","last_became_active":"2024-04-25T18:56:17.558466+0000","last_became_peered":"2024-04-25T18:56:17.558466+0000","last_unstale":"2024-04-25T18:57:16.259553+0000","last_undegraded":"2024-04-25T18:57:16.259553+0000","last_fullsized":"2024-04-25T18:57:16.259553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:22.371076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.244648+0000","last_change":"2024-04-25T18:55:23.233512+0000","last_active":"2024-04-25T18:57:16.244648+0000","last_peered":"2024-04-25T18:57:16.244648+0000","last_clean":"2024-04-25T18:57:16.244648+0000","last_became_active":"2024-04-25T18:55:23.233158+0000","last_became_peered":"2024-04-25T18:55:23.233158+0000","last_unstale":"2024-04-25T18:57:16.244648+0000","last_undegraded":"2024-04-25T18:57:16.244648+0000","last_fullsized":"2024-04-25T18:57:16.244648+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:44.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.264597+0000","last_change":"2024-04-25T18:55:23.225921+0000","last_active":"2024-04-25T18:57:16.264597+0000","last_peered":"2024-04-25T18:57:16.264597+0000","last_clean":"2024-04-25T18:57:16.264597+0000","last_became_active":"2024-04-25T18:55:23.225547+0000","last_became_peered":"2024-04-25T18:55:23.225547+0000","last_unstale":"2024-04-25T18:57:16.264597+0000","last_undegraded":"2024-04-25T18:57:16.264597+0000","last_fullsized":"2024-04-25T18:57:16.264597+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:29.828394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.262596+0000","last_change":"2024-04-25T18:55:23.217111+0000","last_active":"2024-04-25T18:57:16.262596+0000","last_peered":"2024-04-25T18:57:16.262596+0000","last_clean":"2024-04-25T18:57:16.262596+0000","last_became_active":"2024-04-25T18:55:23.216731+0000","last_became_peered":"2024-04-25T18:55:23.216731+0000","last_unstale":"2024-04-25T18:57:16.262596+0000","last_undegraded":"2024-04-25T18:57:16.262596+0000","last_fullsized":"2024-04-25T18:57:16.262596+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:17.931928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.259545+0000","last_change":"2024-04-25T18:56:17.555014+0000","last_active":"2024-04-25T18:57:16.259545+0000","last_peered":"2024-04-25T18:57:16.259545+0000","last_clean":"2024-04-25T18:57:16.259545+0000","last_became_active":"2024-04-25T18:56:17.553863+0000","last_became_peered":"2024-04-25T18:56:17.553863+0000","last_unstale":"2024-04-25T18:57:16.259545+0000","last_undegraded":"2024-04-25T18:57:16.259545+0000","last_fullsized":"2024-04-25T18:57:16.259545+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.240754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.250991+0000","last_change":"2024-04-25T18:55:23.218197+0000","last_active":"2024-04-25T18:57:16.250991+0000","last_peered":"2024-04-25T18:57:16.250991+0000","last_clean":"2024-04-25T18:57:16.250991+0000","last_became_active":"2024-04-25T18:55:23.217665+0000","last_became_peered":"2024-04-25T18:55:23.217665+0000","last_unstale":"2024-04-25T18:57:16.250991+0000","last_undegraded":"2024-04-25T18:57:16.250991+0000","last_fullsized":"2024-04-25T18:57:16.250991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:11.829991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.240858+0000","last_change":"2024-04-25T18:55:23.221533+0000","last_active":"2024-04-25T18:57:16.240858+0000","last_peered":"2024-04-25T18:57:16.240858+0000","last_clean":"2024-04-25T18:57:16.240858+0000","last_became_active":"2024-04-25T18:55:23.220711+0000","last_became_peered":"2024-04-25T18:55:23.220711+0000","last_unstale":"2024-04-25T18:57:16.240858+0000","last_undegraded":"2024-04-25T18:57:16.240858+0000","last_fullsized":"2024-04-25T18:57:16.240858+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:45.185166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.250298+0000","last_change":"2024-04-25T18:55:23.221329+0000","last_active":"2024-04-25T18:57:16.250298+0000","last_peered":"2024-04-25T18:57:16.250298+0000","last_clean":"2024-04-25T18:57:16.250298+0000","last_became_active":"2024-04-25T18:55:23.220156+0000","last_became_peered":"2024-04-25T18:55:23.220156+0000","last_unstale":"2024-04-25T18:57:16.250298+0000","last_undegraded":"2024-04-25T18:57:16.250298+0000","last_fullsized":"2024-04-25T18:57:16.250298+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:46.981199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.260612+0000","last_change":"2024-04-25T18:56:17.558018+0000","last_active":"2024-04-25T18:57:16.260612+0000","last_peered":"2024-04-25T18:57:16.260612+0000","last_clean":"2024-04-25T18:57:16.260612+0000","last_became_active":"2024-04-25T18:56:17.556229+0000","last_became_peered":"2024-04-25T18:56:17.556229+0000","last_unstale":"2024-04-25T18:57:16.260612+0000","last_undegraded":"2024-04-25T18:57:16.260612+0000","last_fullsized":"2024-04-25T18:57:16.260612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:50.949895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.248414+0000","last_change":"2024-04-25T18:55:23.227474+0000","last_active":"2024-04-25T18:57:16.248414+0000","last_peered":"2024-04-25T18:57:16.248414+0000","last_clean":"2024-04-25T18:57:16.248414+0000","last_became_active":"2024-04-25T18:55:23.227068+0000","last_became_peered":"2024-04-25T18:55:23.227068+0000","last_unstale":"2024-04-25T18:57:16.248414+0000","last_undegraded":"2024-04-25T18:57:16.248414+0000","last_fullsized":"2024-04-25T18:57:16.248414+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:13.884833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.271663+0000","last_change":"2024-04-25T18:55:23.230572+0000","last_active":"2024-04-25T18:57:16.271663+0000","last_peered":"2024-04-25T18:57:16.271663+0000","last_clean":"2024-04-25T18:57:16.271663+0000","last_became_active":"2024-04-25T18:55:23.229756+0000","last_became_peered":"2024-04-25T18:55:23.229756+0000","last_unstale":"2024-04-25T18:57:16.271663+0000","last_undegraded":"2024-04-25T18:57:16.271663+0000","last_fullsized":"2024-04-25T18:57:16.271663+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:15.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.271112+0000","last_change":"2024-04-25T18:55:23.227913+0000","last_active":"2024-04-25T18:57:16.271112+0000","last_peered":"2024-04-25T18:57:16.271112+0000","last_clean":"2024-04-25T18:57:16.271112+0000","last_became_active":"2024-04-25T18:55:23.227598+0000","last_became_peered":"2024-04-25T18:55:23.227598+0000","last_unstale":"2024-04-25T18:57:16.271112+0000","last_undegraded":"2024-04-25T18:57:16.271112+0000","last_fullsized":"2024-04-25T18:57:16.271112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:33.555741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.273630+0000","last_change":"2024-04-25T18:55:23.230241+0000","last_active":"2024-04-25T18:57:16.273630+0000","last_peered":"2024-04-25T18:57:16.273630+0000","last_clean":"2024-04-25T18:57:16.273630+0000","last_became_active":"2024-04-25T18:55:23.229898+0000","last_became_peered":"2024-04-25T18:55:23.229898+0000","last_unstale":"2024-04-25T18:57:16.273630+0000","last_undegraded":"2024-04-25T18:57:16.273630+0000","last_fullsized":"2024-04-25T18:57:16.273630+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:27.278475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.274198+0000","last_change":"2024-04-25T18:55:23.240978+0000","last_active":"2024-04-25T18:57:16.274198+0000","last_peered":"2024-04-25T18:57:16.274198+0000","last_clean":"2024-04-25T18:57:16.274198+0000","last_became_active":"2024-04-25T18:55:23.240702+0000","last_became_peered":"2024-04-25T18:55:23.240702+0000","last_unstale":"2024-04-25T18:57:16.274198+0000","last_undegraded":"2024-04-25T18:57:16.274198+0000","last_fullsized":"2024-04-25T18:57:16.274198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:06.053688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.262683+0000","last_change":"2024-04-25T18:55:23.232116+0000","last_active":"2024-04-25T18:57:16.262683+0000","last_peered":"2024-04-25T18:57:16.262683+0000","last_clean":"2024-04-25T18:57:16.262683+0000","last_became_active":"2024-04-25T18:55:23.231586+0000","last_became_peered":"2024-04-25T18:55:23.231586+0000","last_unstale":"2024-04-25T18:57:16.262683+0000","last_undegraded":"2024-04-25T18:57:16.262683+0000","last_fullsized":"2024-04-25T18:57:16.262683+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:32.640823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.272600+0000","last_change":"2024-04-25T18:55:23.224519+0000","last_active":"2024-04-25T18:57:16.272600+0000","last_peered":"2024-04-25T18:57:16.272600+0000","last_clean":"2024-04-25T18:57:16.272600+0000","last_became_active":"2024-04-25T18:55:23.224145+0000","last_became_peered":"2024-04-25T18:55:23.224145+0000","last_unstale":"2024-04-25T18:57:16.272600+0000","last_undegraded":"2024-04-25T18:57:16.272600+0000","last_fullsized":"2024-04-25T18:57:16.272600+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:03.340162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.548403+0000","last_change":"2024-04-25T18:55:19.164977+0000","last_active":"2024-04-25T18:56:17.548403+0000","last_peered":"2024-04-25T18:56:17.548403+0000","last_clean":"2024-04-25T18:56:17.548403+0000","last_became_active":"2024-04-25T18:55:19.164316+0000","last_became_peered":"2024-04-25T18:55:19.164316+0000","last_unstale":"2024-04-25T18:56:17.548403+0000","last_undegraded":"2024-04-25T18:56:17.548403+0000","last_fullsized":"2024-04-25T18:56:17.548403+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:08.031680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.274666+0000","last_change":"2024-04-25T18:56:17.559746+0000","last_active":"2024-04-25T18:57:16.274666+0000","last_peered":"2024-04-25T18:57:16.274666+0000","last_clean":"2024-04-25T18:57:16.274666+0000","last_became_active":"2024-04-25T18:56:17.558761+0000","last_became_peered":"2024-04-25T18:56:17.558761+0000","last_unstale":"2024-04-25T18:57:16.274666+0000","last_undegraded":"2024-04-25T18:57:16.274666+0000","last_fullsized":"2024-04-25T18:57:16.274666+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:43.518760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.547499+0000","last_change":"2024-04-25T18:55:19.179943+0000","last_active":"2024-04-25T18:56:17.547499+0000","last_peered":"2024-04-25T18:56:17.547499+0000","last_clean":"2024-04-25T18:56:17.547499+0000","last_became_active":"2024-04-25T18:55:19.179519+0000","last_became_peered":"2024-04-25T18:55:19.179519+0000","last_unstale":"2024-04-25T18:56:17.547499+0000","last_undegraded":"2024-04-25T18:56:17.547499+0000","last_fullsized":"2024-04-25T18:56:17.547499+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:46.365187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.273246+0000","last_change":"2024-04-25T18:55:23.237965+0000","last_active":"2024-04-25T18:57:16.273246+0000","last_peered":"2024-04-25T18:57:16.273246+0000","last_clean":"2024-04-25T18:57:16.273246+0000","last_became_active":"2024-04-25T18:55:23.237520+0000","last_became_peered":"2024-04-25T18:55:23.237520+0000","last_unstale":"2024-04-25T18:57:16.273246+0000","last_undegraded":"2024-04-25T18:57:16.273246+0000","last_fullsized":"2024-04-25T18:57:16.273246+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:03.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540660+0000","last_change":"2024-04-25T18:55:19.163965+0000","last_active":"2024-04-25T18:56:17.540660+0000","last_peered":"2024-04-25T18:56:17.540660+0000","last_clean":"2024-04-25T18:56:17.540660+0000","last_became_active":"2024-04-25T18:55:19.163390+0000","last_became_peered":"2024-04-25T18:55:19.163390+0000","last_unstale":"2024-04-25T18:56:17.540660+0000","last_undegraded":"2024-04-25T18:56:17.540660+0000","last_fullsized":"2024-04-25T18:56:17.540660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:54.183730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.265552+0000","last_change":"2024-04-25T18:55:23.227124+0000","last_active":"2024-04-25T18:57:16.265552+0000","last_peered":"2024-04-25T18:57:16.265552+0000","last_clean":"2024-04-25T18:57:16.265552+0000","last_became_active":"2024-04-25T18:55:23.226383+0000","last_became_peered":"2024-04-25T18:55:23.226383+0000","last_unstale":"2024-04-25T18:57:16.265552+0000","last_undegraded":"2024-04-25T18:57:16.265552+0000","last_fullsized":"2024-04-25T18:57:16.265552+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:22.011212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":15,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.544679+0000","last_change":"2024-04-25T18:56:17.544303+0000","last_active":"2024-04-25T18:56:17.544679+0000","last_peered":"2024-04-25T18:56:17.544679+0000","last_clean":"2024-04-25T18:56:17.544679+0000","last_became_active":"2024-04-25T18:56:17.542002+0000","last_became_peered":"2024-04-25T18:56:17.542002+0000","last_unstale":"2024-04-25T18:56:17.544679+0000","last_undegraded":"2024-04-25T18:56:17.544679+0000","last_fullsized":"2024-04-25T18:56:17.544679+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:23.848398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.242121+0000","last_change":"2024-04-25T18:55:23.217703+0000","last_active":"2024-04-25T18:57:16.242121+0000","last_peered":"2024-04-25T18:57:16.242121+0000","last_clean":"2024-04-25T18:57:16.242121+0000","last_became_active":"2024-04-25T18:55:23.216767+0000","last_became_peered":"2024-04-25T18:55:23.216767+0000","last_unstale":"2024-04-25T18:57:16.242121+0000","last_undegraded":"2024-04-25T18:57:16.242121+0000","last_fullsized":"2024-04-25T18:57:16.242121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:17.779356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"94'7","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:42.595034+0000","last_change":"2024-04-25T18:55:19.164199+0000","last_active":"2024-04-25T18:57:42.595034+0000","last_peered":"2024-04-25T18:57:42.595034+0000","last_clean":"2024-04-25T18:57:42.595034+0000","last_became_active":"2024-04-25T18:55:19.163849+0000","last_became_peered":"2024-04-25T18:55:19.163849+0000","last_unstale":"2024-04-25T18:57:42.595034+0000","last_undegraded":"2024-04-25T18:57:42.595034+0000","last_fullsized":"2024-04-25T18:57:42.595034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:25.234229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.244875+0000","last_change":"2024-04-25T18:55:23.216916+0000","last_active":"2024-04-25T18:57:16.244875+0000","last_peered":"2024-04-25T18:57:16.244875+0000","last_clean":"2024-04-25T18:57:16.244875+0000","last_became_active":"2024-04-25T18:55:23.216249+0000","last_became_peered":"2024-04-25T18:55:23.216249+0000","last_unstale":"2024-04-25T18:57:16.244875+0000","last_undegraded":"2024-04-25T18:57:16.244875+0000","last_fullsized":"2024-04-25T18:57:16.244875+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:21.522318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540439+0000","last_change":"2024-04-25T18:55:19.171004+0000","last_active":"2024-04-25T18:56:17.540439+0000","last_peered":"2024-04-25T18:56:17.540439+0000","last_clean":"2024-04-25T18:56:17.540439+0000","last_became_active":"2024-04-25T18:55:19.170570+0000","last_became_peered":"2024-04-25T18:55:19.170570+0000","last_unstale":"2024-04-25T18:56:17.540439+0000","last_undegraded":"2024-04-25T18:56:17.540439+0000","last_fullsized":"2024-04-25T18:56:17.540439+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:15:24.954033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.266668+0000","last_change":"2024-04-25T18:56:17.552615+0000","last_active":"2024-04-25T18:57:16.266668+0000","last_peered":"2024-04-25T18:57:16.266668+0000","last_clean":"2024-04-25T18:57:16.266668+0000","last_became_active":"2024-04-25T18:56:17.552154+0000","last_became_peered":"2024-04-25T18:56:17.552154+0000","last_unstale":"2024-04-25T18:57:16.266668+0000","last_undegraded":"2024-04-25T18:57:16.266668+0000","last_fullsized":"2024-04-25T18:57:16.266668+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:01.199859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.549496+0000","last_change":"2024-04-25T18:55:19.173398+0000","last_active":"2024-04-25T18:56:17.549496+0000","last_peered":"2024-04-25T18:56:17.549496+0000","last_clean":"2024-04-25T18:56:17.549496+0000","last_became_active":"2024-04-25T18:55:19.171477+0000","last_became_peered":"2024-04-25T18:55:19.171477+0000","last_unstale":"2024-04-25T18:56:17.549496+0000","last_undegraded":"2024-04-25T18:56:17.549496+0000","last_fullsized":"2024-04-25T18:56:17.549496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:35.230596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.274477+0000","last_change":"2024-04-25T18:55:23.235209+0000","last_active":"2024-04-25T18:57:16.274477+0000","last_peered":"2024-04-25T18:57:16.274477+0000","last_clean":"2024-04-25T18:57:16.274477+0000","last_became_active":"2024-04-25T18:55:23.231898+0000","last_became_peered":"2024-04-25T18:55:23.231898+0000","last_unstale":"2024-04-25T18:57:16.274477+0000","last_undegraded":"2024-04-25T18:57:16.274477+0000","last_fullsized":"2024-04-25T18:57:16.274477+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.521282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540044+0000","last_change":"2024-04-25T18:55:19.163780+0000","last_active":"2024-04-25T18:56:17.540044+0000","last_peered":"2024-04-25T18:56:17.540044+0000","last_clean":"2024-04-25T18:56:17.540044+0000","last_became_active":"2024-04-25T18:55:19.163197+0000","last_became_peered":"2024-04-25T18:55:19.163197+0000","last_unstale":"2024-04-25T18:56:17.540044+0000","last_undegraded":"2024-04-25T18:56:17.540044+0000","last_fullsized":"2024-04-25T18:56:17.540044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:02:26.519160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.264232+0000","last_change":"2024-04-25T18:56:17.556804+0000","last_active":"2024-04-25T18:57:16.264232+0000","last_peered":"2024-04-25T18:57:16.264232+0000","last_clean":"2024-04-25T18:57:16.264232+0000","last_became_active":"2024-04-25T18:56:17.554885+0000","last_became_peered":"2024-04-25T18:56:17.554885+0000","last_unstale":"2024-04-25T18:57:16.264232+0000","last_undegraded":"2024-04-25T18:57:16.264232+0000","last_fullsized":"2024-04-25T18:57:16.264232+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:36.316156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"94'4","reported_seq":145,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:39.590311+0000","last_change":"2024-04-25T18:55:19.170565+0000","last_active":"2024-04-25T18:57:39.590311+0000","last_peered":"2024-04-25T18:57:39.590311+0000","last_clean":"2024-04-25T18:57:39.590311+0000","last_became_active":"2024-04-25T18:55:19.170162+0000","last_became_peered":"2024-04-25T18:55:19.170162+0000","last_unstale":"2024-04-25T18:57:39.590311+0000","last_undegraded":"2024-04-25T18:57:39.590311+0000","last_fullsized":"2024-04-25T18:57:39.590311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:36.606661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.263589+0000","last_change":"2024-04-25T18:55:23.227436+0000","last_active":"2024-04-25T18:57:16.263589+0000","last_peered":"2024-04-25T18:57:16.263589+0000","last_clean":"2024-04-25T18:57:16.263589+0000","last_became_active":"2024-04-25T18:55:23.227044+0000","last_became_peered":"2024-04-25T18:55:23.227044+0000","last_unstale":"2024-04-25T18:57:16.263589+0000","last_undegraded":"2024-04-25T18:57:16.263589+0000","last_fullsized":"2024-04-25T18:57:16.263589+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:34.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583580+0000","last_change":"2024-04-25T18:55:19.167514+0000","last_active":"2024-04-25T18:56:17.583580+0000","last_peered":"2024-04-25T18:56:17.583580+0000","last_clean":"2024-04-25T18:56:17.583580+0000","last_became_active":"2024-04-25T18:55:19.167102+0000","last_became_peered":"2024-04-25T18:55:19.167102+0000","last_unstale":"2024-04-25T18:56:17.583580+0000","last_undegraded":"2024-04-25T18:56:17.583580+0000","last_fullsized":"2024-04-25T18:56:17.583580+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:41.472274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.270537+0000","last_change":"2024-04-25T18:55:23.237012+0000","last_active":"2024-04-25T18:57:16.270537+0000","last_peered":"2024-04-25T18:57:16.270537+0000","last_clean":"2024-04-25T18:57:16.270537+0000","last_became_active":"2024-04-25T18:55:23.230263+0000","last_became_peered":"2024-04-25T18:55:23.230263+0000","last_unstale":"2024-04-25T18:57:16.270537+0000","last_undegraded":"2024-04-25T18:57:16.270537+0000","last_fullsized":"2024-04-25T18:57:16.270537+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:30.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540301+0000","last_change":"2024-04-25T18:55:19.179555+0000","last_active":"2024-04-25T18:56:17.540301+0000","last_peered":"2024-04-25T18:56:17.540301+0000","last_clean":"2024-04-25T18:56:17.540301+0000","last_became_active":"2024-04-25T18:55:19.179190+0000","last_became_peered":"2024-04-25T18:55:19.179190+0000","last_unstale":"2024-04-25T18:56:17.540301+0000","last_undegraded":"2024-04-25T18:56:17.540301+0000","last_fullsized":"2024-04-25T18:56:17.540301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:57.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.255330+0000","last_change":"2024-04-25T18:55:23.219978+0000","last_active":"2024-04-25T18:57:16.255330+0000","last_peered":"2024-04-25T18:57:16.255330+0000","last_clean":"2024-04-25T18:57:16.255330+0000","last_became_active":"2024-04-25T18:55:23.219630+0000","last_became_peered":"2024-04-25T18:55:23.219630+0000","last_unstale":"2024-04-25T18:57:16.255330+0000","last_undegraded":"2024-04-25T18:57:16.255330+0000","last_fullsized":"2024-04-25T18:57:16.255330+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:31.294500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'35","reported_seq":303,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:40.592308+0000","last_change":"2024-04-25T18:55:19.164042+0000","last_active":"2024-04-25T18:57:40.592308+0000","last_peered":"2024-04-25T18:57:40.592308+0000","last_clean":"2024-04-25T18:57:40.592308+0000","last_became_active":"2024-04-25T18:55:19.163604+0000","last_became_peered":"2024-04-25T18:55:19.163604+0000","last_unstale":"2024-04-25T18:57:40.592308+0000","last_undegraded":"2024-04-25T18:57:40.592308+0000","last_fullsized":"2024-04-25T18:57:40.592308+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:55.030715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64973,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":241,"num_read_kb":183,"num_write":36,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.239140+0000","last_change":"2024-04-25T18:55:23.222702+0000","last_active":"2024-04-25T18:57:16.239140+0000","last_peered":"2024-04-25T18:57:16.239140+0000","last_clean":"2024-04-25T18:57:16.239140+0000","last_became_active":"2024-04-25T18:55:23.221849+0000","last_became_peered":"2024-04-25T18:55:23.221849+0000","last_unstale":"2024-04-25T18:57:16.239140+0000","last_undegraded":"2024-04-25T18:57:16.239140+0000","last_fullsized":"2024-04-25T18:57:16.239140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.957027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":21,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.562287+0000","last_change":"2024-04-25T18:56:17.561912+0000","last_active":"2024-04-25T18:56:17.562287+0000","last_peered":"2024-04-25T18:56:17.562287+0000","last_clean":"2024-04-25T18:56:17.562287+0000","last_became_active":"2024-04-25T18:56:17.552120+0000","last_became_peered":"2024-04-25T18:56:17.552120+0000","last_unstale":"2024-04-25T18:56:17.562287+0000","last_undegraded":"2024-04-25T18:56:17.562287+0000","last_fullsized":"2024-04-25T18:56:17.562287+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:30.209802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.276712+0000","last_change":"2024-04-25T18:55:23.233074+0000","last_active":"2024-04-25T18:57:16.276712+0000","last_peered":"2024-04-25T18:57:16.276712+0000","last_clean":"2024-04-25T18:57:16.276712+0000","last_became_active":"2024-04-25T18:55:23.232751+0000","last_became_peered":"2024-04-25T18:55:23.232751+0000","last_unstale":"2024-04-25T18:57:16.276712+0000","last_undegraded":"2024-04-25T18:57:16.276712+0000","last_fullsized":"2024-04-25T18:57:16.276712+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.584929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540218+0000","last_change":"2024-04-25T18:55:19.166093+0000","last_active":"2024-04-25T18:56:17.540218+0000","last_peered":"2024-04-25T18:56:17.540218+0000","last_clean":"2024-04-25T18:56:17.540218+0000","last_became_active":"2024-04-25T18:55:19.165715+0000","last_became_peered":"2024-04-25T18:55:19.165715+0000","last_unstale":"2024-04-25T18:56:17.540218+0000","last_undegraded":"2024-04-25T18:56:17.540218+0000","last_fullsized":"2024-04-25T18:56:17.540218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:32.177943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.262471+0000","last_change":"2024-04-25T18:55:23.224617+0000","last_active":"2024-04-25T18:57:16.262471+0000","last_peered":"2024-04-25T18:57:16.262471+0000","last_clean":"2024-04-25T18:57:16.262471+0000","last_became_active":"2024-04-25T18:55:23.224269+0000","last_became_peered":"2024-04-25T18:55:23.224269+0000","last_unstale":"2024-04-25T18:57:16.262471+0000","last_undegraded":"2024-04-25T18:57:16.262471+0000","last_fullsized":"2024-04-25T18:57:16.262471+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:19.079910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":155,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:43.683821+0000","last_change":"2024-04-25T18:55:19.164971+0000","last_active":"2024-04-25T18:57:43.683821+0000","last_peered":"2024-04-25T18:57:43.683821+0000","last_clean":"2024-04-25T18:57:43.683821+0000","last_became_active":"2024-04-25T18:55:19.164579+0000","last_became_peered":"2024-04-25T18:55:19.164579+0000","last_unstale":"2024-04-25T18:57:43.683821+0000","last_undegraded":"2024-04-25T18:57:43.683821+0000","last_fullsized":"2024-04-25T18:57:43.683821+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:37.419379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.236876+0000","last_change":"2024-04-25T18:55:23.222160+0000","last_active":"2024-04-25T18:57:16.236876+0000","last_peered":"2024-04-25T18:57:16.236876+0000","last_clean":"2024-04-25T18:57:16.236876+0000","last_became_active":"2024-04-25T18:55:23.221812+0000","last_became_peered":"2024-04-25T18:55:23.221812+0000","last_unstale":"2024-04-25T18:57:16.236876+0000","last_undegraded":"2024-04-25T18:57:16.236876+0000","last_fullsized":"2024-04-25T18:57:16.236876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:53:53.388764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540123+0000","last_change":"2024-04-25T18:55:19.179970+0000","last_active":"2024-04-25T18:56:17.540123+0000","last_peered":"2024-04-25T18:56:17.540123+0000","last_clean":"2024-04-25T18:56:17.540123+0000","last_became_active":"2024-04-25T18:55:19.179545+0000","last_became_peered":"2024-04-25T18:55:19.179545+0000","last_unstale":"2024-04-25T18:56:17.540123+0000","last_undegraded":"2024-04-25T18:56:17.540123+0000","last_fullsized":"2024-04-25T18:56:17.540123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:42.841661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.242037+0000","last_change":"2024-04-25T18:55:23.216134+0000","last_active":"2024-04-25T18:57:16.242037+0000","last_peered":"2024-04-25T18:57:16.242037+0000","last_clean":"2024-04-25T18:57:16.242037+0000","last_became_active":"2024-04-25T18:55:23.215535+0000","last_became_peered":"2024-04-25T18:55:23.215535+0000","last_unstale":"2024-04-25T18:57:16.242037+0000","last_undegraded":"2024-04-25T18:57:16.242037+0000","last_fullsized":"2024-04-25T18:57:16.242037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:08.042095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540722+0000","last_change":"2024-04-25T18:55:19.162437+0000","last_active":"2024-04-25T18:56:17.540722+0000","last_peered":"2024-04-25T18:56:17.540722+0000","last_clean":"2024-04-25T18:56:17.540722+0000","last_became_active":"2024-04-25T18:55:19.162005+0000","last_became_peered":"2024-04-25T18:55:19.162005+0000","last_unstale":"2024-04-25T18:56:17.540722+0000","last_undegraded":"2024-04-25T18:56:17.540722+0000","last_fullsized":"2024-04-25T18:56:17.540722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:50:53.967081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.238362+0000","last_change":"2024-04-25T18:55:23.222345+0000","last_active":"2024-04-25T18:57:16.238362+0000","last_peered":"2024-04-25T18:57:16.238362+0000","last_clean":"2024-04-25T18:57:16.238362+0000","last_became_active":"2024-04-25T18:55:23.221142+0000","last_became_peered":"2024-04-25T18:55:23.221142+0000","last_unstale":"2024-04-25T18:57:16.238362+0000","last_undegraded":"2024-04-25T18:57:16.238362+0000","last_fullsized":"2024-04-25T18:57:16.238362+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:53.823585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.550654+0000","last_change":"2024-04-25T18:55:19.181153+0000","last_active":"2024-04-25T18:56:17.550654+0000","last_peered":"2024-04-25T18:56:17.550654+0000","last_clean":"2024-04-25T18:56:17.550654+0000","last_became_active":"2024-04-25T18:55:19.180553+0000","last_became_peered":"2024-04-25T18:55:19.180553+0000","last_unstale":"2024-04-25T18:56:17.550654+0000","last_undegraded":"2024-04-25T18:56:17.550654+0000","last_fullsized":"2024-04-25T18:56:17.550654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:07.829617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.265847+0000","last_change":"2024-04-25T18:55:23.238824+0000","last_active":"2024-04-25T18:57:16.265847+0000","last_peered":"2024-04-25T18:57:16.265847+0000","last_clean":"2024-04-25T18:57:16.265847+0000","last_became_active":"2024-04-25T18:55:23.238228+0000","last_became_peered":"2024-04-25T18:55:23.238228+0000","last_unstale":"2024-04-25T18:57:16.265847+0000","last_undegraded":"2024-04-25T18:57:16.265847+0000","last_fullsized":"2024-04-25T18:57:16.265847+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:16.279254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583366+0000","last_change":"2024-04-25T18:55:19.171186+0000","last_active":"2024-04-25T18:56:17.583366+0000","last_peered":"2024-04-25T18:56:17.583366+0000","last_clean":"2024-04-25T18:56:17.583366+0000","last_became_active":"2024-04-25T18:55:19.170800+0000","last_became_peered":"2024-04-25T18:55:19.170800+0000","last_unstale":"2024-04-25T18:56:17.583366+0000","last_undegraded":"2024-04-25T18:56:17.583366+0000","last_fullsized":"2024-04-25T18:56:17.583366+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.599119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.261311+0000","last_change":"2024-04-25T18:55:23.236951+0000","last_active":"2024-04-25T18:57:16.261311+0000","last_peered":"2024-04-25T18:57:16.261311+0000","last_clean":"2024-04-25T18:57:16.261311+0000","last_became_active":"2024-04-25T18:55:23.236596+0000","last_became_peered":"2024-04-25T18:55:23.236596+0000","last_unstale":"2024-04-25T18:57:16.261311+0000","last_undegraded":"2024-04-25T18:57:16.261311+0000","last_fullsized":"2024-04-25T18:57:16.261311+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:14.634653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.240085+0000","last_change":"2024-04-25T18:55:23.211984+0000","last_active":"2024-04-25T18:57:16.240085+0000","last_peered":"2024-04-25T18:57:16.240085+0000","last_clean":"2024-04-25T18:57:16.240085+0000","last_became_active":"2024-04-25T18:55:23.211591+0000","last_became_peered":"2024-04-25T18:55:23.211591+0000","last_unstale":"2024-04-25T18:57:16.240085+0000","last_undegraded":"2024-04-25T18:57:16.240085+0000","last_fullsized":"2024-04-25T18:57:16.240085+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:09.024891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":150,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:42.680316+0000","last_change":"2024-04-25T18:55:19.180378+0000","last_active":"2024-04-25T18:57:42.680316+0000","last_peered":"2024-04-25T18:57:42.680316+0000","last_clean":"2024-04-25T18:57:42.680316+0000","last_became_active":"2024-04-25T18:55:19.179905+0000","last_became_peered":"2024-04-25T18:55:19.179905+0000","last_unstale":"2024-04-25T18:57:42.680316+0000","last_undegraded":"2024-04-25T18:57:42.680316+0000","last_fullsized":"2024-04-25T18:57:42.680316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:36.434477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":118,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.275037+0000","last_change":"2024-04-25T18:55:23.231489+0000","last_active":"2024-04-25T18:57:16.275037+0000","last_peered":"2024-04-25T18:57:16.275037+0000","last_clean":"2024-04-25T18:57:16.275037+0000","last_became_active":"2024-04-25T18:55:23.231049+0000","last_became_peered":"2024-04-25T18:55:23.231049+0000","last_unstale":"2024-04-25T18:57:16.275037+0000","last_undegraded":"2024-04-25T18:57:16.275037+0000","last_fullsized":"2024-04-25T18:57:16.275037+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:25.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539314+0000","last_change":"2024-04-25T18:55:19.171694+0000","last_active":"2024-04-25T18:56:17.539314+0000","last_peered":"2024-04-25T18:56:17.539314+0000","last_clean":"2024-04-25T18:56:17.539314+0000","last_became_active":"2024-04-25T18:55:19.171296+0000","last_became_peered":"2024-04-25T18:55:19.171296+0000","last_unstale":"2024-04-25T18:56:17.539314+0000","last_undegraded":"2024-04-25T18:56:17.539314+0000","last_fullsized":"2024-04-25T18:56:17.539314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:33.216980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583253+0000","last_change":"2024-04-25T18:53:49.127506+0000","last_active":"2024-04-25T18:56:17.583253+0000","last_peered":"2024-04-25T18:56:17.583253+0000","last_clean":"2024-04-25T18:56:17.583253+0000","last_became_active":"2024-04-25T18:53:49.126974+0000","last_became_peered":"2024-04-25T18:53:49.126974+0000","last_unstale":"2024-04-25T18:56:17.583253+0000","last_undegraded":"2024-04-25T18:56:17.583253+0000","last_fullsized":"2024-04-25T18:56:17.583253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_clean_scrub_stamp":"2024-04-25T18:50:00.765468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.488525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.245945+0000","last_change":"2024-04-25T18:55:23.211697+0000","last_active":"2024-04-25T18:57:16.245945+0000","last_peered":"2024-04-25T18:57:16.245945+0000","last_clean":"2024-04-25T18:57:16.245945+0000","last_became_active":"2024-04-25T18:55:23.211160+0000","last_became_peered":"2024-04-25T18:55:23.211160+0000","last_unstale":"2024-04-25T18:57:16.245945+0000","last_undegraded":"2024-04-25T18:57:16.245945+0000","last_fullsized":"2024-04-25T18:57:16.245945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:03.452578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.583335+0000","last_change":"2024-04-25T18:55:19.165955+0000","last_active":"2024-04-25T18:56:17.583335+0000","last_peered":"2024-04-25T18:56:17.583335+0000","last_clean":"2024-04-25T18:56:17.583335+0000","last_became_active":"2024-04-25T18:55:19.165594+0000","last_became_peered":"2024-04-25T18:55:19.165594+0000","last_unstale":"2024-04-25T18:56:17.583335+0000","last_undegraded":"2024-04-25T18:56:17.583335+0000","last_fullsized":"2024-04-25T18:56:17.583335+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:11.692374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540598+0000","last_change":"2024-04-25T18:55:19.182698+0000","last_active":"2024-04-25T18:56:17.540598+0000","last_peered":"2024-04-25T18:56:17.540598+0000","last_clean":"2024-04-25T18:56:17.540598+0000","last_became_active":"2024-04-25T18:55:19.182393+0000","last_became_peered":"2024-04-25T18:55:19.182393+0000","last_unstale":"2024-04-25T18:56:17.540598+0000","last_undegraded":"2024-04-25T18:56:17.540598+0000","last_fullsized":"2024-04-25T18:56:17.540598+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.513613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.265788+0000","last_change":"2024-04-25T18:55:23.223240+0000","last_active":"2024-04-25T18:57:16.265788+0000","last_peered":"2024-04-25T18:57:16.265788+0000","last_clean":"2024-04-25T18:57:16.265788+0000","last_became_active":"2024-04-25T18:55:23.218253+0000","last_became_peered":"2024-04-25T18:55:23.218253+0000","last_unstale":"2024-04-25T18:57:16.265788+0000","last_undegraded":"2024-04-25T18:57:16.265788+0000","last_fullsized":"2024-04-25T18:57:16.265788+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:40.435633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'13","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.277105+0000","last_change":"2024-04-25T18:55:19.168535+0000","last_active":"2024-04-25T18:57:16.277105+0000","last_peered":"2024-04-25T18:57:16.277105+0000","last_clean":"2024-04-25T18:57:16.277105+0000","last_became_active":"2024-04-25T18:55:19.166735+0000","last_became_peered":"2024-04-25T18:55:19.166735+0000","last_unstale":"2024-04-25T18:57:16.277105+0000","last_undegraded":"2024-04-25T18:57:16.277105+0000","last_fullsized":"2024-04-25T18:57:16.277105+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:50:14.008290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.275975+0000","last_change":"2024-04-25T18:55:23.233989+0000","last_active":"2024-04-25T18:57:16.275975+0000","last_peered":"2024-04-25T18:57:16.275975+0000","last_clean":"2024-04-25T18:57:16.275975+0000","last_became_active":"2024-04-25T18:55:23.226867+0000","last_became_peered":"2024-04-25T18:55:23.226867+0000","last_unstale":"2024-04-25T18:57:16.275975+0000","last_undegraded":"2024-04-25T18:57:16.275975+0000","last_fullsized":"2024-04-25T18:57:16.275975+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:02.314368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.539538+0000","last_change":"2024-04-25T18:55:19.870412+0000","last_active":"2024-04-25T18:56:17.539538+0000","last_peered":"2024-04-25T18:56:17.539538+0000","last_clean":"2024-04-25T18:56:17.539538+0000","last_became_active":"2024-04-25T18:55:19.869970+0000","last_became_peered":"2024-04-25T18:55:19.869970+0000","last_unstale":"2024-04-25T18:56:17.539538+0000","last_undegraded":"2024-04-25T18:56:17.539538+0000","last_fullsized":"2024-04-25T18:56:17.539538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:22.236417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.277236+0000","last_change":"2024-04-25T18:55:23.239172+0000","last_active":"2024-04-25T18:57:16.277236+0000","last_peered":"2024-04-25T18:57:16.277236+0000","last_clean":"2024-04-25T18:57:16.277236+0000","last_became_active":"2024-04-25T18:55:23.238732+0000","last_became_peered":"2024-04-25T18:55:23.238732+0000","last_unstale":"2024-04-25T18:57:16.277236+0000","last_undegraded":"2024-04-25T18:57:16.277236+0000","last_fullsized":"2024-04-25T18:57:16.277236+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:27.459659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540801+0000","last_change":"2024-04-25T18:55:19.179687+0000","last_active":"2024-04-25T18:56:17.540801+0000","last_peered":"2024-04-25T18:56:17.540801+0000","last_clean":"2024-04-25T18:56:17.540801+0000","last_became_active":"2024-04-25T18:55:19.179225+0000","last_became_peered":"2024-04-25T18:55:19.179225+0000","last_unstale":"2024-04-25T18:56:17.540801+0000","last_undegraded":"2024-04-25T18:56:17.540801+0000","last_fullsized":"2024-04-25T18:56:17.540801+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:16:16.266021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.230530+0000","last_change":"2024-04-25T18:55:23.221379+0000","last_active":"2024-04-25T18:57:16.230530+0000","last_peered":"2024-04-25T18:57:16.230530+0000","last_clean":"2024-04-25T18:57:16.230530+0000","last_became_active":"2024-04-25T18:55:23.221037+0000","last_became_peered":"2024-04-25T18:55:23.221037+0000","last_unstale":"2024-04-25T18:57:16.230530+0000","last_undegraded":"2024-04-25T18:57:16.230530+0000","last_fullsized":"2024-04-25T18:57:16.230530+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:27.581084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.546313+0000","last_change":"2024-04-25T18:55:19.171298+0000","last_active":"2024-04-25T18:56:17.546313+0000","last_peered":"2024-04-25T18:56:17.546313+0000","last_clean":"2024-04-25T18:56:17.546313+0000","last_became_active":"2024-04-25T18:55:19.170738+0000","last_became_peered":"2024-04-25T18:55:19.170738+0000","last_unstale":"2024-04-25T18:56:17.546313+0000","last_undegraded":"2024-04-25T18:56:17.546313+0000","last_fullsized":"2024-04-25T18:56:17.546313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.236331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.274256+0000","last_change":"2024-04-25T18:55:23.226577+0000","last_active":"2024-04-25T18:57:16.274256+0000","last_peered":"2024-04-25T18:57:16.274256+0000","last_clean":"2024-04-25T18:57:16.274256+0000","last_became_active":"2024-04-25T18:55:23.226198+0000","last_became_peered":"2024-04-25T18:55:23.226198+0000","last_unstale":"2024-04-25T18:57:16.274256+0000","last_undegraded":"2024-04-25T18:57:16.274256+0000","last_fullsized":"2024-04-25T18:57:16.274256+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:34:59.124523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"92'1","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536406+0000","last_change":"2024-04-25T18:55:19.171917+0000","last_active":"2024-04-25T18:56:17.536406+0000","last_peered":"2024-04-25T18:56:17.536406+0000","last_clean":"2024-04-25T18:56:17.536406+0000","last_became_active":"2024-04-25T18:55:19.171506+0000","last_became_peered":"2024-04-25T18:55:19.171506+0000","last_unstale":"2024-04-25T18:56:17.536406+0000","last_undegraded":"2024-04-25T18:56:17.536406+0000","last_fullsized":"2024-04-25T18:56:17.536406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:53.446504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.275128+0000","last_change":"2024-04-25T18:55:23.237828+0000","last_active":"2024-04-25T18:57:16.275128+0000","last_peered":"2024-04-25T18:57:16.275128+0000","last_clean":"2024-04-25T18:57:16.275128+0000","last_became_active":"2024-04-25T18:55:23.237310+0000","last_became_peered":"2024-04-25T18:55:23.237310+0000","last_unstale":"2024-04-25T18:57:16.275128+0000","last_undegraded":"2024-04-25T18:57:16.275128+0000","last_fullsized":"2024-04-25T18:57:16.275128+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:10.987032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540312+0000","last_change":"2024-04-25T18:55:19.163824+0000","last_active":"2024-04-25T18:56:17.540312+0000","last_peered":"2024-04-25T18:56:17.540312+0000","last_clean":"2024-04-25T18:56:17.540312+0000","last_became_active":"2024-04-25T18:55:19.163465+0000","last_became_peered":"2024-04-25T18:55:19.163465+0000","last_unstale":"2024-04-25T18:56:17.540312+0000","last_undegraded":"2024-04-25T18:56:17.540312+0000","last_fullsized":"2024-04-25T18:56:17.540312+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:38.347100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.262441+0000","last_change":"2024-04-25T18:55:23.224648+0000","last_active":"2024-04-25T18:57:16.262441+0000","last_peered":"2024-04-25T18:57:16.262441+0000","last_clean":"2024-04-25T18:57:16.262441+0000","last_became_active":"2024-04-25T18:55:23.224323+0000","last_became_peered":"2024-04-25T18:55:23.224323+0000","last_unstale":"2024-04-25T18:57:16.262441+0000","last_undegraded":"2024-04-25T18:57:16.262441+0000","last_fullsized":"2024-04-25T18:57:16.262441+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.117792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.536728+0000","last_change":"2024-04-25T18:55:19.870213+0000","last_active":"2024-04-25T18:56:17.536728+0000","last_peered":"2024-04-25T18:56:17.536728+0000","last_clean":"2024-04-25T18:56:17.536728+0000","last_became_active":"2024-04-25T18:55:19.869756+0000","last_became_peered":"2024-04-25T18:55:19.869756+0000","last_unstale":"2024-04-25T18:56:17.536728+0000","last_undegraded":"2024-04-25T18:56:17.536728+0000","last_fullsized":"2024-04-25T18:56:17.536728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:44.695876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.250586+0000","last_change":"2024-04-25T18:55:23.217094+0000","last_active":"2024-04-25T18:57:16.250586+0000","last_peered":"2024-04-25T18:57:16.250586+0000","last_clean":"2024-04-25T18:57:16.250586+0000","last_became_active":"2024-04-25T18:55:23.216696+0000","last_became_peered":"2024-04-25T18:55:23.216696+0000","last_unstale":"2024-04-25T18:57:16.250586+0000","last_undegraded":"2024-04-25T18:57:16.250586+0000","last_fullsized":"2024-04-25T18:57:16.250586+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:07.049349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.540128+0000","last_change":"2024-04-25T18:55:19.166377+0000","last_active":"2024-04-25T18:56:17.540128+0000","last_peered":"2024-04-25T18:56:17.540128+0000","last_clean":"2024-04-25T18:56:17.540128+0000","last_became_active":"2024-04-25T18:55:19.166036+0000","last_became_peered":"2024-04-25T18:55:19.166036+0000","last_unstale":"2024-04-25T18:56:17.540128+0000","last_undegraded":"2024-04-25T18:56:17.540128+0000","last_fullsized":"2024-04-25T18:56:17.540128+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:04.761919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.273051+0000","last_change":"2024-04-25T18:55:23.236893+0000","last_active":"2024-04-25T18:57:16.273051+0000","last_peered":"2024-04-25T18:57:16.273051+0000","last_clean":"2024-04-25T18:57:16.273051+0000","last_became_active":"2024-04-25T18:55:23.236488+0000","last_became_peered":"2024-04-25T18:55:23.236488+0000","last_unstale":"2024-04-25T18:57:16.273051+0000","last_undegraded":"2024-04-25T18:57:16.273051+0000","last_fullsized":"2024-04-25T18:57:16.273051+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:44.605041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:56:17.551204+0000","last_change":"2024-04-25T18:55:19.175076+0000","last_active":"2024-04-25T18:56:17.551204+0000","last_peered":"2024-04-25T18:56:17.551204+0000","last_clean":"2024-04-25T18:56:17.551204+0000","last_became_active":"2024-04-25T18:55:19.174547+0000","last_became_peered":"2024-04-25T18:55:19.174547+0000","last_unstale":"2024-04-25T18:56:17.551204+0000","last_undegraded":"2024-04-25T18:56:17.551204+0000","last_fullsized":"2024-04-25T18:56:17.551204+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:43.791068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.247558+0000","last_change":"2024-04-25T18:55:23.219665+0000","last_active":"2024-04-25T18:57:16.247558+0000","last_peered":"2024-04-25T18:57:16.247558+0000","last_clean":"2024-04-25T18:57:16.247558+0000","last_became_active":"2024-04-25T18:55:23.219264+0000","last_became_peered":"2024-04-25T18:55:23.219264+0000","last_unstale":"2024-04-25T18:57:16.247558+0000","last_undegraded":"2024-04-25T18:57:16.247558+0000","last_fullsized":"2024-04-25T18:57:16.247558+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:19.008120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.239322+0000","last_change":"2024-04-25T18:55:23.220290+0000","last_active":"2024-04-25T18:57:16.239322+0000","last_peered":"2024-04-25T18:57:16.239322+0000","last_clean":"2024-04-25T18:57:16.239322+0000","last_became_active":"2024-04-25T18:55:23.219131+0000","last_became_peered":"2024-04-25T18:55:23.219131+0000","last_unstale":"2024-04-25T18:57:16.239322+0000","last_undegraded":"2024-04-25T18:57:16.239322+0000","last_fullsized":"2024-04-25T18:57:16.239322+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:25:25.678525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.242421+0000","last_change":"2024-04-25T18:55:23.216415+0000","last_active":"2024-04-25T18:57:16.242421+0000","last_peered":"2024-04-25T18:57:16.242421+0000","last_clean":"2024-04-25T18:57:16.242421+0000","last_became_active":"2024-04-25T18:55:23.216066+0000","last_became_peered":"2024-04-25T18:55:23.216066+0000","last_unstale":"2024-04-25T18:57:16.242421+0000","last_undegraded":"2024-04-25T18:57:16.242421+0000","last_fullsized":"2024-04-25T18:57:16.242421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:54.633216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.275888+0000","last_change":"2024-04-25T18:55:23.234034+0000","last_active":"2024-04-25T18:57:16.275888+0000","last_peered":"2024-04-25T18:57:16.275888+0000","last_clean":"2024-04-25T18:57:16.275888+0000","last_became_active":"2024-04-25T18:55:23.233639+0000","last_became_peered":"2024-04-25T18:55:23.233639+0000","last_unstale":"2024-04-25T18:57:16.275888+0000","last_undegraded":"2024-04-25T18:57:16.275888+0000","last_fullsized":"2024-04-25T18:57:16.275888+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:57.117958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.272622+0000","last_change":"2024-04-25T18:56:17.584511+0000","last_active":"2024-04-25T18:57:16.272622+0000","last_peered":"2024-04-25T18:57:16.272622+0000","last_clean":"2024-04-25T18:57:16.272622+0000","last_became_active":"2024-04-25T18:56:17.583959+0000","last_became_peered":"2024-04-25T18:56:17.583959+0000","last_unstale":"2024-04-25T18:57:16.272622+0000","last_undegraded":"2024-04-25T18:57:16.272622+0000","last_fullsized":"2024-04-25T18:57:16.272622+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.622823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.268811+0000","last_change":"2024-04-25T18:55:23.238068+0000","last_active":"2024-04-25T18:57:16.268811+0000","last_peered":"2024-04-25T18:57:16.268811+0000","last_clean":"2024-04-25T18:57:16.268811+0000","last_became_active":"2024-04-25T18:55:23.237773+0000","last_became_peered":"2024-04-25T18:55:23.237773+0000","last_unstale":"2024-04-25T18:57:16.268811+0000","last_undegraded":"2024-04-25T18:57:16.268811+0000","last_fullsized":"2024-04-25T18:57:16.268811+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.688576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.236776+0000","last_change":"2024-04-25T18:55:23.215463+0000","last_active":"2024-04-25T18:57:16.236776+0000","last_peered":"2024-04-25T18:57:16.236776+0000","last_clean":"2024-04-25T18:57:16.236776+0000","last_became_active":"2024-04-25T18:55:23.215068+0000","last_became_peered":"2024-04-25T18:55:23.215068+0000","last_unstale":"2024-04-25T18:57:16.236776+0000","last_undegraded":"2024-04-25T18:57:16.236776+0000","last_fullsized":"2024-04-25T18:57:16.236776+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.450828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.244816+0000","last_change":"2024-04-25T18:55:23.214362+0000","last_active":"2024-04-25T18:57:16.244816+0000","last_peered":"2024-04-25T18:57:16.244816+0000","last_clean":"2024-04-25T18:57:16.244816+0000","last_became_active":"2024-04-25T18:55:23.214035+0000","last_became_peered":"2024-04-25T18:55:23.214035+0000","last_unstale":"2024-04-25T18:57:16.244816+0000","last_undegraded":"2024-04-25T18:57:16.244816+0000","last_fullsized":"2024-04-25T18:57:16.244816+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:33.386859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.266075+0000","last_change":"2024-04-25T18:55:23.235709+0000","last_active":"2024-04-25T18:57:16.266075+0000","last_peered":"2024-04-25T18:57:16.266075+0000","last_clean":"2024-04-25T18:57:16.266075+0000","last_became_active":"2024-04-25T18:55:23.226089+0000","last_became_peered":"2024-04-25T18:55:23.226089+0000","last_unstale":"2024-04-25T18:57:16.266075+0000","last_undegraded":"2024-04-25T18:57:16.266075+0000","last_fullsized":"2024-04-25T18:57:16.266075+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:57.161365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":17,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.267325+0000","last_change":"2024-04-25T18:56:17.561546+0000","last_active":"2024-04-25T18:57:16.267325+0000","last_peered":"2024-04-25T18:57:16.267325+0000","last_clean":"2024-04-25T18:57:16.267325+0000","last_became_active":"2024-04-25T18:56:17.559767+0000","last_became_peered":"2024-04-25T18:56:17.559767+0000","last_unstale":"2024-04-25T18:57:16.267325+0000","last_undegraded":"2024-04-25T18:57:16.267325+0000","last_fullsized":"2024-04-25T18:57:16.267325+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:59.512008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.275457+0000","last_change":"2024-04-25T18:55:23.227620+0000","last_active":"2024-04-25T18:57:16.275457+0000","last_peered":"2024-04-25T18:57:16.275457+0000","last_clean":"2024-04-25T18:57:16.275457+0000","last_became_active":"2024-04-25T18:55:23.227236+0000","last_became_peered":"2024-04-25T18:55:23.227236+0000","last_unstale":"2024-04-25T18:57:16.275457+0000","last_undegraded":"2024-04-25T18:57:16.275457+0000","last_fullsized":"2024-04-25T18:57:16.275457+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:32.707619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.276796+0000","last_change":"2024-04-25T18:55:23.238393+0000","last_active":"2024-04-25T18:57:16.276796+0000","last_peered":"2024-04-25T18:57:16.276796+0000","last_clean":"2024-04-25T18:57:16.276796+0000","last_became_active":"2024-04-25T18:55:23.237726+0000","last_became_peered":"2024-04-25T18:55:23.237726+0000","last_unstale":"2024-04-25T18:57:16.276796+0000","last_undegraded":"2024-04-25T18:57:16.276796+0000","last_fullsized":"2024-04-25T18:57:16.276796+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:48.543014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.268811+0000","last_change":"2024-04-25T18:55:23.231374+0000","last_active":"2024-04-25T18:57:16.268811+0000","last_peered":"2024-04-25T18:57:16.268811+0000","last_clean":"2024-04-25T18:57:16.268811+0000","last_became_active":"2024-04-25T18:55:23.230976+0000","last_became_peered":"2024-04-25T18:55:23.230976+0000","last_unstale":"2024-04-25T18:57:16.268811+0000","last_undegraded":"2024-04-25T18:57:16.268811+0000","last_fullsized":"2024-04-25T18:57:16.268811+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:06.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T18:57:16.239368+0000","last_change":"2024-04-25T18:55:23.217332+0000","last_active":"2024-04-25T18:57:16.239368+0000","last_peered":"2024-04-25T18:57:16.239368+0000","last_clean":"2024-04-25T18:57:16.239368+0000","last_became_active":"2024-04-25T18:55:23.216968+0000","last_became_peered":"2024-04-25T18:55:23.216968+0000","last_unstale":"2024-04-25T18:57:16.239368+0000","last_undegraded":"2024-04-25T18:57:16.239368+0000","last_fullsized":"2024-04-25T18:57:16.239368+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:31.000083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69647,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":733,"num_read_kb":438,"num_write":178,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":208941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252910,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":498791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514548,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37024,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708128,"statfs":{"total":95995035648,"available":95957123072,"internally_reserved":0,"allocated":1728512,"data_stored":1155743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710775,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":498993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972415,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":498913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234053,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36936,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708216,"statfs":{"total":95995035648,"available":95957213184,"internally_reserved":0,"allocated":1638400,"data_stored":1091516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430280,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":498382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1097728,"data_stored":563148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":499653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":564996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117081,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1048576,"data_stored":498693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1036288,"data_stored":499351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691919,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1650688,"data_stored":1090668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":67083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:57:48.817 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T18:57:48.817 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T18:57:48.817 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T18:57:48.817 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T18:57:49.311 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:49.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:49 smithi079 ceph-mon[33002]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:57:49.460 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:49 smithi045 ceph-mon[27776]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:57:49.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:49 smithi062 ceph-mon[32844]: from='client.24835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:57:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:50 smithi079 ceph-mon[33002]: pgmap v400: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:50.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:50 smithi045 ceph-mon[27776]: pgmap v400: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:50.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:50 smithi062 ceph-mon[32844]: pgmap v400: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:51.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.561+0000 7f7a32600700 1 -- 172.21.15.45:0/616745568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a2c109c80 msgr2=0x7f7a2c10dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:51.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.561+0000 7f7a32600700 1 --2- 172.21.15.45:0/616745568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a2c109c80 0x7f7a2c10dcd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7a140099c0 tx=0x7f7a14009cd0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.561+0000 7f7a32600700 1 -- 172.21.15.45:0/616745568 shutdown_connections 2024-04-25T18:57:51.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.561+0000 7f7a32600700 1 --2- 172.21.15.45:0/616745568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2c1092d0 0x7f7a2c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.561+0000 7f7a32600700 1 --2- 172.21.15.45:0/616745568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c10e3b0 0x7f7a2c115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.561+0000 7f7a32600700 1 --2- 172.21.15.45:0/616745568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a2c109c80 0x7f7a2c10dcd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.561+0000 7f7a32600700 1 -- 172.21.15.45:0/616745568 >> 172.21.15.45:0/616745568 conn(0x7f7a2c076210 msgr2=0x7f7a2c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:51.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.561+0000 7f7a32600700 1 -- 172.21.15.45:0/616745568 shutdown_connections 2024-04-25T18:57:51.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.561+0000 7f7a32600700 1 -- 172.21.15.45:0/616745568 wait complete. 2024-04-25T18:57:51.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.562+0000 7f7a32600700 1 Processor -- start 2024-04-25T18:57:51.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a32600700 1 -- start start 2024-04-25T18:57:51.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a32600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a2c1092d0 0x7f7a2c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:51.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a32600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c109c80 0x7f7a2c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:51.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a32600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2c10e3b0 0x7f7a2c1a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:51.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a32600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c119270 con 0x7f7a2c109c80 2024-04-25T18:57:51.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a32600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1190f0 con 0x7f7a2c10e3b0 2024-04-25T18:57:51.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a32600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c1193f0 con 0x7f7a2c1092d0 2024-04-25T18:57:51.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a2b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a2c1092d0 0x7f7a2c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:51.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a23e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c109c80 0x7f7a2c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:51.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a2b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a2c1092d0 0x7f7a2c19f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:51816/0 (socket says 172.21.15.45:51816) 2024-04-25T18:57:51.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a2b400700 1 -- 172.21.15.45:0/3909603244 learned_addr learned my addr 172.21.15.45:0/3909603244 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:51.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.563+0000 7f7a2be00700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2c10e3b0 0x7f7a2c1a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:51.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.564+0000 7f7a2b400700 1 -- 172.21.15.45:0/3909603244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2c10e3b0 msgr2=0x7f7a2c1a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:51.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.564+0000 7f7a2b400700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2c10e3b0 0x7f7a2c1a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.564+0000 7f7a2b400700 1 -- 172.21.15.45:0/3909603244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c109c80 msgr2=0x7f7a2c19fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:51.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.564+0000 7f7a2b400700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c109c80 0x7f7a2c19fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.567 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.564+0000 7f7a2b400700 1 -- 172.21.15.45:0/3909603244 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c1a46e0 con 0x7f7a2c1092d0 2024-04-25T18:57:51.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.564+0000 7f7a2b400700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a2c1092d0 0x7f7a2c19f720 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00e9a0 tx=0x7f7a1c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:51.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.564+0000 7f7a28c00700 1 -- 172.21.15.45:0/3909603244 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c00ca10 con 0x7f7a2c1092d0 2024-04-25T18:57:51.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.565+0000 7f7a28c00700 1 -- 172.21.15.45:0/3909603244 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a1c00cbb0 con 0x7f7a2c1092d0 2024-04-25T18:57:51.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.565+0000 7f7a32600700 1 -- 172.21.15.45:0/3909603244 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c1a49d0 con 0x7f7a2c1092d0 2024-04-25T18:57:51.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.565+0000 7f7a28c00700 1 -- 172.21.15.45:0/3909603244 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c018680 con 0x7f7a2c1092d0 2024-04-25T18:57:51.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.565+0000 7f7a32600700 1 -- 172.21.15.45:0/3909603244 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c1ac250 con 0x7f7a2c1092d0 2024-04-25T18:57:51.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.566+0000 7f7a32600700 1 -- 172.21.15.45:0/3909603244 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c069730 con 0x7f7a2c1092d0 2024-04-25T18:57:51.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.568+0000 7f7a28c00700 1 -- 172.21.15.45:0/3909603244 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7a1c010430 con 0x7f7a2c1092d0 2024-04-25T18:57:51.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.569+0000 7f7a28c00700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a100a3cc0 0x7f7a100a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:51.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.569+0000 7f7a28c00700 1 -- 172.21.15.45:0/3909603244 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f7a1c0d05b0 con 0x7f7a2c1092d0 2024-04-25T18:57:51.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.570+0000 7f7a23e00700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a100a3cc0 0x7f7a100a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:51.572 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.570+0000 7f7a23e00700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a100a3cc0 0x7f7a100a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7a14000c00 tx=0x7f7a14005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:51.576 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.574+0000 7f7a28c00700 1 -- 172.21.15.45:0/3909603244 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a1c09cf10 con 0x7f7a2c1092d0 2024-04-25T18:57:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.855+0000 7f7a32600700 1 -- 172.21.15.45:0/3909603244 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7a2c02df80 con 0x7f7a2c1092d0 2024-04-25T18:57:51.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.857+0000 7f7a28c00700 1 -- 172.21.15.45:0/3909603244 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7a1c09c810 con 0x7f7a2c1092d0 2024-04-25T18:57:51.859 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:57:51.859 INFO:teuthology.orchestra.run.smithi045.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T18:57:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 -- 172.21.15.45:0/3909603244 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a100a3cc0 msgr2=0x7f7a100a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a100a3cc0 0x7f7a100a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7a14000c00 tx=0x7f7a14005fb0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 -- 172.21.15.45:0/3909603244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a2c1092d0 msgr2=0x7f7a2c19f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:51.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a2c1092d0 0x7f7a2c19f720 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00e9a0 tx=0x7f7a1c00ecb0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 -- 172.21.15.45:0/3909603244 shutdown_connections 2024-04-25T18:57:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7a100a3cc0 0x7f7a100a6180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7a2c10e3b0 0x7f7a2c1a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7a2c1092d0 0x7f7a2c19f720 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 --2- 172.21.15.45:0/3909603244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7a2c109c80 0x7f7a2c19fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 -- 172.21.15.45:0/3909603244 >> 172.21.15.45:0/3909603244 conn(0x7f7a2c076210 msgr2=0x7f7a2c077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 -- 172.21.15.45:0/3909603244 shutdown_connections 2024-04-25T18:57:51.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:51.861+0000 7f7a32600700 1 -- 172.21.15.45:0/3909603244 wait complete. 2024-04-25T18:57:52.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:52 smithi062 ceph-mon[32844]: pgmap v401: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:57:52.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:52 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3909603244' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T18:57:52.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:52 smithi079 ceph-mon[33002]: pgmap v401: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:57:52.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:52 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3909603244' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T18:57:52.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:52 smithi045 ceph-mon[27776]: pgmap v401: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:57:52.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:52 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3909603244' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T18:57:52.675 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T18:57:52.676 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:57:53.149 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:54.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:54 smithi079 ceph-mon[33002]: pgmap v402: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:54.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:54 smithi045 ceph-mon[27776]: pgmap v402: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:54.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:54 smithi062 ceph-mon[32844]: pgmap v402: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:55.392 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.390+0000 7fe578e00700 1 -- 172.21.15.45:0/2565630972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe574102ac0 msgr2=0x7fe574102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:55.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.390+0000 7fe578e00700 1 --2- 172.21.15.45:0/2565630972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe574102ac0 0x7fe574102f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009990 tx=0x7fe55c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.390+0000 7fe578e00700 1 -- 172.21.15.45:0/2565630972 shutdown_connections 2024-04-25T18:57:55.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.390+0000 7fe578e00700 1 --2- 172.21.15.45:0/2565630972 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe574103460 0x7fe57410f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.390+0000 7fe578e00700 1 --2- 172.21.15.45:0/2565630972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574108a70 0x7fe574108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.390+0000 7fe578e00700 1 --2- 172.21.15.45:0/2565630972 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe574102ac0 0x7fe574102f20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.390+0000 7fe578e00700 1 -- 172.21.15.45:0/2565630972 >> 172.21.15.45:0/2565630972 conn(0x7fe574079b10 msgr2=0x7fe574079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:55.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.390+0000 7fe578e00700 1 -- 172.21.15.45:0/2565630972 shutdown_connections 2024-04-25T18:57:55.393 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.390+0000 7fe578e00700 1 -- 172.21.15.45:0/2565630972 wait complete. 2024-04-25T18:57:55.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.391+0000 7fe578e00700 1 Processor -- start 2024-04-25T18:57:55.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.391+0000 7fe578e00700 1 -- start start 2024-04-25T18:57:55.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe578e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe574102ac0 0x7fe5741990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:55.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe578e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe574103460 0x7fe574199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe578e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574108a70 0x7fe57419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe578e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe574112c10 con 0x7fe574108a70 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe578e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe574112a90 con 0x7fe574103460 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe578e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe574112d90 con 0x7fe574102ac0 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe572800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574108a70 0x7fe57419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe571e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe574102ac0 0x7fe5741990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe56be00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe574103460 0x7fe574199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe571e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe574102ac0 0x7fe5741990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:36110/0 (socket says 172.21.15.45:36110) 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe571e00700 1 -- 172.21.15.45:0/1498909965 learned_addr learned my addr 172.21.15.45:0/1498909965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.392+0000 7fe572800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574108a70 0x7fe57419d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56934/0 (socket says 172.21.15.45:56934) 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe572800700 1 -- 172.21.15.45:0/1498909965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe574102ac0 msgr2=0x7fe5741990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe572800700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe574102ac0 0x7fe5741990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe572800700 1 -- 172.21.15.45:0/1498909965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe574103460 msgr2=0x7fe574199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe572800700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe574103460 0x7fe574199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe572800700 1 -- 172.21.15.45:0/1498909965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe57419e0d0 con 0x7fe574108a70 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe571e00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe574102ac0 0x7fe5741990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe56be00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe574103460 0x7fe574199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe572800700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574108a70 0x7fe57419d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00efc0 tx=0x7fe56c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe56aa00700 1 -- 172.21.15.45:0/1498909965 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c0092e0 con 0x7fe574108a70 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe578e00700 1 -- 172.21.15.45:0/1498909965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe57419e360 con 0x7fe574108a70 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe56aa00700 1 -- 172.21.15.45:0/1498909965 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe56c004bb0 con 0x7fe574108a70 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.393+0000 7fe56aa00700 1 -- 172.21.15.45:0/1498909965 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c004020 con 0x7fe574108a70 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.394+0000 7fe578e00700 1 -- 172.21.15.45:0/1498909965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe574074c70 con 0x7fe574108a70 2024-04-25T18:57:55.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.395+0000 7fe578e00700 1 -- 172.21.15.45:0/1498909965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5741041b0 con 0x7fe574108a70 2024-04-25T18:57:55.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.397+0000 7fe56aa00700 1 -- 172.21.15.45:0/1498909965 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe56c01b030 con 0x7fe574108a70 2024-04-25T18:57:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.399+0000 7fe56aa00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe5580a3c40 0x7fe5580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.399+0000 7fe56aa00700 1 -- 172.21.15.45:0/1498909965 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fe56c0cff40 con 0x7fe574108a70 2024-04-25T18:57:55.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.399+0000 7fe571e00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe5580a3c40 0x7fe5580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:55.402 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.400+0000 7fe571e00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe5580a3c40 0x7fe5580a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe564011550 tx=0x7fe564009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:55.406 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.404+0000 7fe56aa00700 1 -- 172.21.15.45:0/1498909965 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe56c022020 con 0x7fe574108a70 2024-04-25T18:57:55.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.690+0000 7fe578e00700 1 -- 172.21.15.45:0/1498909965 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe57402df80 con 0x7fe574108a70 2024-04-25T18:57:55.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.692+0000 7fe56aa00700 1 -- 172.21.15.45:0/1498909965 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fe56c010040 con 0x7fe574108a70 2024-04-25T18:57:55.695 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:57:55.695 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:55:50.062944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24512]},"id":1}]} 2024-04-25T18:57:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.695+0000 7fe578e00700 1 -- 172.21.15.45:0/1498909965 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe5580a3c40 msgr2=0x7fe5580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.695+0000 7fe578e00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe5580a3c40 0x7fe5580a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe564011550 tx=0x7fe564009260 comp rx=0 tx=0).stop 2024-04-25T18:57:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.695+0000 7fe578e00700 1 -- 172.21.15.45:0/1498909965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574108a70 msgr2=0x7fe57419d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:55.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.695+0000 7fe578e00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574108a70 0x7fe57419d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe56c00efc0 tx=0x7fe56c00c3a0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.695+0000 7fe578e00700 1 -- 172.21.15.45:0/1498909965 shutdown_connections 2024-04-25T18:57:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.695+0000 7fe578e00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe5580a3c40 0x7fe5580a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.695+0000 7fe578e00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe574108a70 0x7fe57419d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.695+0000 7fe578e00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe574102ac0 0x7fe5741990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.695+0000 7fe578e00700 1 --2- 172.21.15.45:0/1498909965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe574103460 0x7fe574199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.695+0000 7fe578e00700 1 -- 172.21.15.45:0/1498909965 >> 172.21.15.45:0/1498909965 conn(0x7fe574079b10 msgr2=0x7fe574100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:55.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.696+0000 7fe578e00700 1 -- 172.21.15.45:0/1498909965 shutdown_connections 2024-04-25T18:57:55.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:55.696+0000 7fe578e00700 1 -- 172.21.15.45:0/1498909965 wait complete. 2024-04-25T18:57:55.699 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-25T18:57:56.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:56 smithi045 ceph-mon[27776]: pgmap v403: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:56.401 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:56 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1498909965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:57:56.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:56 smithi079 ceph-mon[33002]: pgmap v403: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:56.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:56 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1498909965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:57:56.477 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:57:56.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:56 smithi062 ceph-mon[32844]: pgmap v403: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:57:56.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:56 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1498909965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:57:56.944 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:57:57.380 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:57 smithi045 ceph-mon[27776]: pgmap v404: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:57:57.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:57 smithi079 ceph-mon[33002]: pgmap v404: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:57:57.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:57 smithi062 ceph-mon[32844]: pgmap v404: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:57:59.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.345+0000 7f3d42200700 1 -- 172.21.15.45:0/3276102062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c06a750 msgr2=0x7f3d3c06ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:59.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.345+0000 7f3d42200700 1 --2- 172.21.15.45:0/3276102062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c06a750 0x7f3d3c06ab30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009990 tx=0x7f3d2c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:57:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.345+0000 7f3d42200700 1 -- 172.21.15.45:0/3276102062 shutdown_connections 2024-04-25T18:57:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.345+0000 7f3d42200700 1 --2- 172.21.15.45:0/3276102062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c10f4f0 0x7f3d3c1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.345+0000 7f3d42200700 1 --2- 172.21.15.45:0/3276102062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c06a750 0x7f3d3c06ab30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.345+0000 7f3d42200700 1 --2- 172.21.15.45:0/3276102062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d3c06b070 0x7f3d3c10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.345+0000 7f3d42200700 1 -- 172.21.15.45:0/3276102062 >> 172.21.15.45:0/3276102062 conn(0x7f3d3c0788f0 msgr2=0x7f3d3c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.345+0000 7f3d42200700 1 -- 172.21.15.45:0/3276102062 shutdown_connections 2024-04-25T18:57:59.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.345+0000 7f3d42200700 1 -- 172.21.15.45:0/3276102062 wait complete. 2024-04-25T18:57:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.346+0000 7f3d42200700 1 Processor -- start 2024-04-25T18:57:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.346+0000 7f3d42200700 1 -- start start 2024-04-25T18:57:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.346+0000 7f3d42200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d3c06a750 0x7f3d3c19b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d42200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c06b070 0x7f3d3c19b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d42200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c10f4f0 0x7f3d3c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d42200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c114e50 con 0x7f3d3c06a750 2024-04-25T18:57:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d42200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c114cd0 con 0x7f3d3c06b070 2024-04-25T18:57:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d42200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c114fd0 con 0x7f3d3c10f4f0 2024-04-25T18:57:59.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d3b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d3c06a750 0x7f3d3c19b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:59.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d32a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c06b070 0x7f3d3c19b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:59.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d3b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d3c06a750 0x7f3d3c19b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56956/0 (socket says 172.21.15.45:56956) 2024-04-25T18:57:59.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d32a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c06b070 0x7f3d3c19b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:44068/0 (socket says 172.21.15.45:44068) 2024-04-25T18:57:59.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d3b400700 1 -- 172.21.15.45:0/1792089613 learned_addr learned my addr 172.21.15.45:0/1792089613 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:57:59.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d3be00700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c10f4f0 0x7f3d3c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d32a00700 1 -- 172.21.15.45:0/1792089613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c10f4f0 msgr2=0x7f3d3c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d32a00700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c10f4f0 0x7f3d3c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d32a00700 1 -- 172.21.15.45:0/1792089613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d3c06a750 msgr2=0x7f3d3c19b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d32a00700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d3c06a750 0x7f3d3c19b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.347+0000 7f3d32a00700 1 -- 172.21.15.45:0/1792089613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c1a0360 con 0x7f3d3c06b070 2024-04-25T18:57:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.348+0000 7f3d3be00700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c10f4f0 0x7f3d3c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:57:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.348+0000 7f3d32a00700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c06b070 0x7f3d3c19b8e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400e9f0 tx=0x7f3d2400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.348+0000 7f3d38c00700 1 -- 172.21.15.45:0/1792089613 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2400cb70 con 0x7f3d3c06b070 2024-04-25T18:57:59.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.348+0000 7f3d42200700 1 -- 172.21.15.45:0/1792089613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c1a0650 con 0x7f3d3c06b070 2024-04-25T18:57:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.348+0000 7f3d38c00700 1 -- 172.21.15.45:0/1792089613 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d2400ee90 con 0x7f3d3c06b070 2024-04-25T18:57:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.348+0000 7f3d38c00700 1 -- 172.21.15.45:0/1792089613 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24003d40 con 0x7f3d3c06b070 2024-04-25T18:57:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.348+0000 7f3d42200700 1 -- 172.21.15.45:0/1792089613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c1a7ed0 con 0x7f3d3c06b070 2024-04-25T18:57:59.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.350+0000 7f3d42200700 1 -- 172.21.15.45:0/1792089613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d3c069730 con 0x7f3d3c06b070 2024-04-25T18:57:59.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.352+0000 7f3d38c00700 1 -- 172.21.15.45:0/1792089613 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3d24003ee0 con 0x7f3d3c06b070 2024-04-25T18:57:59.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.353+0000 7f3d38c00700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3d200a3c40 0x7f3d200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:57:59.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.353+0000 7f3d38c00700 1 -- 172.21.15.45:0/1792089613 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f3d240d0570 con 0x7f3d3c06b070 2024-04-25T18:57:59.355 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.353+0000 7f3d3b400700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3d200a3c40 0x7f3d200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:57:59.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.354+0000 7f3d3b400700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3d200a3c40 0x7f3d200a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c006010 tx=0x7f3d2c00b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:57:59.359 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.357+0000 7f3d38c00700 1 -- 172.21.15.45:0/1792089613 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d2409ced0 con 0x7f3d3c06b070 2024-04-25T18:57:59.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:57:59 smithi045 ceph-mon[27776]: pgmap v405: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:59.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:57:59 smithi062 ceph-mon[32844]: pgmap v405: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:57:59.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.642+0000 7f3d42200700 1 -- 172.21.15.45:0/1792089613 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d3c02df80 con 0x7f3d3c06b070 2024-04-25T18:57:59.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.644+0000 7f3d38c00700 1 -- 172.21.15.45:0/1792089613 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7f3d2409c7d0 con 0x7f3d3c06b070 2024-04-25T18:57:59.647 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:57:59.647 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:55:50.062944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24512]},"id":1}]} 2024-04-25T18:57:59.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 -- 172.21.15.45:0/1792089613 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3d200a3c40 msgr2=0x7f3d200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3d200a3c40 0x7f3d200a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c006010 tx=0x7f3d2c00b280 comp rx=0 tx=0).stop 2024-04-25T18:57:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 -- 172.21.15.45:0/1792089613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c06b070 msgr2=0x7f3d3c19b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:57:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c06b070 0x7f3d3c19b8e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3d2400e9f0 tx=0x7f3d2400ed00 comp rx=0 tx=0).stop 2024-04-25T18:57:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 -- 172.21.15.45:0/1792089613 shutdown_connections 2024-04-25T18:57:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f3d200a3c40 0x7f3d200a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:59.650 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3d3c10f4f0 0x7f3d3c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:59.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3d3c06a750 0x7f3d3c19b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:59.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 --2- 172.21.15.45:0/1792089613 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3d3c06b070 0x7f3d3c19b8e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:57:59.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 -- 172.21.15.45:0/1792089613 >> 172.21.15.45:0/1792089613 conn(0x7f3d3c0788f0 msgr2=0x7f3d3c1109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:57:59.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 -- 172.21.15.45:0/1792089613 shutdown_connections 2024-04-25T18:57:59.651 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:57:59.647+0000 7f3d42200700 1 -- 172.21.15.45:0/1792089613 wait complete. 2024-04-25T18:57:59.652 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-25T18:57:59.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:57:59 smithi079 ceph-mon[33002]: pgmap v405: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:00.546 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:58:00.630 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:00 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1792089613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:00.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:00 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1792089613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:00.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:00 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1792089613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:01.023 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:01.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:01 smithi045 ceph-mon[27776]: pgmap v406: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:01.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:01 smithi079 ceph-mon[33002]: pgmap v406: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:01.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:01 smithi062 ceph-mon[32844]: pgmap v406: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:03.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.373+0000 7f7af8a00700 1 -- 172.21.15.45:0/2040630440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7af4113920 msgr2=0x7f7af4115d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:03.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.373+0000 7f7af8a00700 1 --2- 172.21.15.45:0/2040630440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7af4113920 0x7f7af4115d10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b0a0 tx=0x7f7aec00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.373+0000 7f7af8a00700 1 -- 172.21.15.45:0/2040630440 shutdown_connections 2024-04-25T18:58:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.373+0000 7f7af8a00700 1 --2- 172.21.15.45:0/2040630440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7af4113920 0x7f7af4115d10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.373+0000 7f7af8a00700 1 --2- 172.21.15.45:0/2040630440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7af4076f70 0x7f7af4077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.373+0000 7f7af8a00700 1 --2- 172.21.15.45:0/2040630440 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4077890 0x7f7af41133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.373+0000 7f7af8a00700 1 -- 172.21.15.45:0/2040630440 >> 172.21.15.45:0/2040630440 conn(0x7f7af4100770 msgr2=0x7f7af4102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.374+0000 7f7af8a00700 1 -- 172.21.15.45:0/2040630440 shutdown_connections 2024-04-25T18:58:03.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.374+0000 7f7af8a00700 1 -- 172.21.15.45:0/2040630440 wait complete. 2024-04-25T18:58:03.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.375+0000 7f7af8a00700 1 Processor -- start 2024-04-25T18:58:03.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.375+0000 7f7af8a00700 1 -- start start 2024-04-25T18:58:03.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.375+0000 7f7af8a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7af4076f70 0x7f7af41a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:03.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.375+0000 7f7af8a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7af4077890 0x7f7af41a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:03.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.375+0000 7f7af8a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4113920 0x7f7af419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:03.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.375+0000 7f7af8a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41192a0 con 0x7f7af4076f70 2024-04-25T18:58:03.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.375+0000 7f7af8a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4119120 con 0x7f7af4113920 2024-04-25T18:58:03.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.375+0000 7f7af8a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4119420 con 0x7f7af4077890 2024-04-25T18:58:03.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af1400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7af4077890 0x7f7af41a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:03.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7af4076f70 0x7f7af41a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:03.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af1400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7af4077890 0x7f7af41a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:36148/0 (socket says 172.21.15.45:36148) 2024-04-25T18:58:03.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af1400700 1 -- 172.21.15.45:0/1735070518 learned_addr learned my addr 172.21.15.45:0/1735070518 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:03.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af2800700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4113920 0x7f7af419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:03.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af1400700 1 -- 172.21.15.45:0/1735070518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4113920 msgr2=0x7f7af419f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:03.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af1400700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4113920 0x7f7af419f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af1400700 1 -- 172.21.15.45:0/1735070518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7af4076f70 msgr2=0x7f7af41a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:03.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af1400700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7af4076f70 0x7f7af41a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af1400700 1 -- 172.21.15.45:0/1735070518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af419fd40 con 0x7f7af4077890 2024-04-25T18:58:03.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.376+0000 7f7af2800700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4113920 0x7f7af419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:03.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.377+0000 7f7af1400700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7af4077890 0x7f7af41a5910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000e9a0 tx=0x7f7ae000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:03.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.379+0000 7f7aeaa00700 1 -- 172.21.15.45:0/1735070518 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae000ca10 con 0x7f7af4077890 2024-04-25T18:58:03.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.379+0000 7f7aeaa00700 1 -- 172.21.15.45:0/1735070518 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ae000cbb0 con 0x7f7af4077890 2024-04-25T18:58:03.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.379+0000 7f7af8a00700 1 -- 172.21.15.45:0/1735070518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af41a0030 con 0x7f7af4077890 2024-04-25T18:58:03.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.379+0000 7f7aeaa00700 1 -- 172.21.15.45:0/1735070518 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0018680 con 0x7f7af4077890 2024-04-25T18:58:03.381 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.379+0000 7f7af8a00700 1 -- 172.21.15.45:0/1735070518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af41ac2a0 con 0x7f7af4077890 2024-04-25T18:58:03.382 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.380+0000 7f7af8a00700 1 -- 172.21.15.45:0/1735070518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af4069730 con 0x7f7af4077890 2024-04-25T18:58:03.384 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.382+0000 7f7aeaa00700 1 -- 172.21.15.45:0/1735070518 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ae0010430 con 0x7f7af4077890 2024-04-25T18:58:03.385 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.383+0000 7f7aeaa00700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ad80a3cc0 0x7f7ad80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:03.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.384+0000 7f7aeaa00700 1 -- 172.21.15.45:0/1735070518 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f7ae00d0510 con 0x7f7af4077890 2024-04-25T18:58:03.386 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.384+0000 7f7af1e00700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ad80a3cc0 0x7f7ad80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:03.387 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.384+0000 7f7af1e00700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ad80a3cc0 0x7f7ad80a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009c60 tx=0x7f7ae4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:03.389 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.388+0000 7f7aeaa00700 1 -- 172.21.15.45:0/1735070518 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ae009ce70 con 0x7f7af4077890 2024-04-25T18:58:03.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:03 smithi045 ceph-mon[27776]: pgmap v407: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:03.626 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.624+0000 7f7af8a00700 1 -- 172.21.15.45:0/1735070518 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7af402df80 con 0x7f7af4077890 2024-04-25T18:58:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.627+0000 7f7aeaa00700 1 -- 172.21.15.45:0/1735070518 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+22654 (secure 0 0 0) 0x7f7ae009c770 con 0x7f7af4077890 2024-04-25T18:58:03.629 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:58:03.629 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T18:56:17.531513+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:55:17.591864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:55:22.070218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:58:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 -- 172.21.15.45:0/1735070518 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ad80a3cc0 msgr2=0x7f7ad80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ad80a3cc0 0x7f7ad80a6180 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009c60 tx=0x7f7ae4009260 comp rx=0 tx=0).stop 2024-04-25T18:58:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 -- 172.21.15.45:0/1735070518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7af4077890 msgr2=0x7f7af41a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:03.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7af4077890 0x7f7af41a5910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7ae000e9a0 tx=0x7f7ae000ecb0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 -- 172.21.15.45:0/1735070518 shutdown_connections 2024-04-25T18:58:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f7ad80a3cc0 0x7f7ad80a6180 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7af4113920 0x7f7af419f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7af4076f70 0x7f7af41a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 --2- 172.21.15.45:0/1735070518 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7af4077890 0x7f7af41a5910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 -- 172.21.15.45:0/1735070518 >> 172.21.15.45:0/1735070518 conn(0x7f7af4100770 msgr2=0x7f7af4102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 -- 172.21.15.45:0/1735070518 shutdown_connections 2024-04-25T18:58:03.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:03.629+0000 7f7af8a00700 1 -- 172.21.15.45:0/1735070518 wait complete. 2024-04-25T18:58:03.686 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:03 smithi079 ceph-mon[33002]: pgmap v407: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:03.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:03 smithi062 ceph-mon[32844]: pgmap v407: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:04.505 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:58:04.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:04 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1735070518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:04.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:04 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1735070518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:04.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:04 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1735070518' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:04.961 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:05.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:05 smithi079 ceph-mon[33002]: pgmap v408: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:05.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:05 smithi045 ceph-mon[27776]: pgmap v408: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:05.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:05 smithi062 ceph-mon[32844]: pgmap v408: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.263+0000 7fa63d800700 1 -- 172.21.15.45:0/1985629842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa638109fc0 msgr2=0x7fa638111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.263+0000 7fa63d800700 1 --2- 172.21.15.45:0/1985629842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa638109fc0 0x7fa638111b90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa63000b0d0 tx=0x7fa63000b3e0 comp rx=0 tx=0).stop 2024-04-25T18:58:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.263+0000 7fa63d800700 1 -- 172.21.15.45:0/1985629842 shutdown_connections 2024-04-25T18:58:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.263+0000 7fa63d800700 1 --2- 172.21.15.45:0/1985629842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa638109fc0 0x7fa638111b90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.263+0000 7fa63d800700 1 --2- 172.21.15.45:0/1985629842 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa638104ee0 0x7fa6381052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.263+0000 7fa63d800700 1 --2- 172.21.15.45:0/1985629842 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638105890 0x7fa6381098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.263+0000 7fa63d800700 1 -- 172.21.15.45:0/1985629842 >> 172.21.15.45:0/1985629842 conn(0x7fa638100770 msgr2=0x7fa638102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.263+0000 7fa63d800700 1 -- 172.21.15.45:0/1985629842 shutdown_connections 2024-04-25T18:58:07.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.263+0000 7fa63d800700 1 -- 172.21.15.45:0/1985629842 wait complete. 2024-04-25T18:58:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.264+0000 7fa63d800700 1 Processor -- start 2024-04-25T18:58:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.264+0000 7fa63d800700 1 -- start start 2024-04-25T18:58:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa63d800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa638104ee0 0x7fa63819b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa63d800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa638105890 0x7fa63819b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa63d800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638109fc0 0x7fa63819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa63d800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa638114e80 con 0x7fa638105890 2024-04-25T18:58:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa63d800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa638114d00 con 0x7fa638109fc0 2024-04-25T18:58:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa63d800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa638115000 con 0x7fa638104ee0 2024-04-25T18:58:07.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa636000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa638105890 0x7fa63819b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa636a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa638104ee0 0x7fa63819b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa636000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa638105890 0x7fa63819b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52090/0 (socket says 172.21.15.45:52090) 2024-04-25T18:58:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa636000700 1 -- 172.21.15.45:0/3654187669 learned_addr learned my addr 172.21.15.45:0/3654187669 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.265+0000 7fa637400700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638109fc0 0x7fa63819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.266+0000 7fa636a00700 1 -- 172.21.15.45:0/3654187669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638109fc0 msgr2=0x7fa63819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.266+0000 7fa636a00700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638109fc0 0x7fa63819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.266+0000 7fa636a00700 1 -- 172.21.15.45:0/3654187669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa638105890 msgr2=0x7fa63819b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.266+0000 7fa636a00700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa638105890 0x7fa63819b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.266+0000 7fa636a00700 1 -- 172.21.15.45:0/3654187669 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6381a0320 con 0x7fa638104ee0 2024-04-25T18:58:07.268 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.266+0000 7fa636a00700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa638104ee0 0x7fa63819b310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa62800e9f0 tx=0x7fa62800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.266+0000 7fa62f400700 1 -- 172.21.15.45:0/3654187669 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62800cb70 con 0x7fa638104ee0 2024-04-25T18:58:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.266+0000 7fa62f400700 1 -- 172.21.15.45:0/3654187669 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62800ee90 con 0x7fa638104ee0 2024-04-25T18:58:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.266+0000 7fa62f400700 1 -- 172.21.15.45:0/3654187669 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa628003d40 con 0x7fa638104ee0 2024-04-25T18:58:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.267+0000 7fa63d800700 1 -- 172.21.15.45:0/3654187669 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6381a0610 con 0x7fa638104ee0 2024-04-25T18:58:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.267+0000 7fa63d800700 1 -- 172.21.15.45:0/3654187669 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6381a7e90 con 0x7fa638104ee0 2024-04-25T18:58:07.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.268+0000 7fa63d800700 1 -- 172.21.15.45:0/3654187669 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa638069730 con 0x7fa638104ee0 2024-04-25T18:58:07.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.270+0000 7fa62f400700 1 -- 172.21.15.45:0/3654187669 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa628003ee0 con 0x7fa638104ee0 2024-04-25T18:58:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.271+0000 7fa62f400700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa61c0a3cc0 0x7fa61c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:07.272 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.272+0000 7fa62f400700 1 -- 172.21.15.45:0/3654187669 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa6280d06e0 con 0x7fa638104ee0 2024-04-25T18:58:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.272+0000 7fa636000700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa61c0a3cc0 0x7fa61c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:07.273 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.272+0000 7fa636000700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa61c0a3cc0 0x7fa61c0a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa638103ef0 tx=0x7fa624009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:07.274 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.273+0000 7fa62f400700 1 -- 172.21.15.45:0/3654187669 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa628053ff0 con 0x7fa638104ee0 2024-04-25T18:58:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.553+0000 7fa63d800700 1 -- 172.21.15.45:0/3654187669 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa63802df80 con 0x7fa638104ee0 2024-04-25T18:58:07.555 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:07 smithi045 ceph-mon[27776]: pgmap v409: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:58:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.554+0000 7fa62f400700 1 -- 172.21.15.45:0/3654187669 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fa62809d040 con 0x7fa638104ee0 2024-04-25T18:58:07.556 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:58:07.556 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:55:50.062944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24512]},"id":1}]} 2024-04-25T18:58:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.557+0000 7fa63d800700 1 -- 172.21.15.45:0/3654187669 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa61c0a3cc0 msgr2=0x7fa61c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.557+0000 7fa63d800700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa61c0a3cc0 0x7fa61c0a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa638103ef0 tx=0x7fa624009260 comp rx=0 tx=0).stop 2024-04-25T18:58:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.557+0000 7fa63d800700 1 -- 172.21.15.45:0/3654187669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa638104ee0 msgr2=0x7fa63819b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.557+0000 7fa63d800700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa638104ee0 0x7fa63819b310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa62800e9f0 tx=0x7fa62800ed00 comp rx=0 tx=0).stop 2024-04-25T18:58:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.558+0000 7fa63d800700 1 -- 172.21.15.45:0/3654187669 shutdown_connections 2024-04-25T18:58:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.558+0000 7fa63d800700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa61c0a3cc0 0x7fa61c0a6180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.558+0000 7fa63d800700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa638109fc0 0x7fa63819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.558+0000 7fa63d800700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa638104ee0 0x7fa63819b310 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.558+0000 7fa63d800700 1 --2- 172.21.15.45:0/3654187669 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa638105890 0x7fa63819b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.558+0000 7fa63d800700 1 -- 172.21.15.45:0/3654187669 >> 172.21.15.45:0/3654187669 conn(0x7fa638100770 msgr2=0x7fa6381012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.558+0000 7fa63d800700 1 -- 172.21.15.45:0/3654187669 shutdown_connections 2024-04-25T18:58:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:07.559+0000 7fa63d800700 1 -- 172.21.15.45:0/3654187669 wait complete. 2024-04-25T18:58:07.561 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-25T18:58:07.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:07 smithi079 ceph-mon[33002]: pgmap v409: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:58:07.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:07 smithi062 ceph-mon[32844]: pgmap v409: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T18:58:08.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T18:55:26.025536+0000', 'modified': '2024-04-25T18:55:50.062944+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24512, 'mds_2': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6835/1201787807', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 1201787807}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 1201787807}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/2985890701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 2985890701}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 2985890701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6837/3290580293', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 3290580293}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 3290580293}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.79:6839/235958219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 235958219}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 235958219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.45:6841/2490735053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6840', 'nonce': 2490735053}, {'type': 'v1', 'addr': '172.21.15.45:6841', 'nonce': 2490735053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.62:6839/3288398937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 3288398937}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 3288398937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24524, 24505, 24512]} 2024-04-25T18:58:08.372 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T18:58:08.372 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T18:58:08.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:08 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3654187669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:08.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:08 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3654187669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:08.848 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:08.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:08 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3654187669' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:09.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:09 smithi079 ceph-mon[33002]: pgmap v410: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:09.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:09 smithi045 ceph-mon[27776]: pgmap v410: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:09.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:09 smithi062 ceph-mon[32844]: pgmap v410: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:11.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.073+0000 7f8874400700 1 -- 172.21.15.45:0/2751319798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f886c074c70 msgr2=0x7f886c075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:11.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.073+0000 7f8874400700 1 --2- 172.21.15.45:0/2751319798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f886c074c70 0x7f886c075050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8860009990 tx=0x7f8860009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.074+0000 7f8874400700 1 -- 172.21.15.45:0/2751319798 shutdown_connections 2024-04-25T18:58:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.074+0000 7f8874400700 1 --2- 172.21.15.45:0/2751319798 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c10f440 0x7f886c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.074+0000 7f8874400700 1 --2- 172.21.15.45:0/2751319798 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f886c074c70 0x7f886c075050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.074+0000 7f8874400700 1 --2- 172.21.15.45:0/2751319798 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f886c075590 0x7f886c10ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.074+0000 7f8874400700 1 -- 172.21.15.45:0/2751319798 >> 172.21.15.45:0/2751319798 conn(0x7f886c079ba0 msgr2=0x7f886c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.074+0000 7f8874400700 1 -- 172.21.15.45:0/2751319798 shutdown_connections 2024-04-25T18:58:11.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.074+0000 7f8874400700 1 -- 172.21.15.45:0/2751319798 wait complete. 2024-04-25T18:58:11.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.075+0000 7f8874400700 1 Processor -- start 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.075+0000 7f8874400700 1 -- start start 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8874400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f886c074c70 0x7f886c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8874400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f886c075590 0x7f886c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8874400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c10f440 0x7f886c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8874400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f886c114e70 con 0x7f886c074c70 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8874400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f886c114cf0 con 0x7f886c10f440 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8874400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f886c114ff0 con 0x7f886c075590 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8871a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f886c074c70 0x7f886c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8871a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f886c074c70 0x7f886c19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52106/0 (socket says 172.21.15.45:52106) 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8871a00700 1 -- 172.21.15.45:0/729454198 learned_addr learned my addr 172.21.15.45:0/729454198 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8872400700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c10f440 0x7f886c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8871000700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f886c075590 0x7f886c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8871a00700 1 -- 172.21.15.45:0/729454198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f886c075590 msgr2=0x7f886c19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8871a00700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f886c075590 0x7f886c19b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8871a00700 1 -- 172.21.15.45:0/729454198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c10f440 msgr2=0x7f886c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8871a00700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c10f440 0x7f886c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8871a00700 1 -- 172.21.15.45:0/729454198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886c1a03e0 con 0x7f886c074c70 2024-04-25T18:58:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.076+0000 7f8872400700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c10f440 0x7f886c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:11.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.077+0000 7f8871a00700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f886c074c70 0x7f886c19b420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8860000c00 tx=0x7f88600057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.077+0000 7f886a000700 1 -- 172.21.15.45:0/729454198 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860027070 con 0x7f886c074c70 2024-04-25T18:58:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.077+0000 7f886a000700 1 -- 172.21.15.45:0/729454198 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8860022470 con 0x7f886c074c70 2024-04-25T18:58:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.077+0000 7f8874400700 1 -- 172.21.15.45:0/729454198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886c1a0670 con 0x7f886c074c70 2024-04-25T18:58:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.077+0000 7f886a000700 1 -- 172.21.15.45:0/729454198 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8860017650 con 0x7f886c074c70 2024-04-25T18:58:11.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.077+0000 7f8874400700 1 -- 172.21.15.45:0/729454198 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f886c1a7f50 con 0x7f886c074c70 2024-04-25T18:58:11.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.081+0000 7f886a000700 1 -- 172.21.15.45:0/729454198 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f886002a070 con 0x7f886c074c70 2024-04-25T18:58:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.082+0000 7f886a000700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f88540a3cc0 0x7f88540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.082+0000 7f886a000700 1 -- 172.21.15.45:0/729454198 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f88600d1070 con 0x7f886c074c70 2024-04-25T18:58:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.082+0000 7f887edefb80 1 Processor -- start 2024-04-25T18:58:11.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.082+0000 7f8871000700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f88540a3cc0 0x7f88540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f887edefb80 1 -- start start 2024-04-25T18:58:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f887edefb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff3e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f887edefb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3ec10 0x559b4ff49570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f887edefb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff49ab0 0x559b4ff4bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f887edefb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559b4fe91e40 con 0x559b4ff3ec10 2024-04-25T18:58:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f887edefb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559b4fe386e0 con 0x559b4ff3e2f0 2024-04-25T18:58:11.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f887edefb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559b4fe8de00 con 0x559b4ff49ab0 2024-04-25T18:58:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f8871000700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f88540a3cc0 0x7f88540a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f886c111140 tx=0x7f8858009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f8871a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff3e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f8871000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3ec10 0x559b4ff49570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f8871a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff3e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36748/0 (socket says 172.21.15.45:36748) 2024-04-25T18:58:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f8871a00700 1 -- 172.21.15.45:0/1112477694 learned_addr learned my addr 172.21.15.45:0/1112477694 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.083+0000 7f8872400700 1 --2- 172.21.15.45:0/1112477694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff49ab0 0x559b4ff4bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.084+0000 7f8871a00700 1 -- 172.21.15.45:0/1112477694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff49ab0 msgr2=0x559b4ff4bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.084+0000 7f8871a00700 1 --2- 172.21.15.45:0/1112477694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff49ab0 0x559b4ff4bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.084+0000 7f8871a00700 1 -- 172.21.15.45:0/1112477694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3ec10 msgr2=0x559b4ff49570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.084+0000 7f8871a00700 1 --2- 172.21.15.45:0/1112477694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3ec10 0x559b4ff49570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.084+0000 7f8871a00700 1 -- 172.21.15.45:0/1112477694 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4fe2e630 con 0x559b4ff3e2f0 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.084+0000 7f8871a00700 1 --2- 172.21.15.45:0/1112477694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff3e6d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f88600a5a70 tx=0x7f8860009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ec7d4dba7b07cd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.084+0000 7f885ea00700 1 -- 172.21.15.45:0/1112477694 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886009d3c0 con 0x559b4ff3e2f0 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.084+0000 7f885ea00700 1 -- 172.21.15.45:0/1112477694 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88600a1770 con 0x559b4ff3e2f0 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.084+0000 7f885ea00700 1 -- 172.21.15.45:0/1112477694 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886009d820 con 0x559b4ff3e2f0 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.085+0000 7f887edefb80 1 -- 172.21.15.45:0/1112477694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 msgr2=0x559b4ff3e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.085+0000 7f887edefb80 1 --2- 172.21.15.45:0/1112477694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff3e6d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f88600a5a70 tx=0x7f8860009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.085+0000 7f887edefb80 1 -- 172.21.15.45:0/1112477694 shutdown_connections 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.085+0000 7f887edefb80 1 --2- 172.21.15.45:0/1112477694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff3e6d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.085+0000 7f887edefb80 1 --2- 172.21.15.45:0/1112477694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff49ab0 0x559b4ff4bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.085+0000 7f887edefb80 1 --2- 172.21.15.45:0/1112477694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3ec10 0x559b4ff49570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.085+0000 7f887edefb80 1 -- 172.21.15.45:0/1112477694 >> 172.21.15.45:0/1112477694 conn(0x559b4fe6d550 msgr2=0x559b4ff3a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:11.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.085+0000 7f887edefb80 1 -- 172.21.15.45:0/1112477694 shutdown_connections 2024-04-25T18:58:11.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.085+0000 7f887edefb80 1 -- 172.21.15.45:0/1112477694 wait complete. 2024-04-25T18:58:11.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.086+0000 7f887edefb80 1 -- start start 2024-04-25T18:58:11.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.086+0000 7f887edefb80 1 Processor -- start 2024-04-25T18:58:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f887edefb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff49210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f887edefb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 0x559b4ff46510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f887edefb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff49ab0 0x559b4ff46a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f887edefb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559b4ff4c3e0 con 0x559b4ff49ab0 2024-04-25T18:58:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f887edefb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559b4fe4b810 con 0x559b4ff3e2f0 2024-04-25T18:58:11.092 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f887edefb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559b4ff4c560 con 0x559b4ff3ec10 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f887edefb80 10 client.0 _ll_register_callbacks cb 0x559b4fe2a6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f8871a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff49210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f8872400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff49ab0 0x559b4ff46a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f8871a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff49210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36754/0 (socket says 172.21.15.45:36754) 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f8871a00700 1 -- 172.21.15.45:0/1929400119 learned_addr learned my addr 172.21.15.45:0/1929400119 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f8871000700 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 0x559b4ff46510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f8871a00700 1 -- 172.21.15.45:0/1929400119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 msgr2=0x559b4ff46510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f8871a00700 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 0x559b4ff46510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f8871a00700 1 -- 172.21.15.45:0/1929400119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff49ab0 msgr2=0x559b4ff46a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.087+0000 7f8871a00700 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff49ab0 0x559b4ff46a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.088+0000 7f8871a00700 1 -- 172.21.15.45:0/1929400119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4fddf880 con 0x559b4ff3e2f0 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.088+0000 7f8871a00700 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff49210 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f88600098e0 tx=0x7f8860005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1e4ca305f9bb520 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.088+0000 7f8853400700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.088+0000 7f8853400700 1 -- 172.21.15.45:0/1929400119 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88600ba020 con 0x559b4ff3e2f0 2024-04-25T18:58:11.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.088+0000 7f8853400700 1 -- 172.21.15.45:0/1929400119 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f886009d820 con 0x559b4ff3e2f0 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.088+0000 7f887edefb80 1 -- 172.21.15.45:0/1929400119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88600a1770 con 0x559b4ff3e2f0 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.088+0000 7f8853400700 1 -- 172.21.15.45:0/1929400119 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f88600bc2c0 con 0x559b4ff3e2f0 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.088+0000 7f887edefb80 10 client.24872 fetch_fsmap learned FSMap version 25 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.088+0000 7f887edefb80 1 -- 172.21.15.45:0/1929400119 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559b4fe28aa0 con 0x559b4ff3e2f0 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.089+0000 7f8853400700 1 -- 172.21.15.45:0/1929400119 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88600a1770 con 0x559b4ff3e2f0 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.089+0000 7f8853400700 1 -- 172.21.15.45:0/1929400119 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8860099110 con 0x559b4ff3e2f0 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.089+0000 7f887edefb80 10 client.24872 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.089+0000 7f887edefb80 10 client.24872 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.089+0000 7f887edefb80 20 client.24872 populate_metadata read hostname 'smithi045' 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.089+0000 7f887edefb80 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559b4ff58df0 0x559b4ff5a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.089+0000 7f887edefb80 4 client.24872 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.089+0000 7f887edefb80 1 -- 172.21.15.45:0/1929400119 --> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b4fe9f630 con 0x559b4ff58df0 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.090+0000 7f8871000700 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559b4ff58df0 0x559b4ff5a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.090+0000 7f8871000700 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559b4ff58df0 0x559b4ff5a260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=38096f5a86b63c70 server_cookie=53dc6d9509cc48b0 in_seq=0 out_seq=0 2024-04-25T18:58:11.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.090+0000 7f8853400700 10 client.24872 ms_handle_connect on v2:172.21.15.79:6838/235958219 2024-04-25T18:58:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.091+0000 7f8853400700 1 -- 172.21.15.45:0/1929400119 <== mds.2 v2:172.21.15.79:6838/235958219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b4fe9f630 con 0x559b4ff58df0 2024-04-25T18:58:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.091+0000 7f8853400700 10 client.24872 handle_command_reply: tid=0 2024-04-25T18:58:11.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.091+0000 7f887edefb80 1 client.24872 shutdown 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 1 -- 172.21.15.45:0/1929400119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 msgr2=0x559b4ff49210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff49210 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f88600098e0 tx=0x7f8860005ef0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 1 -- 172.21.15.45:0/1929400119 shutdown_connections 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559b4ff58df0 0x559b4ff5a260 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3e2f0 0x559b4ff49210 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff49ab0 0x559b4ff46a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 1 --2- 172.21.15.45:0/1929400119 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 0x559b4ff46510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f8853400700 0 client.24872 ms_handle_reset on v2:172.21.15.79:6838/235958219 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 1 -- 172.21.15.45:0/1929400119 >> 172.21.15.45:0/1929400119 conn(0x559b4fe6d550 msgr2=0x559b4ff39950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 1 -- 172.21.15.45:0/1929400119 shutdown_connections 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 1 -- 172.21.15.45:0/1929400119 wait complete. 2024-04-25T18:58:11.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.092+0000 7f887edefb80 20 client.24872 trim_cache size 0 max 16384 2024-04-25T18:58:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.115+0000 7f887edefb80 1 Processor -- start 2024-04-25T18:58:11.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.115+0000 7f887edefb80 1 -- start start 2024-04-25T18:58:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.115+0000 7f887edefb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff54f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.115+0000 7f887edefb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3ec10 0x559b4ff3a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.115+0000 7f887edefb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff49ab0 0x559b4ff3a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f887edefb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559b4ff585a0 con 0x559b4ff3e2f0 2024-04-25T18:58:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f887edefb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559b4ff58c70 con 0x559b4ff3ec10 2024-04-25T18:58:11.117 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f887edefb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559b4ff59340 con 0x559b4ff49ab0 2024-04-25T18:58:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8871a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff54f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8871a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff54f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52134/0 (socket says 172.21.15.45:52134) 2024-04-25T18:58:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8871a00700 1 -- 172.21.15.45:0/2681073012 learned_addr learned my addr 172.21.15.45:0/2681073012 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8871000700 1 --2- 172.21.15.45:0/2681073012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3ec10 0x559b4ff3a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8872400700 1 --2- 172.21.15.45:0/2681073012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff49ab0 0x559b4ff3a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8871000700 1 -- 172.21.15.45:0/2681073012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff49ab0 msgr2=0x559b4ff3a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8871000700 1 --2- 172.21.15.45:0/2681073012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff49ab0 0x559b4ff3a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8871000700 1 -- 172.21.15.45:0/2681073012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 msgr2=0x559b4ff54f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:11.119 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8871000700 1 --2- 172.21.15.45:0/2681073012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff54f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8871000700 1 -- 172.21.15.45:0/2681073012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4ff3fb60 con 0x559b4ff3ec10 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.116+0000 7f8871a00700 1 --2- 172.21.15.45:0/2681073012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff54f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.117+0000 7f8871000700 1 --2- 172.21.15.45:0/2681073012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3ec10 0x559b4ff3a0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x559b4ff5aac0 tx=0x7f8858014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e2ad058c0ff3c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.117+0000 7f885e000700 1 -- 172.21.15.45:0/2681073012 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885801db10 con 0x559b4ff3ec10 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.117+0000 7f885e000700 1 -- 172.21.15.45:0/2681073012 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8858004ce0 con 0x559b4ff3ec10 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.117+0000 7f885e000700 1 -- 172.21.15.45:0/2681073012 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858005550 con 0x559b4ff3ec10 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.117+0000 7f887edefb80 1 -- 172.21.15.45:0/2681073012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3ec10 msgr2=0x559b4ff3a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.117+0000 7f887edefb80 1 --2- 172.21.15.45:0/2681073012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3ec10 0x559b4ff3a0d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x559b4ff5aac0 tx=0x7f8858014000 comp rx=0 tx=0).stop 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.118+0000 7f887edefb80 1 -- 172.21.15.45:0/2681073012 shutdown_connections 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.118+0000 7f887edefb80 1 --2- 172.21.15.45:0/2681073012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff49ab0 0x559b4ff3a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.118+0000 7f887edefb80 1 --2- 172.21.15.45:0/2681073012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff54f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.118+0000 7f887edefb80 1 --2- 172.21.15.45:0/2681073012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff3ec10 0x559b4ff3a0d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.118+0000 7f887edefb80 1 -- 172.21.15.45:0/2681073012 >> 172.21.15.45:0/2681073012 conn(0x559b4fe6d550 msgr2=0x559b4ff485d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:11.120 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.118+0000 7f887edefb80 1 -- 172.21.15.45:0/2681073012 shutdown_connections 2024-04-25T18:58:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.118+0000 7f887edefb80 1 -- 172.21.15.45:0/2681073012 wait complete. 2024-04-25T18:58:11.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.118+0000 7f887edefb80 1 -- start start 2024-04-25T18:58:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.118+0000 7f887edefb80 1 Processor -- start 2024-04-25T18:58:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f887edefb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f887edefb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 0x559b4ff63c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f887edefb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff49ab0 0x559b4ff5c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f887edefb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559b4ff5a0e0 con 0x559b4ff3e2f0 2024-04-25T18:58:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f887edefb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559b4ff59a10 con 0x559b4ff49ab0 2024-04-25T18:58:11.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f887edefb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559b4ff5a7b0 con 0x559b4ff3ec10 2024-04-25T18:58:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f887edefb80 10 client.0 _ll_register_callbacks cb 0x559b4fe2a6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:58:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f8871a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f8871000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 0x559b4ff63c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f8871a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff636e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52136/0 (socket says 172.21.15.45:52136) 2024-04-25T18:58:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f8871a00700 1 -- 172.21.15.45:0/1608485967 learned_addr learned my addr 172.21.15.45:0/1608485967 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.119+0000 7f8872400700 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff49ab0 0x559b4ff5c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f8871000700 1 -- 172.21.15.45:0/1608485967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff49ab0 msgr2=0x559b4ff5c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f8871000700 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff49ab0 0x559b4ff5c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f8871000700 1 -- 172.21.15.45:0/1608485967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 msgr2=0x559b4ff636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f8871000700 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f8871000700 1 -- 172.21.15.45:0/1608485967 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4ff5cda0 con 0x559b4ff3ec10 2024-04-25T18:58:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f8871a00700 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f8871000700 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 0x559b4ff63c20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x559b4ff5ad90 tx=0x7f8858004f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=752188f784c9fe6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f8853400700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T18:58:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f8853400700 1 -- 172.21.15.45:0/1608485967 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885803a020 con 0x559b4ff3ec10 2024-04-25T18:58:11.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f887edefb80 1 -- 172.21.15.45:0/1608485967 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8858004ce0 con 0x559b4ff3ec10 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.120+0000 7f8853400700 1 -- 172.21.15.45:0/1608485967 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f8858009040 con 0x559b4ff3ec10 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f8853400700 1 -- 172.21.15.45:0/1608485967 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8858045050 con 0x559b4ff3ec10 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f887edefb80 10 client.24883 fetch_fsmap learned FSMap version 25 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f887edefb80 1 -- 172.21.15.45:0/1608485967 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559b4ffc0990 con 0x559b4ff3ec10 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f8853400700 1 -- 172.21.15.45:0/1608485967 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8858004ce0 con 0x559b4ff3ec10 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f8853400700 1 -- 172.21.15.45:0/1608485967 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f885802c170 con 0x559b4ff3ec10 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f887edefb80 10 client.24883 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f887edefb80 10 client.24883 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f887edefb80 20 client.24883 populate_metadata read hostname 'smithi045' 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f887edefb80 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559b4ffc53c0 0x559b4ffc40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f887edefb80 4 client.24883 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.121+0000 7f887edefb80 1 -- 172.21.15.45:0/1608485967 --> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] -- command(tid 0: {"prefix": "status"}) -- 0x559b4fe996d0 con 0x559b4ffc53c0 2024-04-25T18:58:11.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.122+0000 7f8871a00700 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559b4ffc53c0 0x559b4ffc40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.122+0000 7f8871a00700 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559b4ffc53c0 0x559b4ffc40d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f555c38ecde08220 server_cookie=eadb447b9770c13d in_seq=0 out_seq=0 2024-04-25T18:58:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.122+0000 7f8853400700 10 client.24883 ms_handle_connect on v2:172.21.15.79:6838/235958219 2024-04-25T18:58:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.122+0000 7f8853400700 1 -- 172.21.15.45:0/1608485967 <== mds.2 v2:172.21.15.79:6838/235958219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559b4fe996d0 con 0x559b4ffc53c0 2024-04-25T18:58:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.122+0000 7f8853400700 10 client.24883 handle_command_reply: tid=0 2024-04-25T18:58:11.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.122+0000 7f887edefb80 1 client.24883 shutdown 2024-04-25T18:58:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.122+0000 7f887edefb80 1 -- 172.21.15.45:0/1608485967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 msgr2=0x559b4ff63c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.122+0000 7f887edefb80 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 0x559b4ff63c20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x559b4ff5ad90 tx=0x7f8858004f20 comp rx=0 tx=0).stop 2024-04-25T18:58:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.123+0000 7f887edefb80 1 -- 172.21.15.45:0/1608485967 shutdown_connections 2024-04-25T18:58:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.123+0000 7f887edefb80 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559b4ffc53c0 0x559b4ffc40d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.123+0000 7f887edefb80 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559b4ff49ab0 0x559b4ff5c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.123+0000 7f887edefb80 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559b4ff3e2f0 0x559b4ff636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.123+0000 7f8853400700 0 client.24883 ms_handle_reset on v2:172.21.15.79:6838/235958219 2024-04-25T18:58:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.123+0000 7f887edefb80 1 --2- 172.21.15.45:0/1608485967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559b4ff3ec10 0x559b4ff63c20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.123+0000 7f887edefb80 1 -- 172.21.15.45:0/1608485967 >> 172.21.15.45:0/1608485967 conn(0x559b4fe6d550 msgr2=0x559b4ff40210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:11.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.123+0000 7f887edefb80 1 -- 172.21.15.45:0/1608485967 shutdown_connections 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.123+0000 7f887edefb80 1 -- 172.21.15.45:0/1608485967 wait complete. 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.123+0000 7f887edefb80 20 client.24883 trim_cache size 0 max 16384 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24505, 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 147.748073089, 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 25, 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-25T18:58:11.129 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:58:11.130 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 198.99944173700001 2024-04-25T18:58:11.130 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T18:58:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 -- 172.21.15.45:0/729454198 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f88540a3cc0 msgr2=0x7f88540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f88540a3cc0 0x7f88540a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f886c111140 tx=0x7f8858009260 comp rx=0 tx=0).stop 2024-04-25T18:58:11.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 -- 172.21.15.45:0/729454198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f886c074c70 msgr2=0x7f886c19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f886c074c70 0x7f886c19b420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8860000c00 tx=0x7f88600057c0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 -- 172.21.15.45:0/729454198 shutdown_connections 2024-04-25T18:58:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f88540a3cc0 0x7f88540a6180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f886c10f440 0x7f886c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f886c074c70 0x7f886c19b420 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 --2- 172.21.15.45:0/729454198 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f886c075590 0x7f886c19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 -- 172.21.15.45:0/729454198 >> 172.21.15.45:0/729454198 conn(0x7f886c079ba0 msgr2=0x7f886c100f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 -- 172.21.15.45:0/729454198 shutdown_connections 2024-04-25T18:58:11.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:11.124+0000 7f8874400700 1 -- 172.21.15.45:0/729454198 wait complete. 2024-04-25T18:58:11.666 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:11 smithi062 ceph-mon[32844]: pgmap v411: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:11.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:11 smithi079 ceph-mon[33002]: pgmap v411: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:11.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:11 smithi045 ceph-mon[27776]: pgmap v411: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:11.968 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.a status 2024-04-25T18:58:12.440 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:13.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:13 smithi079 ceph-mon[33002]: pgmap v412: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:13.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:13 smithi045 ceph-mon[27776]: pgmap v412: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:13.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:13 smithi062 ceph-mon[32844]: pgmap v412: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.708+0000 7ff0d5600700 1 -- 172.21.15.45:0/2964881948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0d0074c70 msgr2=0x7ff0d0075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.708+0000 7ff0d5600700 1 --2- 172.21.15.45:0/2964881948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0d0074c70 0x7ff0d0075050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009f90 tx=0x7ff0c000b330 comp rx=0 tx=0).stop 2024-04-25T18:58:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.708+0000 7ff0d5600700 1 -- 172.21.15.45:0/2964881948 shutdown_connections 2024-04-25T18:58:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.708+0000 7ff0d5600700 1 --2- 172.21.15.45:0/2964881948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d010f4a0 0x7ff0d0111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.708+0000 7ff0d5600700 1 --2- 172.21.15.45:0/2964881948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0d0074c70 0x7ff0d0075050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.708+0000 7ff0d5600700 1 --2- 172.21.15.45:0/2964881948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d0075590 0x7ff0d010ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.708+0000 7ff0d5600700 1 -- 172.21.15.45:0/2964881948 >> 172.21.15.45:0/2964881948 conn(0x7ff0d0079ba0 msgr2=0x7ff0d0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.708+0000 7ff0d5600700 1 -- 172.21.15.45:0/2964881948 shutdown_connections 2024-04-25T18:58:14.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.708+0000 7ff0d5600700 1 -- 172.21.15.45:0/2964881948 wait complete. 2024-04-25T18:58:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.709+0000 7ff0d5600700 1 Processor -- start 2024-04-25T18:58:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0d5600700 1 -- start start 2024-04-25T18:58:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0d5600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d0074c70 0x7ff0d019b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0d5600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d0075590 0x7ff0d019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0d5600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0d010f4a0 0x7ff0d019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0d5600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0114e90 con 0x7ff0d0074c70 2024-04-25T18:58:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0d5600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0114d10 con 0x7ff0d0075590 2024-04-25T18:58:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0d5600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d0115010 con 0x7ff0d010f4a0 2024-04-25T18:58:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0cea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d0074c70 0x7ff0d019b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0cea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d0074c70 0x7ff0d019b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38906/0 (socket says 172.21.15.45:38906) 2024-04-25T18:58:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0cea00700 1 -- 172.21.15.45:0/2742592621 learned_addr learned my addr 172.21.15.45:0/2742592621 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.710+0000 7ff0ce000700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d0075590 0x7ff0d019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.711+0000 7ff0cf400700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0d010f4a0 0x7ff0d019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.711+0000 7ff0cea00700 1 -- 172.21.15.45:0/2742592621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0d010f4a0 msgr2=0x7ff0d019fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.711+0000 7ff0cea00700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0d010f4a0 0x7ff0d019fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.711+0000 7ff0cea00700 1 -- 172.21.15.45:0/2742592621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d0075590 msgr2=0x7ff0d019b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.711+0000 7ff0cea00700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d0075590 0x7ff0d019b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.711+0000 7ff0cea00700 1 -- 172.21.15.45:0/2742592621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d01a0280 con 0x7ff0d0074c70 2024-04-25T18:58:14.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.711+0000 7ff0cea00700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d0074c70 0x7ff0d019b2c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009f60 tx=0x7ff0c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:14.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.712+0000 7ff0c7400700 1 -- 172.21.15.45:0/2742592621 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0021070 con 0x7ff0d0074c70 2024-04-25T18:58:14.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.712+0000 7ff0d5600700 1 -- 172.21.15.45:0/2742592621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d01a0510 con 0x7ff0d0074c70 2024-04-25T18:58:14.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.712+0000 7ff0c7400700 1 -- 172.21.15.45:0/2742592621 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c0007ac0 con 0x7ff0d0074c70 2024-04-25T18:58:14.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.712+0000 7ff0c7400700 1 -- 172.21.15.45:0/2742592621 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0005040 con 0x7ff0d0074c70 2024-04-25T18:58:14.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.712+0000 7ff0d5600700 1 -- 172.21.15.45:0/2742592621 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d01a7df0 con 0x7ff0d0074c70 2024-04-25T18:58:14.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.717+0000 7ff0c7400700 1 -- 172.21.15.45:0/2742592621 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff0c0024070 con 0x7ff0d0074c70 2024-04-25T18:58:14.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.718+0000 7ff0dff50b80 1 Processor -- start 2024-04-25T18:58:14.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.718+0000 7ff0dff50b80 1 -- start start 2024-04-25T18:58:14.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.718+0000 7ff0c7400700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff0b40a3c40 0x7ff0b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.718+0000 7ff0c7400700 1 -- 172.21.15.45:0/2742592621 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7ff0c000f020 con 0x7ff0d0074c70 2024-04-25T18:58:14.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.718+0000 7ff0cf400700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff0b40a3c40 0x7ff0b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.718+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f62562f0 0x55e0f62566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6256c10 0x55e0f6269d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f626a2c0 0x55e0f626c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e0f61a9e40 con 0x55e0f6256c10 2024-04-25T18:58:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e0f61506e0 con 0x55e0f62562f0 2024-04-25T18:58:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e0f61a5e00 con 0x55e0f626a2c0 2024-04-25T18:58:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0cea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f62562f0 0x55e0f62566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0ce000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6256c10 0x55e0f6269d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0cea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f62562f0 0x55e0f62566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:44458/0 (socket says 172.21.15.45:44458) 2024-04-25T18:58:14.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0cf400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f626a2c0 0x55e0f626c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0cea00700 1 -- 172.21.15.45:0/134905912 learned_addr learned my addr 172.21.15.45:0/134905912 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0cf400700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff0b40a3c40 0x7ff0b40a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0d006b270 tx=0x7ff0c8011440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0cea00700 1 -- 172.21.15.45:0/134905912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f626a2c0 msgr2=0x55e0f626c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0cea00700 1 --2- 172.21.15.45:0/134905912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f626a2c0 0x55e0f626c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0cea00700 1 -- 172.21.15.45:0/134905912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6256c10 msgr2=0x55e0f6269d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.719+0000 7ff0cea00700 1 --2- 172.21.15.45:0/134905912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6256c10 0x55e0f6269d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.720+0000 7ff0cea00700 1 -- 172.21.15.45:0/134905912 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f6146630 con 0x55e0f62562f0 2024-04-25T18:58:14.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.720+0000 7ff0cea00700 1 --2- 172.21.15.45:0/134905912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f62562f0 0x55e0f62566d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0099fb0 tx=0x7ff0c000cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b9a8c6af088ac14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.720+0000 7ff0bbe00700 1 -- 172.21.15.45:0/134905912 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c009e250 con 0x55e0f62562f0 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.720+0000 7ff0bbe00700 1 -- 172.21.15.45:0/134905912 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c009db50 con 0x55e0f62562f0 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.720+0000 7ff0bbe00700 1 -- 172.21.15.45:0/134905912 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c009ddb0 con 0x55e0f62562f0 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.721+0000 7ff0dff50b80 1 -- 172.21.15.45:0/134905912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f62562f0 msgr2=0x55e0f62566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.721+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/134905912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f62562f0 0x55e0f62566d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0099fb0 tx=0x7ff0c000cf50 comp rx=0 tx=0).stop 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.721+0000 7ff0dff50b80 1 -- 172.21.15.45:0/134905912 shutdown_connections 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.721+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/134905912 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f62562f0 0x55e0f62566d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.721+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/134905912 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f626a2c0 0x55e0f626c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.721+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/134905912 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6256c10 0x55e0f6269d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.721+0000 7ff0dff50b80 1 -- 172.21.15.45:0/134905912 >> 172.21.15.45:0/134905912 conn(0x55e0f6185550 msgr2=0x55e0f6252790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.721+0000 7ff0dff50b80 1 -- 172.21.15.45:0/134905912 shutdown_connections 2024-04-25T18:58:14.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.721+0000 7ff0dff50b80 1 -- 172.21.15.45:0/134905912 wait complete. 2024-04-25T18:58:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.722+0000 7ff0dff50b80 1 -- start start 2024-04-25T18:58:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.722+0000 7ff0dff50b80 1 Processor -- start 2024-04-25T18:58:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f6269a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6256c10 0x55e0f6266d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f626a2c0 0x55e0f62672b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e0f61506e0 con 0x55e0f62562f0 2024-04-25T18:58:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e0f6163810 con 0x55e0f626a2c0 2024-04-25T18:58:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e0f626cbf0 con 0x55e0f6256c10 2024-04-25T18:58:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0dff50b80 10 client.0 _ll_register_callbacks cb 0x55e0f61426d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:58:14.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0cea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f6269a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0cea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f6269a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38920/0 (socket says 172.21.15.45:38920) 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0cea00700 1 -- 172.21.15.45:0/697391427 learned_addr learned my addr 172.21.15.45:0/697391427 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0ce000700 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6256c10 0x55e0f6266d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0cf400700 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f626a2c0 0x55e0f62672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0cea00700 1 -- 172.21.15.45:0/697391427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6256c10 msgr2=0x55e0f6266d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0cea00700 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6256c10 0x55e0f6266d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0cea00700 1 -- 172.21.15.45:0/697391427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f626a2c0 msgr2=0x55e0f62672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0cea00700 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f626a2c0 0x55e0f62672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0cea00700 1 -- 172.21.15.45:0/697391427 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f60f7880 con 0x55e0f62562f0 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0ce000700 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6256c10 0x55e0f6266d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.723+0000 7ff0cf400700 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f626a2c0 0x55e0f62672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.724+0000 7ff0cea00700 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f6269a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000b6c0 tx=0x7ff0c00a6730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36e62525a3b6897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.724+0000 7ff0b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.724+0000 7ff0b8c00700 1 -- 172.21.15.45:0/697391427 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c009ddb0 con 0x55e0f62562f0 2024-04-25T18:58:14.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.724+0000 7ff0b8c00700 1 -- 172.21.15.45:0/697391427 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7ff0c009df50 con 0x55e0f62562f0 2024-04-25T18:58:14.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.724+0000 7ff0dff50b80 1 -- 172.21.15.45:0/697391427 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c009db50 con 0x55e0f62562f0 2024-04-25T18:58:14.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.724+0000 7ff0b8c00700 1 -- 172.21.15.45:0/697391427 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0c00b7020 con 0x55e0f62562f0 2024-04-25T18:58:14.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.724+0000 7ff0dff50b80 10 client.14772 fetch_fsmap learned FSMap version 25 2024-04-25T18:58:14.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.724+0000 7ff0dff50b80 1 -- 172.21.15.45:0/697391427 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0f6140aa0 con 0x55e0f62562f0 2024-04-25T18:58:14.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.725+0000 7ff0b8c00700 1 -- 172.21.15.45:0/697391427 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c009db50 con 0x55e0f62562f0 2024-04-25T18:58:14.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.725+0000 7ff0b8c00700 1 -- 172.21.15.45:0/697391427 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff0c0011070 con 0x55e0f62562f0 2024-04-25T18:58:14.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.725+0000 7ff0dff50b80 10 client.14772 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.725+0000 7ff0dff50b80 10 client.14772 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.725+0000 7ff0dff50b80 20 client.14772 populate_metadata read hostname 'smithi045' 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.725+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55e0f6279660 0x55e0f627aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.725+0000 7ff0dff50b80 4 client.14772 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.725+0000 7ff0dff50b80 1 -- 172.21.15.45:0/697391427 --> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0f61b7630 con 0x55e0f6279660 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.725+0000 7ff0ce000700 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55e0f6279660 0x55e0f627aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.726+0000 7ff0ce000700 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55e0f6279660 0x55e0f627aad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a39c562acb4b5ccc server_cookie=dd71e3d8ca875b5d in_seq=0 out_seq=0 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.726+0000 7ff0b8c00700 10 client.14772 ms_handle_connect on v2:172.21.15.45:6840/2490735053 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0b8c00700 1 -- 172.21.15.45:0/697391427 <== mds.1 v2:172.21.15.45:6840/2490735053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0f61b7630 con 0x55e0f6279660 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0b8c00700 10 client.14772 handle_command_reply: tid=0 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0dff50b80 1 client.14772 shutdown 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0dff50b80 1 -- 172.21.15.45:0/697391427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 msgr2=0x55e0f6269a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f6269a70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000b6c0 tx=0x7ff0c00a6730 comp rx=0 tx=0).stop 2024-04-25T18:58:14.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0dff50b80 1 -- 172.21.15.45:0/697391427 shutdown_connections 2024-04-25T18:58:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55e0f6279660 0x55e0f627aad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f626a2c0 0x55e0f62672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0b8c00700 0 client.14772 ms_handle_reset on v2:172.21.15.45:6840/2490735053 2024-04-25T18:58:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f6269a70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/697391427 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6256c10 0x55e0f6266d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.727+0000 7ff0dff50b80 1 -- 172.21.15.45:0/697391427 >> 172.21.15.45:0/697391427 conn(0x55e0f6185550 msgr2=0x55e0f6251950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.728+0000 7ff0dff50b80 1 -- 172.21.15.45:0/697391427 shutdown_connections 2024-04-25T18:58:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.728+0000 7ff0dff50b80 1 -- 172.21.15.45:0/697391427 wait complete. 2024-04-25T18:58:14.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.728+0000 7ff0dff50b80 20 client.14772 trim_cache size 0 max 16384 2024-04-25T18:58:14.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.751+0000 7ff0dff50b80 1 Processor -- start 2024-04-25T18:58:14.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.751+0000 7ff0dff50b80 1 -- start start 2024-04-25T18:58:14.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6185550 0x55e0f62625b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f62520d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f6256c10 0x55e0f6252610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e0f6278e10 con 0x55e0f62562f0 2024-04-25T18:58:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e0f62794e0 con 0x55e0f6256c10 2024-04-25T18:58:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e0f6279bb0 con 0x55e0f6185550 2024-04-25T18:58:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0ce000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f62520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0ce000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f62520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38926/0 (socket says 172.21.15.45:38926) 2024-04-25T18:58:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0ce000700 1 -- 172.21.15.45:0/1347702825 learned_addr learned my addr 172.21.15.45:0/1347702825 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0cf400700 1 --2- 172.21.15.45:0/1347702825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f6256c10 0x55e0f6252610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.752+0000 7ff0cea00700 1 --2- 172.21.15.45:0/1347702825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6185550 0x55e0f62625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.753+0000 7ff0ce000700 1 -- 172.21.15.45:0/1347702825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6185550 msgr2=0x55e0f62625b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.753+0000 7ff0ce000700 1 --2- 172.21.15.45:0/1347702825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6185550 0x55e0f62625b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.753+0000 7ff0ce000700 1 -- 172.21.15.45:0/1347702825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f6256c10 msgr2=0x55e0f6252610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.753+0000 7ff0ce000700 1 --2- 172.21.15.45:0/1347702825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f6256c10 0x55e0f6252610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.753+0000 7ff0ce000700 1 -- 172.21.15.45:0/1347702825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0f6267ca0 con 0x55e0f62562f0 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.753+0000 7ff0ce000700 1 --2- 172.21.15.45:0/1347702825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f62520d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009840 tx=0x7ff0bc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cf70ff5ccb2d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.753+0000 7ff0bb400700 1 -- 172.21.15.45:0/1347702825 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc011e00 con 0x55e0f62562f0 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.753+0000 7ff0bb400700 1 -- 172.21.15.45:0/1347702825 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0bc004500 con 0x55e0f62562f0 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.753+0000 7ff0bb400700 1 -- 172.21.15.45:0/1347702825 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0053a0 con 0x55e0f62562f0 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.754+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1347702825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 msgr2=0x55e0f62520d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.754+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/1347702825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f62520d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009840 tx=0x7ff0bc009310 comp rx=0 tx=0).stop 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.754+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1347702825 shutdown_connections 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.754+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/1347702825 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f6256c10 0x55e0f6252610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.754+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/1347702825 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f6185550 0x55e0f62625b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.754+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/1347702825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f62562f0 0x55e0f62520d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.754+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1347702825 >> 172.21.15.45:0/1347702825 conn(0x55e0f624e710 msgr2=0x55e0f62671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.754+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1347702825 shutdown_connections 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.754+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1347702825 wait complete. 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.755+0000 7ff0dff50b80 1 -- start start 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.755+0000 7ff0dff50b80 1 Processor -- start 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.755+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6185550 0x55e0f6284460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.755+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f62562f0 0x55e0f6283070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.755+0000 7ff0dff50b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f6256c10 0x55e0f62835d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.755+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e0f627a950 con 0x55e0f6185550 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.755+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e0f627a280 con 0x55e0f6256c10 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.755+0000 7ff0dff50b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e0f627b020 con 0x55e0f62562f0 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.755+0000 7ff0dff50b80 10 client.0 _ll_register_callbacks cb 0x55e0f61426d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:58:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.756+0000 7ff0cea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6185550 0x55e0f6284460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.756+0000 7ff0cea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6185550 0x55e0f6284460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38934/0 (socket says 172.21.15.45:38934) 2024-04-25T18:58:14.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.756+0000 7ff0cea00700 1 -- 172.21.15.45:0/1613250404 learned_addr learned my addr 172.21.15.45:0/1613250404 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:14.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.756+0000 7ff0ce000700 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f62562f0 0x55e0f6283070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.756+0000 7ff0cf400700 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f6256c10 0x55e0f62835d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.756+0000 7ff0cea00700 1 -- 172.21.15.45:0/1613250404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f62562f0 msgr2=0x55e0f6283070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:14.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.756+0000 7ff0cea00700 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f62562f0 0x55e0f6283070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.756+0000 7ff0cea00700 1 -- 172.21.15.45:0/1613250404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f6256c10 msgr2=0x55e0f62835d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:14.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.756+0000 7ff0cea00700 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f6256c10 0x55e0f62835d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.756+0000 7ff0cea00700 1 -- 172.21.15.45:0/1613250404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0f6283e10 con 0x55e0f6185550 2024-04-25T18:58:14.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.757+0000 7ff0cea00700 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6185550 0x55e0f6284460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00a6200 tx=0x7ff0c0099df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf02dfeb0fa39c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:14.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.757+0000 7ff0b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T18:58:14.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.757+0000 7ff0b8c00700 1 -- 172.21.15.45:0/1613250404 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c009d4b0 con 0x55e0f6185550 2024-04-25T18:58:14.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.757+0000 7ff0b8c00700 1 -- 172.21.15.45:0/1613250404 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7ff0c009dfb0 con 0x55e0f6185550 2024-04-25T18:58:14.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.757+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1613250404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0bc004500 con 0x55e0f6185550 2024-04-25T18:58:14.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.757+0000 7ff0b8c00700 1 -- 172.21.15.45:0/1613250404 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff0c0081050 con 0x55e0f6185550 2024-04-25T18:58:14.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.757+0000 7ff0dff50b80 10 client.14778 fetch_fsmap learned FSMap version 25 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.757+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1613250404 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0f6283e10 con 0x55e0f6185550 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.757+0000 7ff0b8c00700 1 -- 172.21.15.45:0/1613250404 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0bc004500 con 0x55e0f6185550 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.758+0000 7ff0b8c00700 1 -- 172.21.15.45:0/1613250404 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff0c0011070 con 0x55e0f6185550 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.758+0000 7ff0dff50b80 10 client.14778 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.758+0000 7ff0dff50b80 10 client.14778 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.758+0000 7ff0dff50b80 20 client.14778 populate_metadata read hostname 'smithi045' 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.758+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55e0f626a2c0 0x55e0f62e4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.758+0000 7ff0dff50b80 4 client.14778 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.758+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1613250404 --> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] -- command(tid 0: {"prefix": "status"}) -- 0x55e0f61b16d0 con 0x55e0f626a2c0 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.758+0000 7ff0ce000700 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55e0f626a2c0 0x55e0f62e4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0ce000700 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55e0f626a2c0 0x55e0f62e4bb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4903e43318e7613c server_cookie=363e5b9016e1e290 in_seq=0 out_seq=0 2024-04-25T18:58:14.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0b8c00700 10 client.14778 ms_handle_connect on v2:172.21.15.45:6840/2490735053 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0b8c00700 1 -- 172.21.15.45:0/1613250404 <== mds.1 v2:172.21.15.45:6840/2490735053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55e0f61b16d0 con 0x55e0f626a2c0 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0b8c00700 10 client.14778 handle_command_reply: tid=0 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0dff50b80 1 client.14778 shutdown 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1613250404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6185550 msgr2=0x55e0f6284460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6185550 0x55e0f6284460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff0c00a6200 tx=0x7ff0c0099df0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1613250404 shutdown_connections 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55e0f626a2c0 0x55e0f62e4bb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e0f6256c10 0x55e0f62835d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e0f6185550 0x55e0f6284460 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0dff50b80 1 --2- 172.21.15.45:0/1613250404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e0f62562f0 0x55e0f6283070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1613250404 >> 172.21.15.45:0/1613250404 conn(0x55e0f624e710 msgr2=0x55e0f62685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.759+0000 7ff0b8c00700 0 client.14778 ms_handle_reset on v2:172.21.15.45:6840/2490735053 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.760+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1613250404 shutdown_connections 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.760+0000 7ff0dff50b80 1 -- 172.21.15.45:0/1613250404 wait complete. 2024-04-25T18:58:14.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.760+0000 7ff0dff50b80 20 client.14778 trim_cache size 0 max 16384 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24512, 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 152.394123345, 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 25, 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-25T18:58:14.765 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:58:14.766 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 192.268283096 2024-04-25T18:58:14.766 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T18:58:14.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.760+0000 7ff0d5600700 1 -- 172.21.15.45:0/2742592621 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff0b40a3c40 msgr2=0x7ff0b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.760+0000 7ff0d5600700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff0b40a3c40 0x7ff0b40a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff0d006b270 tx=0x7ff0c8011440 comp rx=0 tx=0).stop 2024-04-25T18:58:14.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.760+0000 7ff0d5600700 1 -- 172.21.15.45:0/2742592621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d0074c70 msgr2=0x7ff0d019b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:14.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.760+0000 7ff0d5600700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d0074c70 0x7ff0d019b2c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff0c0009f60 tx=0x7ff0c000ccb0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.761+0000 7ff0d5600700 1 -- 172.21.15.45:0/2742592621 shutdown_connections 2024-04-25T18:58:14.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.761+0000 7ff0d5600700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7ff0b40a3c40 0x7ff0b40a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.761+0000 7ff0d5600700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff0d010f4a0 0x7ff0d019fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.761+0000 7ff0d5600700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff0d0074c70 0x7ff0d019b2c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.761+0000 7ff0d5600700 1 --2- 172.21.15.45:0/2742592621 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff0d0075590 0x7ff0d019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:14.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.761+0000 7ff0d5600700 1 -- 172.21.15.45:0/2742592621 >> 172.21.15.45:0/2742592621 conn(0x7ff0d0079ba0 msgr2=0x7ff0d00ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:14.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.761+0000 7ff0d5600700 1 -- 172.21.15.45:0/2742592621 shutdown_connections 2024-04-25T18:58:14.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:14.761+0000 7ff0d5600700 1 -- 172.21.15.45:0/2742592621 wait complete. 2024-04-25T18:58:15.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:15 smithi045 ceph-mon[27776]: pgmap v413: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:15.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:15 smithi079 ceph-mon[33002]: pgmap v413: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:15.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T18:58:15.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:15 smithi062 ceph-mon[32844]: pgmap v413: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:16.184 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:17.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:17 smithi045 ceph-mon[27776]: pgmap v414: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:17.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:17 smithi062 ceph-mon[32844]: pgmap v414: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:17 smithi079 ceph-mon[33002]: pgmap v414: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.598+0000 7fa090000700 1 -- 172.21.15.45:0/678415240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa088074c70 msgr2=0x7fa088075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.598+0000 7fa090000700 1 --2- 172.21.15.45:0/678415240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa088074c70 0x7fa088075050 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa07c009990 tx=0x7fa07c009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.598+0000 7fa090000700 1 -- 172.21.15.45:0/678415240 shutdown_connections 2024-04-25T18:58:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.598+0000 7fa090000700 1 --2- 172.21.15.45:0/678415240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa088074c70 0x7fa088075050 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.598+0000 7fa090000700 1 --2- 172.21.15.45:0/678415240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa08810f4a0 0x7fa088111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.598+0000 7fa090000700 1 --2- 172.21.15.45:0/678415240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa088075590 0x7fa08810ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.598+0000 7fa090000700 1 -- 172.21.15.45:0/678415240 >> 172.21.15.45:0/678415240 conn(0x7fa088079ba0 msgr2=0x7fa088079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:18.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.598+0000 7fa090000700 1 -- 172.21.15.45:0/678415240 shutdown_connections 2024-04-25T18:58:18.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.598+0000 7fa090000700 1 -- 172.21.15.45:0/678415240 wait complete. 2024-04-25T18:58:18.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.599+0000 7fa090000700 1 Processor -- start 2024-04-25T18:58:18.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.599+0000 7fa090000700 1 -- start start 2024-04-25T18:58:18.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa090000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa088074c70 0x7fa088102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa090000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa088075590 0x7fa088103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa090000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08810f4a0 0x7fa088103640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa090000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa088114e70 con 0x7fa08810f4a0 2024-04-25T18:58:18.601 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa090000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa088114cf0 con 0x7fa088075590 2024-04-25T18:58:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa090000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa088114ff0 con 0x7fa088074c70 2024-04-25T18:58:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa08e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08810f4a0 0x7fa088103640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa08cc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa088075590 0x7fa088103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa08e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08810f4a0 0x7fa088103640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38950/0 (socket says 172.21.15.45:38950) 2024-04-25T18:58:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa08cc00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa088075590 0x7fa088103100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:44504/0 (socket says 172.21.15.45:44504) 2024-04-25T18:58:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa08e000700 1 -- 172.21.15.45:0/1006286922 learned_addr learned my addr 172.21.15.45:0/1006286922 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:18.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.600+0000 7fa08d600700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa088074c70 0x7fa088102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa08d600700 1 -- 172.21.15.45:0/1006286922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa088075590 msgr2=0x7fa088103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa08d600700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa088075590 0x7fa088103100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa08d600700 1 -- 172.21.15.45:0/1006286922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08810f4a0 msgr2=0x7fa088103640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa08d600700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08810f4a0 0x7fa088103640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa08d600700 1 -- 172.21.15.45:0/1006286922 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0881aba70 con 0x7fa088074c70 2024-04-25T18:58:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa08e000700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08810f4a0 0x7fa088103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa08cc00700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa088075590 0x7fa088103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:58:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa08d600700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa088074c70 0x7fa088102bc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0098e0 tx=0x7fa07c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:18.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa086000700 1 -- 172.21.15.45:0/1006286922 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c027070 con 0x7fa088074c70 2024-04-25T18:58:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa086000700 1 -- 172.21.15.45:0/1006286922 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa07c022470 con 0x7fa088074c70 2024-04-25T18:58:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.601+0000 7fa090000700 1 -- 172.21.15.45:0/1006286922 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0881abd00 con 0x7fa088074c70 2024-04-25T18:58:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.602+0000 7fa086000700 1 -- 172.21.15.45:0/1006286922 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07c017650 con 0x7fa088074c70 2024-04-25T18:58:18.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.602+0000 7fa090000700 1 -- 172.21.15.45:0/1006286922 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0881ac1b0 con 0x7fa088074c70 2024-04-25T18:58:18.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.606+0000 7fa086000700 1 -- 172.21.15.45:0/1006286922 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa07c02a070 con 0x7fa088074c70 2024-04-25T18:58:18.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.606+0000 7fa09a8e2b80 1 Processor -- start 2024-04-25T18:58:18.607 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.607+0000 7fa09a8e2b80 1 -- start start 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.607+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 0x55a0773086d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.607+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077308c10 0x55a077313570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.607+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077313ab0 0x55a077315ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.607+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a07725be40 con 0x55a077313ab0 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.607+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a0772026e0 con 0x55a077308c10 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.607+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a077257e00 con 0x55a0773082f0 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.607+0000 7fa08e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 0x55a0773086d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.607+0000 7fa08e000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 0x55a0773086d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:34660/0 (socket says 172.21.15.45:34660) 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.607+0000 7fa08e000700 1 -- 172.21.15.45:0/1468961529 learned_addr learned my addr 172.21.15.45:0/1468961529 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.608+0000 7fa08d600700 1 --2- 172.21.15.45:0/1468961529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077308c10 0x55a077313570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.609 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.608+0000 7fa08cc00700 1 --2- 172.21.15.45:0/1468961529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077313ab0 0x55a077315ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.608+0000 7fa08e000700 1 -- 172.21.15.45:0/1468961529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077308c10 msgr2=0x55a077313570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.608+0000 7fa08e000700 1 --2- 172.21.15.45:0/1468961529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077308c10 0x55a077313570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.608+0000 7fa08e000700 1 -- 172.21.15.45:0/1468961529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077313ab0 msgr2=0x55a077315ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.608+0000 7fa08e000700 1 --2- 172.21.15.45:0/1468961529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077313ab0 0x55a077315ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.608+0000 7fa08e000700 1 -- 172.21.15.45:0/1468961529 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0771f8630 con 0x55a0773082f0 2024-04-25T18:58:18.610 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.608+0000 7fa08cc00700 1 --2- 172.21.15.45:0/1468961529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077313ab0 0x55a077315ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:58:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.608+0000 7fa086000700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa0700a3cc0 0x7fa0700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.608+0000 7fa08e000700 1 --2- 172.21.15.45:0/1468961529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 0x55a0773086d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa080009510 tx=0x7fa080009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f559cef5e0e182ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.609+0000 7fa086000700 1 -- 172.21.15.45:0/1006286922 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa07c0d1090 con 0x7fa088074c70 2024-04-25T18:58:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.609+0000 7fa07aa00700 1 -- 172.21.15.45:0/1468961529 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080019070 con 0x55a0773082f0 2024-04-25T18:58:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.609+0000 7fa08d600700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa0700a3cc0 0x7fa0700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.609+0000 7fa07aa00700 1 -- 172.21.15.45:0/1468961529 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa080004db0 con 0x55a0773082f0 2024-04-25T18:58:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.609+0000 7fa07aa00700 1 -- 172.21.15.45:0/1468961529 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080005600 con 0x55a0773082f0 2024-04-25T18:58:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.610+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/1468961529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 msgr2=0x55a0773086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.610+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/1468961529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 0x55a0773086d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa080009510 tx=0x7fa080009920 comp rx=0 tx=0).stop 2024-04-25T18:58:18.611 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.610+0000 7fa08d600700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa0700a3cc0 0x7fa0700a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0059d0 tx=0x7fa07c099040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.610+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/1468961529 shutdown_connections 2024-04-25T18:58:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.610+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/1468961529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077313ab0 0x55a077315ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.610+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/1468961529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 0x55a0773086d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.610+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/1468961529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077308c10 0x55a077313570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.610+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/1468961529 >> 172.21.15.45:0/1468961529 conn(0x55a077237550 msgr2=0x55a077304790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.610+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/1468961529 shutdown_connections 2024-04-25T18:58:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.610+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/1468961529 wait complete. 2024-04-25T18:58:18.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.611+0000 7fa09a8e2b80 1 -- start start 2024-04-25T18:58:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.611+0000 7fa09a8e2b80 1 Processor -- start 2024-04-25T18:58:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.612+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a0773082f0 0x55a077313260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.612+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 0x55a077310560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.612+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077313ab0 0x55a077310aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.612+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a0773165a0 con 0x55a077308c10 2024-04-25T18:58:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.612+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a077215810 con 0x55a0773082f0 2024-04-25T18:58:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.612+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a077316720 con 0x55a077313ab0 2024-04-25T18:58:18.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.612+0000 7fa09a8e2b80 10 client.0 _ll_register_callbacks cb 0x55a0771f46d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:58:18.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08e000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a0773082f0 0x55a077313260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08cc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 0x55a077310560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08e000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a0773082f0 0x55a077313260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:44522/0 (socket says 172.21.15.45:44522) 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08e000700 1 -- 172.21.15.45:0/154155815 learned_addr learned my addr 172.21.15.45:0/154155815 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08e000700 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077313ab0 0x55a077310aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08e000700 1 -- 172.21.15.45:0/154155815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077313ab0 msgr2=0x55a077310aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08e000700 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077313ab0 0x55a077310aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08e000700 1 -- 172.21.15.45:0/154155815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 msgr2=0x55a077310560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08e000700 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 0x55a077310560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08e000700 1 -- 172.21.15.45:0/154155815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0771a9880 con 0x55a0773082f0 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa08e000700 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a0773082f0 0x55a077313260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa080005060 tx=0x7fa080005160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b6cea7a097a8fd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.613+0000 7fa06f400700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.614+0000 7fa06f400700 1 -- 172.21.15.45:0/154155815 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080019030 con 0x55a0773082f0 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.614+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/154155815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa080004db0 con 0x55a0773082f0 2024-04-25T18:58:18.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.614+0000 7fa06f400700 1 -- 172.21.15.45:0/154155815 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa080003c00 con 0x55a0773082f0 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.614+0000 7fa06f400700 1 -- 172.21.15.45:0/154155815 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa080026040 con 0x55a0773082f0 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.614+0000 7fa09a8e2b80 10 client.24908 fetch_fsmap learned FSMap version 25 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.614+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/154155815 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a0771f2aa0 con 0x55a0773082f0 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.615+0000 7fa06f400700 1 -- 172.21.15.45:0/154155815 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa080004db0 con 0x55a0773082f0 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.615+0000 7fa06f400700 1 -- 172.21.15.45:0/154155815 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa08002d500 con 0x55a0773082f0 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.615+0000 7fa09a8e2b80 10 client.24908 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.615+0000 7fa09a8e2b80 10 client.24908 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.615+0000 7fa09a8e2b80 20 client.24908 populate_metadata read hostname 'smithi045' 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.615+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55a077323050 0x55a0773244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.615+0000 7fa09a8e2b80 4 client.24908 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.615+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/154155815 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a077269630 con 0x55a077323050 2024-04-25T18:58:18.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.615+0000 7fa08cc00700 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55a077323050 0x55a0773244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.616+0000 7fa08cc00700 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55a077323050 0x55a0773244c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0e3d76fb92fcfc server_cookie=af84748503ec7eac in_seq=0 out_seq=0 2024-04-25T18:58:18.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.616+0000 7fa06f400700 10 client.24908 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T18:58:18.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa06f400700 1 -- 172.21.15.45:0/154155815 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a077269630 con 0x55a077323050 2024-04-25T18:58:18.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa06f400700 10 client.24908 handle_command_reply: tid=0 2024-04-25T18:58:18.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa09a8e2b80 1 client.24908 shutdown 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/154155815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a0773082f0 msgr2=0x55a077313260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a0773082f0 0x55a077313260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa080005060 tx=0x7fa080005160 comp rx=0 tx=0).stop 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/154155815 shutdown_connections 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55a077323050 0x55a0773244c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a0773082f0 0x55a077313260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077313ab0 0x55a077310aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/154155815 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 0x55a077310560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa06f400700 0 client.24908 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.617+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/154155815 >> 172.21.15.45:0/154155815 conn(0x55a077237550 msgr2=0x55a07730ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.618+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/154155815 shutdown_connections 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.618+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/154155815 wait complete. 2024-04-25T18:58:18.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.618+0000 7fa09a8e2b80 20 client.24908 trim_cache size 0 max 16384 2024-04-25T18:58:18.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.661+0000 7fa09a8e2b80 1 Processor -- start 2024-04-25T18:58:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.661+0000 7fa09a8e2b80 1 -- start start 2024-04-25T18:58:18.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0773082f0 0x55a077322980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077308c10 0x55a077310a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077313ab0 0x55a07730b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a077322800 con 0x55a0773082f0 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a077322ed0 con 0x55a077313ab0 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a0773235a0 con 0x55a077308c10 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa08e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0773082f0 0x55a077322980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa08cc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077308c10 0x55a077310a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa08e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0773082f0 0x55a077322980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38980/0 (socket says 172.21.15.45:38980) 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa08cc00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077308c10 0x55a077310a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:34674/0 (socket says 172.21.15.45:34674) 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.662+0000 7fa08e000700 1 -- 172.21.15.45:0/3089034428 learned_addr learned my addr 172.21.15.45:0/3089034428 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.663+0000 7fa08e000700 1 --2- 172.21.15.45:0/3089034428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077313ab0 0x55a07730b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.663+0000 7fa08cc00700 1 -- 172.21.15.45:0/3089034428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077313ab0 msgr2=0x55a07730b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.663+0000 7fa08cc00700 1 --2- 172.21.15.45:0/3089034428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077313ab0 0x55a07730b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.663+0000 7fa08cc00700 1 -- 172.21.15.45:0/3089034428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0773082f0 msgr2=0x55a077322980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:18.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.663+0000 7fa08cc00700 1 --2- 172.21.15.45:0/3089034428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0773082f0 0x55a077322980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.665 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.663+0000 7fa08cc00700 1 -- 172.21.15.45:0/3089034428 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a077324830 con 0x55a077308c10 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.663+0000 7fa08cc00700 1 --2- 172.21.15.45:0/3089034428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077308c10 0x55a077310a50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55a077311b30 tx=0x7fa074003bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80be32ea8b0b06e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.663+0000 7fa07a000700 1 -- 172.21.15.45:0/3089034428 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074004a30 con 0x55a077308c10 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.664+0000 7fa07a000700 1 -- 172.21.15.45:0/3089034428 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0740054d0 con 0x55a077308c10 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.664+0000 7fa07a000700 1 -- 172.21.15.45:0/3089034428 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0740099c0 con 0x55a077308c10 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.664+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/3089034428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077308c10 msgr2=0x55a077310a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.664+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/3089034428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077308c10 0x55a077310a50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x55a077311b30 tx=0x7fa074003bb0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.664+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/3089034428 shutdown_connections 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.664+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/3089034428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077313ab0 0x55a07730b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.664+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/3089034428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a0773082f0 0x55a077322980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.664+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/3089034428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a077308c10 0x55a077310a50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.664+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/3089034428 >> 172.21.15.45:0/3089034428 conn(0x55a077237550 msgr2=0x55a077310340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:18.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.665+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/3089034428 shutdown_connections 2024-04-25T18:58:18.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.665+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/3089034428 wait complete. 2024-04-25T18:58:18.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.665+0000 7fa09a8e2b80 1 -- start start 2024-04-25T18:58:18.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.665+0000 7fa09a8e2b80 1 Processor -- start 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 0x55a07732d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 0x55a07732de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa09a8e2b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077313ab0 0x55a0773268e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a077324340 con 0x55a077308c10 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55a077323c70 con 0x55a077313ab0 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa09a8e2b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55a077324a10 con 0x55a0773082f0 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa09a8e2b80 10 client.0 _ll_register_callbacks cb 0x55a0771f46d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa08cc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 0x55a07732de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa08cc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 0x55a07732de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38994/0 (socket says 172.21.15.45:38994) 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa08cc00700 1 -- 172.21.15.45:0/4289514469 learned_addr learned my addr 172.21.15.45:0/4289514469 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa08e000700 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 0x55a07732d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.666+0000 7fa08e000700 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077313ab0 0x55a0773268e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.667+0000 7fa08cc00700 1 -- 172.21.15.45:0/4289514469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 msgr2=0x55a07732d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.667+0000 7fa08cc00700 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 0x55a07732d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.667+0000 7fa08cc00700 1 -- 172.21.15.45:0/4289514469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077313ab0 msgr2=0x55a0773268e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:58:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.667+0000 7fa08cc00700 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077313ab0 0x55a0773268e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.667+0000 7fa08cc00700 1 -- 172.21.15.45:0/4289514469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a077326fb0 con 0x55a077308c10 2024-04-25T18:58:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.667+0000 7fa08cc00700 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 0x55a07732de80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa0740056c0 tx=0x7fa0740056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61baf608ebf32472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.667+0000 7fa06f400700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T18:58:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.667+0000 7fa06f400700 1 -- 172.21.15.45:0/4289514469 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa074005320 con 0x55a077308c10 2024-04-25T18:58:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.667+0000 7fa06f400700 1 -- 172.21.15.45:0/4289514469 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa074004750 con 0x55a077308c10 2024-04-25T18:58:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.667+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/4289514469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0740054d0 con 0x55a077308c10 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa06f400700 1 -- 172.21.15.45:0/4289514469 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa074004990 con 0x55a077308c10 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa09a8e2b80 10 client.14793 fetch_fsmap learned FSMap version 25 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/4289514469 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a07738ab40 con 0x55a077308c10 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa06f400700 1 -- 172.21.15.45:0/4289514469 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0740054d0 con 0x55a077308c10 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa06f400700 1 -- 172.21.15.45:0/4289514469 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa074031060 con 0x55a077308c10 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa09a8e2b80 10 client.14793 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa09a8e2b80 10 client.14793 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa09a8e2b80 20 client.14793 populate_metadata read hostname 'smithi045' 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55a07738f580 0x55a07738e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa09a8e2b80 4 client.14793 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.668+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/4289514469 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "status"}) -- 0x55a0772636d0 con 0x55a07738f580 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.669+0000 7fa08e000700 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55a07738f580 0x55a07738e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.669+0000 7fa08e000700 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55a07738f580 0x55a07738e290 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77bee4bacc3b9221 server_cookie=782107e7209982e3 in_seq=0 out_seq=0 2024-04-25T18:58:18.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.669+0000 7fa06f400700 10 client.14793 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa06f400700 1 -- 172.21.15.45:0/4289514469 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55a0772636d0 con 0x55a07738f580 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa06f400700 10 client.14793 handle_command_reply: tid=0 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 client.14793 shutdown 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/4289514469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 msgr2=0x55a07732de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 0x55a07732de80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa0740056c0 tx=0x7fa0740056f0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/4289514469 shutdown_connections 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55a07738f580 0x55a07738e290 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55a077313ab0 0x55a0773268e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa06f400700 0 client.14793 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55a0773082f0 0x55a07732d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 --2- 172.21.15.45:0/4289514469 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a077308c10 0x55a07732de80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/4289514469 >> 172.21.15.45:0/4289514469 conn(0x55a077237550 msgr2=0x55a077311d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/4289514469 shutdown_connections 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 1 -- 172.21.15.45:0/4289514469 wait complete. 2024-04-25T18:58:18.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.670+0000 7fa09a8e2b80 20 client.14793 trim_cache size 0 max 16384 2024-04-25T18:58:18.674 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T18:58:18.674 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:58:18.674 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T18:58:18.674 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24524, 2024-04-25T18:58:18.674 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T18:58:18.674 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T18:58:18.674 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T18:58:18.674 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 172.429675038, 2024-04-25T18:58:18.674 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 25, 2024-04-25T18:58:18.674 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-25T18:58:18.675 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:58:18.675 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 193.258899945 2024-04-25T18:58:18.675 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T18:58:18.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 -- 172.21.15.45:0/1006286922 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa0700a3cc0 msgr2=0x7fa0700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa0700a3cc0 0x7fa0700a6180 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0059d0 tx=0x7fa07c099040 comp rx=0 tx=0).stop 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 -- 172.21.15.45:0/1006286922 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa088074c70 msgr2=0x7fa088102bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa088074c70 0x7fa088102bc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0098e0 tx=0x7fa07c0047b0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 -- 172.21.15.45:0/1006286922 shutdown_connections 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa0700a3cc0 0x7fa0700a6180 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa08810f4a0 0x7fa088103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa088074c70 0x7fa088102bc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 --2- 172.21.15.45:0/1006286922 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa088075590 0x7fa088103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 -- 172.21.15.45:0/1006286922 >> 172.21.15.45:0/1006286922 conn(0x7fa088079ba0 msgr2=0x7fa088100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 -- 172.21.15.45:0/1006286922 shutdown_connections 2024-04-25T18:58:18.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:18.671+0000 7fa090000700 1 -- 172.21.15.45:0/1006286922 wait complete. 2024-04-25T18:58:19.647 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T18:58:19.647 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T18:58:19.656 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T18:58:19.656 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T18:58:19.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:19 smithi045 ceph-mon[27776]: pgmap v415: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:19.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:19 smithi062 ceph-mon[32844]: pgmap v415: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:19.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:19 smithi079 ceph-mon[33002]: pgmap v415: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:20.094 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:21.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:21 smithi045 ceph-mon[27776]: pgmap v416: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:21.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:21 smithi062 ceph-mon[32844]: pgmap v416: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:21.935 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:21 smithi079 ceph-mon[33002]: pgmap v416: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:22.395 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.393+0000 7f4640a00700 1 -- 172.21.15.45:0/253381127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f463c10ab70 msgr2=0x7f463c111420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.393+0000 7f4640a00700 1 --2- 172.21.15.45:0/253381127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f463c10ab70 0x7f463c111420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f463400b0d0 tx=0x7f463400b3e0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.393+0000 7f4640a00700 1 -- 172.21.15.45:0/253381127 shutdown_connections 2024-04-25T18:58:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.393+0000 7f4640a00700 1 --2- 172.21.15.45:0/253381127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f463c10ab70 0x7f463c111420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.393+0000 7f4640a00700 1 --2- 172.21.15.45:0/253381127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f463c108fd0 0x7f463c1093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.393+0000 7f4640a00700 1 --2- 172.21.15.45:0/253381127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f463c10a1d0 0x7f463c10a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.393+0000 7f4640a00700 1 -- 172.21.15.45:0/253381127 >> 172.21.15.45:0/253381127 conn(0x7f463c077fa0 msgr2=0x7f463c07a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.393+0000 7f4640a00700 1 -- 172.21.15.45:0/253381127 shutdown_connections 2024-04-25T18:58:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.393+0000 7f4640a00700 1 -- 172.21.15.45:0/253381127 wait complete. 2024-04-25T18:58:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.394+0000 7f4640a00700 1 Processor -- start 2024-04-25T18:58:22.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.395+0000 7f4640a00700 1 -- start start 2024-04-25T18:58:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.395+0000 7f4640a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f463c108fd0 0x7f463c19f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.395+0000 7f4640a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f463c10a1d0 0x7f463c19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.395+0000 7f4640a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f463c10ab70 0x7f463c1a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.395+0000 7f4640a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f463c114aa0 con 0x7f463c10ab70 2024-04-25T18:58:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.395+0000 7f4640a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f463c114920 con 0x7f463c10a1d0 2024-04-25T18:58:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.395+0000 7f4640a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f463c114c20 con 0x7f463c108fd0 2024-04-25T18:58:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.395+0000 7f463b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f463c108fd0 0x7f463c19f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.396+0000 7f463b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f463c108fd0 0x7f463c19f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:34700/0 (socket says 172.21.15.45:34700) 2024-04-25T18:58:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.396+0000 7f463b400700 1 -- 172.21.15.45:0/3118806446 learned_addr learned my addr 172.21.15.45:0/3118806446 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:22.398 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.396+0000 7f463aa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f463c10a1d0 0x7f463c19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.396+0000 7f463be00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f463c10ab70 0x7f463c1a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.396+0000 7f463aa00700 1 -- 172.21.15.45:0/3118806446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f463c108fd0 msgr2=0x7f463c19f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.396+0000 7f463aa00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f463c108fd0 0x7f463c19f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.396+0000 7f463aa00700 1 -- 172.21.15.45:0/3118806446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f463c10ab70 msgr2=0x7f463c1a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.396+0000 7f463aa00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f463c10ab70 0x7f463c1a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.396+0000 7f463aa00700 1 -- 172.21.15.45:0/3118806446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c1a6ce0 con 0x7f463c10a1d0 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.397+0000 7f463aa00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f463c10a1d0 0x7f463c19f5c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f462800e9f0 tx=0x7f462800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.397+0000 7f4633e00700 1 -- 172.21.15.45:0/3118806446 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462800c9c0 con 0x7f463c10a1d0 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.397+0000 7f4640a00700 1 -- 172.21.15.45:0/3118806446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c1a6fd0 con 0x7f463c10a1d0 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.397+0000 7f4633e00700 1 -- 172.21.15.45:0/3118806446 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f462800cb60 con 0x7f463c10a1d0 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.397+0000 7f4633e00700 1 -- 172.21.15.45:0/3118806446 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4628018680 con 0x7f463c10a1d0 2024-04-25T18:58:22.399 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.397+0000 7f4640a00700 1 -- 172.21.15.45:0/3118806446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1a74d0 con 0x7f463c10a1d0 2024-04-25T18:58:22.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.398+0000 7f4640a00700 1 -- 172.21.15.45:0/3118806446 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463c069730 con 0x7f463c10a1d0 2024-04-25T18:58:22.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.400+0000 7f4633e00700 1 -- 172.21.15.45:0/3118806446 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4628010430 con 0x7f463c10a1d0 2024-04-25T18:58:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.402+0000 7f4633e00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46200a3c40 0x7f46200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.402+0000 7f4633e00700 1 -- 172.21.15.45:0/3118806446 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f46280d0590 con 0x7f463c10a1d0 2024-04-25T18:58:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.402+0000 7f463b400700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46200a3c40 0x7f46200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:22.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.403+0000 7f463b400700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46200a3c40 0x7f46200a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f462c005950 tx=0x7f462c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:22.407 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.406+0000 7f4633e00700 1 -- 172.21.15.45:0/3118806446 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f462809cef0 con 0x7f463c10a1d0 2024-04-25T18:58:22.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.659+0000 7f4640a00700 1 -- 172.21.15.45:0/3118806446 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f463c062e80 con 0x7f46200a3c40 2024-04-25T18:58:22.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.661+0000 7f4633e00700 1 -- 172.21.15.45:0/3118806446 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f463c062e80 con 0x7f46200a3c40 2024-04-25T18:58:22.663 INFO:teuthology.orchestra.run.smithi045.stdout:Backend: cephadm 2024-04-25T18:58:22.663 INFO:teuthology.orchestra.run.smithi045.stdout:Available: Yes 2024-04-25T18:58:22.663 INFO:teuthology.orchestra.run.smithi045.stdout:Paused: No 2024-04-25T18:58:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.664+0000 7f4640a00700 1 -- 172.21.15.45:0/3118806446 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46200a3c40 msgr2=0x7f46200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.664+0000 7f4640a00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46200a3c40 0x7f46200a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f462c005950 tx=0x7f462c009260 comp rx=0 tx=0).stop 2024-04-25T18:58:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.664+0000 7f4640a00700 1 -- 172.21.15.45:0/3118806446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f463c10a1d0 msgr2=0x7f463c19f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.664+0000 7f4640a00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f463c10a1d0 0x7f463c19f5c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f462800e9f0 tx=0x7f462800ed00 comp rx=0 tx=0).stop 2024-04-25T18:58:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.665+0000 7f4640a00700 1 -- 172.21.15.45:0/3118806446 shutdown_connections 2024-04-25T18:58:22.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.665+0000 7f4640a00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f46200a3c40 0x7f46200a6100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.665+0000 7f4640a00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f463c10ab70 0x7f463c1a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.665+0000 7f4640a00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f463c108fd0 0x7f463c19f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.665+0000 7f4640a00700 1 --2- 172.21.15.45:0/3118806446 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f463c10a1d0 0x7f463c19f5c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.665+0000 7f4640a00700 1 -- 172.21.15.45:0/3118806446 >> 172.21.15.45:0/3118806446 conn(0x7f463c077fa0 msgr2=0x7f463c10d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.665+0000 7f4640a00700 1 -- 172.21.15.45:0/3118806446 shutdown_connections 2024-04-25T18:58:22.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:22.665+0000 7f4640a00700 1 -- 172.21.15.45:0/3118806446 wait complete. 2024-04-25T18:58:23.459 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T18:58:23.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:23 smithi045 ceph-mon[27776]: pgmap v417: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:23.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:23 smithi045 ceph-mon[27776]: from='client.24920 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:23.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:23 smithi062 ceph-mon[32844]: pgmap v417: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:23.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:23 smithi062 ceph-mon[32844]: from='client.24920 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:23.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:23 smithi079 ceph-mon[33002]: pgmap v417: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:23.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:23 smithi079 ceph-mon[33002]: from='client.24920 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:23.947 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:25.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:25 smithi045 ceph-mon[27776]: pgmap v418: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:25.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:25 smithi062 ceph-mon[32844]: pgmap v418: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:25.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:25 smithi079 ceph-mon[33002]: pgmap v418: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:26.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.300+0000 7f2bb1600700 1 -- 172.21.15.45:0/3170829025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac103cc0 msgr2=0x7f2bac104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:26.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.300+0000 7f2bb1600700 1 --2- 172.21.15.45:0/3170829025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac103cc0 0x7f2bac104120 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009920 tx=0x7f2b94009c30 comp rx=0 tx=0).stop 2024-04-25T18:58:26.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.301+0000 7f2bb1600700 1 -- 172.21.15.45:0/3170829025 shutdown_connections 2024-04-25T18:58:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.301+0000 7f2bb1600700 1 --2- 172.21.15.45:0/3170829025 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bac104660 0x7f2bac10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.301+0000 7f2bb1600700 1 --2- 172.21.15.45:0/3170829025 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bac102ac0 0x7f2bac102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.301+0000 7f2bb1600700 1 --2- 172.21.15.45:0/3170829025 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac103cc0 0x7f2bac104120 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.301+0000 7f2bb1600700 1 -- 172.21.15.45:0/3170829025 >> 172.21.15.45:0/3170829025 conn(0x7f2bac0fe060 msgr2=0x7f2bac1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.301+0000 7f2bb1600700 1 -- 172.21.15.45:0/3170829025 shutdown_connections 2024-04-25T18:58:26.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.301+0000 7f2bb1600700 1 -- 172.21.15.45:0/3170829025 wait complete. 2024-04-25T18:58:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.302+0000 7f2bb1600700 1 Processor -- start 2024-04-25T18:58:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.302+0000 7f2bb1600700 1 -- start start 2024-04-25T18:58:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.302+0000 7f2bb1600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bac102ac0 0x7f2bac19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.302+0000 7f2bb1600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bac103cc0 0x7f2bac19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.302+0000 7f2bb1600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac104660 0x7f2bac1a2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.302+0000 7f2bb1600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10e580 con 0x7f2bac103cc0 2024-04-25T18:58:26.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.302+0000 7f2bb1600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10e400 con 0x7f2bac104660 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.302+0000 7f2bb1600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2bac10e700 con 0x7f2bac102ac0 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2ba2000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bac103cc0 0x7f2bac19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2baaa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bac102ac0 0x7f2bac19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2ba2000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bac103cc0 0x7f2bac19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38744/0 (socket says 172.21.15.45:38744) 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2ba2000700 1 -- 172.21.15.45:0/767232228 learned_addr learned my addr 172.21.15.45:0/767232228 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2bab400700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac104660 0x7f2bac1a2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2baaa00700 1 -- 172.21.15.45:0/767232228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac104660 msgr2=0x7f2bac1a2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2baaa00700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac104660 0x7f2bac1a2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2baaa00700 1 -- 172.21.15.45:0/767232228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bac103cc0 msgr2=0x7f2bac19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2baaa00700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bac103cc0 0x7f2bac19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2baaa00700 1 -- 172.21.15.45:0/767232228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bac1a2b30 con 0x7f2bac102ac0 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.303+0000 7f2ba2000700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bac103cc0 0x7f2bac19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.304+0000 7f2baaa00700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bac102ac0 0x7f2bac19aed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00eb90 tx=0x7f2b9c00eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:26.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.304+0000 7f2ba3e00700 1 -- 172.21.15.45:0/767232228 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c00cc90 con 0x7f2bac102ac0 2024-04-25T18:58:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.304+0000 7f2ba3e00700 1 -- 172.21.15.45:0/767232228 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9c00ce30 con 0x7f2bac102ac0 2024-04-25T18:58:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.304+0000 7f2bb1600700 1 -- 172.21.15.45:0/767232228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bac1a2e20 con 0x7f2bac102ac0 2024-04-25T18:58:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.304+0000 7f2ba3e00700 1 -- 172.21.15.45:0/767232228 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c018880 con 0x7f2bac102ac0 2024-04-25T18:58:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.304+0000 7f2bb1600700 1 -- 172.21.15.45:0/767232228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bac1a3270 con 0x7f2bac102ac0 2024-04-25T18:58:26.307 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.305+0000 7f2bb1600700 1 -- 172.21.15.45:0/767232228 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bac1a34a0 con 0x7f2bac102ac0 2024-04-25T18:58:26.308 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.307+0000 7f2ba3e00700 1 -- 172.21.15.45:0/767232228 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2b9c01f020 con 0x7f2bac102ac0 2024-04-25T18:58:26.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.308+0000 7f2ba3e00700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2b900a3c40 0x7f2b900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:26.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.309+0000 7f2ba3e00700 1 -- 172.21.15.45:0/767232228 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f2b9c0d04a0 con 0x7f2bac102ac0 2024-04-25T18:58:26.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.309+0000 7f2ba2000700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2b900a3c40 0x7f2b900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:26.310 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.309+0000 7f2ba2000700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2b900a3c40 0x7f2b900a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400faf0 tx=0x7f2b94019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:26.312 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.311+0000 7f2ba3e00700 1 -- 172.21.15.45:0/767232228 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b9c09ce00 con 0x7f2bac102ac0 2024-04-25T18:58:26.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.558+0000 7f2bb1600700 1 -- 172.21.15.45:0/767232228 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2bac076390 con 0x7f2b900a3c40 2024-04-25T18:58:26.575 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.574+0000 7f2ba3e00700 1 -- 172.21.15.45:0/767232228 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f2bac076390 con 0x7f2b900a3c40 2024-04-25T18:58:26.575 INFO:teuthology.orchestra.run.smithi045.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.a smithi045 running (3m) 42s ago 3m 18.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6f834433d81b 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.b smithi062 running (3m) 61s ago 3m 29.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5d57a1af6b5f 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.c smithi079 running (3m) 61s ago 3m 20.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3a4bf803c7fb 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.d smithi045 running (3m) 42s ago 3m 13.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 571d4c0dbe36 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.e smithi062 running (3m) 61s ago 3m 14.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8de2efef5e02 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.f smithi079 running (3m) 61s ago 3m 19.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 1de7d6a87493 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.g smithi045 running (3m) 42s ago 3m 13.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 753e25814128 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.h smithi062 running (3m) 61s ago 3m 21.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 fdce5165caa7 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.i smithi079 running (3m) 61s ago 3m 18.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d9faa66d057d 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.j smithi045 running (3m) 42s ago 3m 13.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 9f0469c61ac3 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.k smithi062 running (3m) 61s ago 3m 14.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 7e4a287086ff 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mds.l smithi079 running (4m) 61s ago 4m 16.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b7ee805eec8b 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.x smithi045 *:9283,8765 running (12m) 42s ago 12m 515M - 19.0.0-3156-gd69f13d8 da4b004c09e8 ffacc8ad311a 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.y smithi062 *:8443,8765 running (9m) 61s ago 9m 453M - 19.0.0-3156-gd69f13d8 da4b004c09e8 4cd562e72ce5 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mgr.z smithi079 *:8443,8765 running (9m) 61s ago 9m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 792d1af13380 2024-04-25T18:58:26.576 INFO:teuthology.orchestra.run.smithi045.stdout:mon.a smithi045 running (12m) 42s ago 12m 47.9M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 c3ffc389232c 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:mon.b smithi062 running (10m) 61s ago 10m 42.8M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 24591487aabe 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:mon.c smithi079 running (10m) 61s ago 10m 41.4M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 3fda186e9518 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 smithi045 running (9m) 42s ago 9m 54.8M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 c2b27313cd21 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 smithi062 running (8m) 61s ago 8m 57.9M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b365222624b9 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 smithi079 running (8m) 61s ago 8m 51.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 52100f97f58c 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 smithi045 running (8m) 42s ago 8m 53.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 195f82b657df 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 smithi062 running (7m) 61s ago 7m 46.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 00bbd8fe0418 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 smithi079 running (7m) 61s ago 7m 49.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 159235f83c85 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 smithi045 running (6m) 42s ago 6m 44.1M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 baa69a5f6f9c 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 smithi062 running (6m) 61s ago 6m 56.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 5fe7527872ca 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 smithi079 running (5m) 61s ago 5m 49.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 2e6bd85a1f9f 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 smithi045 running (5m) 42s ago 5m 51.3M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 e3df6e03419c 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 smithi062 running (4m) 61s ago 4m 56.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 0fbf5dc1aac6 2024-04-25T18:58:26.577 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 smithi079 running (4m) 61s ago 4m 56.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 8c1c6f37a70c 2024-04-25T18:58:26.578 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 -- 172.21.15.45:0/767232228 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2b900a3c40 msgr2=0x7f2b900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:26.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2b900a3c40 0x7f2b900a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400faf0 tx=0x7f2b94019040 comp rx=0 tx=0).stop 2024-04-25T18:58:26.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 -- 172.21.15.45:0/767232228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bac102ac0 msgr2=0x7f2bac19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:26.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bac102ac0 0x7f2bac19aed0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00eb90 tx=0x7f2b9c00eea0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 -- 172.21.15.45:0/767232228 shutdown_connections 2024-04-25T18:58:26.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f2b900a3c40 0x7f2b900a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2bac104660 0x7f2bac1a2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.579 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2bac102ac0 0x7f2bac19aed0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 --2- 172.21.15.45:0/767232228 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bac103cc0 0x7f2bac19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:26.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 -- 172.21.15.45:0/767232228 >> 172.21.15.45:0/767232228 conn(0x7f2bac0fe060 msgr2=0x7f2bac074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:26.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 -- 172.21.15.45:0/767232228 shutdown_connections 2024-04-25T18:58:26.580 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:26.578+0000 7f2bb1600700 1 -- 172.21.15.45:0/767232228 wait complete. 2024-04-25T18:58:27.368 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T18:58:27.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:27 smithi045 ceph-mon[27776]: pgmap v419: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:27.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:27 smithi045 ceph-mon[27776]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:27.834 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:27.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:27 smithi062 ceph-mon[32844]: pgmap v419: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:27.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:27 smithi062 ceph-mon[32844]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:27.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:27 smithi079 ceph-mon[33002]: pgmap v419: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:27.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:27 smithi079 ceph-mon[33002]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:29.617 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:29 smithi045 ceph-mon[27776]: pgmap v420: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:29.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:29 smithi062 ceph-mon[32844]: pgmap v420: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:29.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:29 smithi079 ceph-mon[33002]: pgmap v420: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.149+0000 7fe3a8c00700 1 -- 172.21.15.45:0/2122770809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3a410a980 msgr2=0x7fe3a410ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.149+0000 7fe3a8c00700 1 --2- 172.21.15.45:0/2122770809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3a410a980 0x7fe3a410ce10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b0a0 tx=0x7fe39c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.149+0000 7fe3a8c00700 1 -- 172.21.15.45:0/2122770809 shutdown_connections 2024-04-25T18:58:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.149+0000 7fe3a8c00700 1 --2- 172.21.15.45:0/2122770809 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3a410a980 0x7fe3a410ce10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.149+0000 7fe3a8c00700 1 --2- 172.21.15.45:0/2122770809 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a4076a70 0x7fe3a4074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.149+0000 7fe3a8c00700 1 --2- 172.21.15.45:0/2122770809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3a40754a0 0x7fe3a4075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.149+0000 7fe3a8c00700 1 -- 172.21.15.45:0/2122770809 >> 172.21.15.45:0/2122770809 conn(0x7fe3a40fe010 msgr2=0x7fe3a4100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.149+0000 7fe3a8c00700 1 -- 172.21.15.45:0/2122770809 shutdown_connections 2024-04-25T18:58:30.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.150+0000 7fe3a8c00700 1 -- 172.21.15.45:0/2122770809 wait complete. 2024-04-25T18:58:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.151+0000 7fe3a8c00700 1 Processor -- start 2024-04-25T18:58:30.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.151+0000 7fe3a8c00700 1 -- start start 2024-04-25T18:58:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a8c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a40754a0 0x7fe3a419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a8c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3a4076a70 0x7fe3a419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a8c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3a410a980 0x7fe3a41a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a8c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4110790 con 0x7fe3a40754a0 2024-04-25T18:58:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a8c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4110610 con 0x7fe3a410a980 2024-04-25T18:58:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a8c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a4110910 con 0x7fe3a4076a70 2024-04-25T18:58:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a40754a0 0x7fe3a419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:30.153 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a40754a0 0x7fe3a419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38756/0 (socket says 172.21.15.45:38756) 2024-04-25T18:58:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a1e00700 1 -- 172.21.15.45:0/3895192543 learned_addr learned my addr 172.21.15.45:0/3895192543 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a2800700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3a410a980 0x7fe3a41a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:30.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe399400700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3a4076a70 0x7fe3a419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a2800700 1 -- 172.21.15.45:0/3895192543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3a4076a70 msgr2=0x7fe3a419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a2800700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3a4076a70 0x7fe3a419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a2800700 1 -- 172.21.15.45:0/3895192543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a40754a0 msgr2=0x7fe3a419adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a2800700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a40754a0 0x7fe3a419adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.152+0000 7fe3a2800700 1 -- 172.21.15.45:0/3895192543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a41a2a10 con 0x7fe3a410a980 2024-04-25T18:58:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.153+0000 7fe399400700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3a4076a70 0x7fe3a419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.153+0000 7fe3a1e00700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a40754a0 0x7fe3a419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.153+0000 7fe3a2800700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3a410a980 0x7fe3a41a2370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b0a0 tx=0x7fe39c004c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:30.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.153+0000 7fe39b400700 1 -- 172.21.15.45:0/3895192543 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c021070 con 0x7fe3a410a980 2024-04-25T18:58:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.153+0000 7fe39b400700 1 -- 172.21.15.45:0/3895192543 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe39c007b80 con 0x7fe3a410a980 2024-04-25T18:58:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.153+0000 7fe39b400700 1 -- 172.21.15.45:0/3895192543 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c0052a0 con 0x7fe3a410a980 2024-04-25T18:58:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.153+0000 7fe3a8c00700 1 -- 172.21.15.45:0/3895192543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a41a2ca0 con 0x7fe3a410a980 2024-04-25T18:58:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.153+0000 7fe3a8c00700 1 -- 172.21.15.45:0/3895192543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a41a3190 con 0x7fe3a410a980 2024-04-25T18:58:30.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.154+0000 7fe3a8c00700 1 -- 172.21.15.45:0/3895192543 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a4069730 con 0x7fe3a410a980 2024-04-25T18:58:30.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.156+0000 7fe39b400700 1 -- 172.21.15.45:0/3895192543 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe39c024070 con 0x7fe3a410a980 2024-04-25T18:58:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.158+0000 7fe39b400700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe3880a3c40 0x7fe3880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.158+0000 7fe39b400700 1 -- 172.21.15.45:0/3895192543 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fe39c00f020 con 0x7fe3a410a980 2024-04-25T18:58:30.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.158+0000 7fe3a1e00700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe3880a3c40 0x7fe3880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:30.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.159+0000 7fe3a1e00700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe3880a3c40 0x7fe3880a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe394005950 tx=0x7fe394009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:30.163 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.162+0000 7fe39b400700 1 -- 172.21.15.45:0/3895192543 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe39c02b020 con 0x7fe3a410a980 2024-04-25T18:58:30.403 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.402+0000 7fe3a8c00700 1 -- 172.21.15.45:0/3895192543 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fe3a4062e80 con 0x7fe3880a3c40 2024-04-25T18:58:30.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.419+0000 7fe39b400700 1 -- 172.21.15.45:0/3895192543 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fe3a4062e80 con 0x7fe3880a3c40 2024-04-25T18:58:30.420 INFO:teuthology.orchestra.run.smithi045.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T18:58:30.420 INFO:teuthology.orchestra.run.smithi045.stdout:mds.all 12/12 65s ago 4m smithi045=a;smithi045=d;smithi045=g;smithi045=j;smithi062=b;smithi062=e;smithi062=h;smithi062=k;smithi079=c;smithi079=f;smithi079=i;smithi079=l;count:12 2024-04-25T18:58:30.420 INFO:teuthology.orchestra.run.smithi045.stdout:mgr 3/3 65s ago 9m smithi045=x;smithi062=y;smithi079=z;count:3 2024-04-25T18:58:30.421 INFO:teuthology.orchestra.run.smithi045.stdout:mon 3/3 65s ago 10m smithi045:172.21.15.45=a;smithi062:172.21.15.62=b;smithi079:172.21.15.79=c;count:3 2024-04-25T18:58:30.421 INFO:teuthology.orchestra.run.smithi045.stdout:osd 12 65s ago - 2024-04-25T18:58:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.422+0000 7fe3a8c00700 1 -- 172.21.15.45:0/3895192543 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe3880a3c40 msgr2=0x7fe3880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.422+0000 7fe3a8c00700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe3880a3c40 0x7fe3880a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe394005950 tx=0x7fe394009260 comp rx=0 tx=0).stop 2024-04-25T18:58:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.422+0000 7fe3a8c00700 1 -- 172.21.15.45:0/3895192543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3a410a980 msgr2=0x7fe3a41a2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:30.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.422+0000 7fe3a8c00700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3a410a980 0x7fe3a41a2370 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00b0a0 tx=0x7fe39c004c60 comp rx=0 tx=0).stop 2024-04-25T18:58:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.422+0000 7fe3a8c00700 1 -- 172.21.15.45:0/3895192543 shutdown_connections 2024-04-25T18:58:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.422+0000 7fe3a8c00700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fe3880a3c40 0x7fe3880a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.422+0000 7fe3a8c00700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe3a410a980 0x7fe3a41a2370 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.422+0000 7fe3a8c00700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe3a40754a0 0x7fe3a419adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.422+0000 7fe3a8c00700 1 --2- 172.21.15.45:0/3895192543 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3a4076a70 0x7fe3a419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.422+0000 7fe3a8c00700 1 -- 172.21.15.45:0/3895192543 >> 172.21.15.45:0/3895192543 conn(0x7fe3a40fe010 msgr2=0x7fe3a4106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.423+0000 7fe3a8c00700 1 -- 172.21.15.45:0/3895192543 shutdown_connections 2024-04-25T18:58:30.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:30.423+0000 7fe3a8c00700 1 -- 172.21.15.45:0/3895192543 wait complete. 2024-04-25T18:58:31.248 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T18:58:31.728 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:31.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:31 smithi062 ceph-mon[32844]: pgmap v421: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:31.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:31 smithi062 ceph-mon[32844]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:31.883 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:31 smithi045 ceph-mon[27776]: pgmap v421: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:31.883 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:31 smithi045 ceph-mon[27776]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:31.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:31 smithi079 ceph-mon[33002]: pgmap v421: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:31.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:31 smithi079 ceph-mon[33002]: from='client.24932 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:33.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:33 smithi045 ceph-mon[27776]: pgmap v422: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:33.736 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:33 smithi079 ceph-mon[33002]: pgmap v422: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:33.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:33 smithi062 ceph-mon[32844]: pgmap v422: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.975+0000 7f0d56800700 1 -- 172.21.15.45:0/1232514695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d501090e0 msgr2=0x7f0d50109500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.975+0000 7f0d56800700 1 --2- 172.21.15.45:0/1232514695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d501090e0 0x7f0d50109500 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0d400099c0 tx=0x7f0d40009cd0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.975+0000 7f0d56800700 1 -- 172.21.15.45:0/1232514695 shutdown_connections 2024-04-25T18:58:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.975+0000 7f0d56800700 1 --2- 172.21.15.45:0/1232514695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d5010ac80 0x7f0d50111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.975+0000 7f0d56800700 1 --2- 172.21.15.45:0/1232514695 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d501090e0 0x7f0d50109500 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.975+0000 7f0d56800700 1 --2- 172.21.15.45:0/1232514695 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d5010a2e0 0x7f0d5010a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.975+0000 7f0d56800700 1 -- 172.21.15.45:0/1232514695 >> 172.21.15.45:0/1232514695 conn(0x7f0d500780b0 msgr2=0x7f0d5007a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.975+0000 7f0d56800700 1 -- 172.21.15.45:0/1232514695 shutdown_connections 2024-04-25T18:58:33.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.975+0000 7f0d56800700 1 -- 172.21.15.45:0/1232514695 wait complete. 2024-04-25T18:58:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d56800700 1 Processor -- start 2024-04-25T18:58:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d56800700 1 -- start start 2024-04-25T18:58:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d56800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d501090e0 0x7f0d5019f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d56800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d5010a2e0 0x7f0d5019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d56800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d5010ac80 0x7f0d501a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d56800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50114bd0 con 0x7f0d5010a2e0 2024-04-25T18:58:33.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d56800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50114a50 con 0x7f0d501090e0 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d56800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0d50114d50 con 0x7f0d5010ac80 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d46a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d5010a2e0 0x7f0d5019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d46a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d5010a2e0 0x7f0d5019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:38772/0 (socket says 172.21.15.45:38772) 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d46a00700 1 -- 172.21.15.45:0/2010798147 learned_addr learned my addr 172.21.15.45:0/2010798147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d4fe00700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d5010ac80 0x7f0d501a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.977+0000 7f0d4f400700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d501090e0 0x7f0d5019f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.978+0000 7f0d46a00700 1 -- 172.21.15.45:0/2010798147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d5010ac80 msgr2=0x7f0d501a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.978+0000 7f0d46a00700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d5010ac80 0x7f0d501a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.978+0000 7f0d46a00700 1 -- 172.21.15.45:0/2010798147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d501090e0 msgr2=0x7f0d5019f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.978+0000 7f0d46a00700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d501090e0 0x7f0d5019f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.978+0000 7f0d46a00700 1 -- 172.21.15.45:0/2010798147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d501a6e70 con 0x7f0d5010a2e0 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.978+0000 7f0d4f400700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d501090e0 0x7f0d5019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:33.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.978+0000 7f0d46a00700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d5010a2e0 0x7f0d5019f750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c3a0 tx=0x7f0d3800c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:33.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.979+0000 7f0d4cc00700 1 -- 172.21.15.45:0/2010798147 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3800ce20 con 0x7f0d5010a2e0 2024-04-25T18:58:33.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.979+0000 7f0d4cc00700 1 -- 172.21.15.45:0/2010798147 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d380090d0 con 0x7f0d5010a2e0 2024-04-25T18:58:33.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.979+0000 7f0d4cc00700 1 -- 172.21.15.45:0/2010798147 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38004700 con 0x7f0d5010a2e0 2024-04-25T18:58:33.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.979+0000 7f0d56800700 1 -- 172.21.15.45:0/2010798147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d501a7100 con 0x7f0d5010a2e0 2024-04-25T18:58:33.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.979+0000 7f0d56800700 1 -- 172.21.15.45:0/2010798147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501a75f0 con 0x7f0d5010a2e0 2024-04-25T18:58:33.982 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.980+0000 7f0d56800700 1 -- 172.21.15.45:0/2010798147 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d50069730 con 0x7f0d5010a2e0 2024-04-25T18:58:33.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.982+0000 7f0d4cc00700 1 -- 172.21.15.45:0/2010798147 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0d3801a020 con 0x7f0d5010a2e0 2024-04-25T18:58:33.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.984+0000 7f0d4cc00700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0d340a3cc0 0x7f0d340a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.984+0000 7f0d4cc00700 1 -- 172.21.15.45:0/2010798147 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f0d380cfda0 con 0x7f0d5010a2e0 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.984+0000 7f0d4f400700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0d340a3cc0 0x7f0d340a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:33.986 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.985+0000 7f0d4f400700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0d340a3cc0 0x7f0d340a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b580 tx=0x7f0d40005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:33.991 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:33.989+0000 7f0d4cc00700 1 -- 172.21.15.45:0/2010798147 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0d38010040 con 0x7f0d5010a2e0 2024-04-25T18:58:34.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.226+0000 7f0d56800700 1 -- 172.21.15.45:0/2010798147 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f0d50062e80 con 0x7f0d340a3cc0 2024-04-25T18:58:34.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.230+0000 7f0d4cc00700 1 -- 172.21.15.45:0/2010798147 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f0d50062e80 con 0x7f0d340a3cc0 2024-04-25T18:58:34.232 INFO:teuthology.orchestra.run.smithi045.stdout:HOST ADDR LABELS STATUS 2024-04-25T18:58:34.232 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 172.21.15.45 2024-04-25T18:58:34.232 INFO:teuthology.orchestra.run.smithi045.stdout:smithi062 172.21.15.62 2024-04-25T18:58:34.233 INFO:teuthology.orchestra.run.smithi045.stdout:smithi079 172.21.15.79 2024-04-25T18:58:34.233 INFO:teuthology.orchestra.run.smithi045.stdout:3 hosts in cluster 2024-04-25T18:58:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 -- 172.21.15.45:0/2010798147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0d340a3cc0 msgr2=0x7f0d340a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:34.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0d340a3cc0 0x7f0d340a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b580 tx=0x7f0d40005fb0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 -- 172.21.15.45:0/2010798147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d5010a2e0 msgr2=0x7f0d5019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d5010a2e0 0x7f0d5019f750 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800c3a0 tx=0x7f0d3800c6b0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 -- 172.21.15.45:0/2010798147 shutdown_connections 2024-04-25T18:58:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f0d340a3cc0 0x7f0d340a6180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0d501090e0 0x7f0d5019f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d5010ac80 0x7f0d501a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 --2- 172.21.15.45:0/2010798147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0d5010a2e0 0x7f0d5019f750 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 -- 172.21.15.45:0/2010798147 >> 172.21.15.45:0/2010798147 conn(0x7f0d500780b0 msgr2=0x7f0d5010d5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.233+0000 7f0d56800700 1 -- 172.21.15.45:0/2010798147 shutdown_connections 2024-04-25T18:58:34.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:34.234+0000 7f0d56800700 1 -- 172.21.15.45:0/2010798147 wait complete. 2024-04-25T18:58:35.125 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T18:58:35.606 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:35.752 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:35 smithi045 ceph-mon[27776]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:35.753 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:35 smithi045 ceph-mon[27776]: pgmap v423: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:35.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:35 smithi062 ceph-mon[32844]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:35.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:35 smithi062 ceph-mon[32844]: pgmap v423: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:35.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:35 smithi079 ceph-mon[33002]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:35.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:35 smithi079 ceph-mon[33002]: pgmap v423: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:37.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.624+0000 7fc015e00700 1 -- 172.21.15.45:0/2224265882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc010104580 msgr2=0x7fc01010ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:37.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.624+0000 7fc015e00700 1 --2- 172.21.15.45:0/2224265882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc010104580 0x7fc01010ae30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc00800b0d0 tx=0x7fc00800b3e0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.624+0000 7fc015e00700 1 -- 172.21.15.45:0/2224265882 shutdown_connections 2024-04-25T18:58:37.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.624+0000 7fc015e00700 1 --2- 172.21.15.45:0/2224265882 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc010104580 0x7fc01010ae30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.624+0000 7fc015e00700 1 --2- 172.21.15.45:0/2224265882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0101029e0 0x7fc010102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.624+0000 7fc015e00700 1 --2- 172.21.15.45:0/2224265882 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc010103be0 0x7fc010104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.624+0000 7fc015e00700 1 -- 172.21.15.45:0/2224265882 >> 172.21.15.45:0/2224265882 conn(0x7fc0100fdf80 msgr2=0x7fc0101003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:37.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.625+0000 7fc015e00700 1 -- 172.21.15.45:0/2224265882 shutdown_connections 2024-04-25T18:58:37.628 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.625+0000 7fc015e00700 1 -- 172.21.15.45:0/2224265882 wait complete. 2024-04-25T18:58:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.626+0000 7fc015e00700 1 Processor -- start 2024-04-25T18:58:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.626+0000 7fc015e00700 1 -- start start 2024-04-25T18:58:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.626+0000 7fc015e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0101029e0 0x7fc01019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.626+0000 7fc015e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc010103be0 0x7fc01019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.626+0000 7fc015e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc010104580 0x7fc0101a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.626+0000 7fc015e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc01010e460 con 0x7fc010104580 2024-04-25T18:58:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.626+0000 7fc015e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc01010e2e0 con 0x7fc0101029e0 2024-04-25T18:58:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.626+0000 7fc015e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc01010e5e0 con 0x7fc010103be0 2024-04-25T18:58:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc014a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0101029e0 0x7fc01019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:37.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc00fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc010103be0 0x7fc01019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc014a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0101029e0 0x7fc01019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:45886/0 (socket says 172.21.15.45:45886) 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc014a00700 1 -- 172.21.15.45:0/2204775395 learned_addr learned my addr 172.21.15.45:0/2204775395 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc00fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc010103be0 0x7fc01019b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:38260/0 (socket says 172.21.15.45:38260) 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc015400700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc010104580 0x7fc0101a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc014a00700 1 -- 172.21.15.45:0/2204775395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc010103be0 msgr2=0x7fc01019b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc014a00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc010103be0 0x7fc01019b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc014a00700 1 -- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc010104580 msgr2=0x7fc0101a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc014a00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc010104580 0x7fc0101a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc014a00700 1 -- 172.21.15.45:0/2204775395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0101a2960 con 0x7fc0101029e0 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc015400700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc010104580 0x7fc0101a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc00fe00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc010103be0 0x7fc01019b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.627+0000 7fc014a00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0101029e0 0x7fc01019ad10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc00400e970 tx=0x7fc00400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:37.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.628+0000 7fc00d600700 1 -- 172.21.15.45:0/2204775395 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00400cbd0 con 0x7fc0101029e0 2024-04-25T18:58:37.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.628+0000 7fc015e00700 1 -- 172.21.15.45:0/2204775395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0101a2c50 con 0x7fc0101029e0 2024-04-25T18:58:37.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.628+0000 7fc00d600700 1 -- 172.21.15.45:0/2204775395 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0040044d0 con 0x7fc0101029e0 2024-04-25T18:58:37.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.628+0000 7fc00d600700 1 -- 172.21.15.45:0/2204775395 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc004003df0 con 0x7fc0101029e0 2024-04-25T18:58:37.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.628+0000 7fc015e00700 1 -- 172.21.15.45:0/2204775395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0101a3190 con 0x7fc0101029e0 2024-04-25T18:58:37.632 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.629+0000 7fc015e00700 1 -- 172.21.15.45:0/2204775395 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc010069730 con 0x7fc0101029e0 2024-04-25T18:58:37.633 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.631+0000 7fc00d600700 1 -- 172.21.15.45:0/2204775395 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc004003f90 con 0x7fc0101029e0 2024-04-25T18:58:37.634 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.632+0000 7fc00d600700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3c40 0x7fbff80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:37.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.633+0000 7fc00d600700 1 -- 172.21.15.45:0/2204775395 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fc0040d08a0 con 0x7fc0101029e0 2024-04-25T18:58:37.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.633+0000 7fc00fe00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3c40 0x7fbff80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:37.635 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.633+0000 7fc00fe00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3c40 0x7fbff80a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009680 tx=0x7fbffc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:37.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.635+0000 7fc00d600700 1 -- 172.21.15.45:0/2204775395 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc00409d200 con 0x7fc0101029e0 2024-04-25T18:58:37.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:37 smithi045 ceph-mon[27776]: pgmap v424: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:37.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:37 smithi062 ceph-mon[32844]: pgmap v424: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:37.875 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.873+0000 7fc015e00700 1 -- 172.21.15.45:0/2204775395 --> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc0101a33c0 con 0x7fbff80a3c40 2024-04-25T18:58:37.879 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.877+0000 7fc00d600700 1 -- 172.21.15.45:0/2204775395 <== mgr.14152 v2:172.21.15.45:6800/943541413 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc0101a33c0 con 0x7fbff80a3c40 2024-04-25T18:58:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T18:58:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 5m ago Has a FileSystem, LVM detected 2024-04-25T18:58:37.879 INFO:teuthology.orchestra.run.smithi045.stdout:smithi062 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440094400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-25T18:58:37.880 INFO:teuthology.orchestra.run.smithi045.stdout:smithi079 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013G400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-25T18:58:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 -- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3c40 msgr2=0x7fbff80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3c40 0x7fbff80a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009680 tx=0x7fbffc009260 comp rx=0 tx=0).stop 2024-04-25T18:58:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 -- 172.21.15.45:0/2204775395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0101029e0 msgr2=0x7fc01019ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:37.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0101029e0 0x7fc01019ad10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc00400e970 tx=0x7fc00400ec80 comp rx=0 tx=0).stop 2024-04-25T18:58:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 -- 172.21.15.45:0/2204775395 shutdown_connections 2024-04-25T18:58:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fbff80a3c40 0x7fbff80a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0101029e0 0x7fc01019ad10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc010104580 0x7fc0101a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 --2- 172.21.15.45:0/2204775395 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc010103be0 0x7fc01019b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 -- 172.21.15.45:0/2204775395 >> 172.21.15.45:0/2204775395 conn(0x7fc0100fdf80 msgr2=0x7fc010074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 -- 172.21.15.45:0/2204775395 shutdown_connections 2024-04-25T18:58:37.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:37.880+0000 7fc015e00700 1 -- 172.21.15.45:0/2204775395 wait complete. 2024-04-25T18:58:37.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:37 smithi079 ceph-mon[33002]: pgmap v424: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:38.633 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:38 smithi045 ceph-mon[27776]: from='client.24944 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:38.712 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T18:58:38.723 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T18:58:38.723 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T18:58:38.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:38 smithi062 ceph-mon[32844]: from='client.24944 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:38.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:38 smithi079 ceph-mon[33002]: from='client.24944 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:58:39.196 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:39.687 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:39 smithi045 ceph-mon[27776]: pgmap v425: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:39.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:39 smithi062 ceph-mon[32844]: pgmap v425: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:39.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:39 smithi079 ceph-mon[33002]: pgmap v425: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:41.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.470+0000 7f3959e00700 1 -- 172.21.15.45:0/1711532166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3954103cf0 msgr2=0x7f3954104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:41.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.470+0000 7f3959e00700 1 --2- 172.21.15.45:0/1711532166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3954103cf0 0x7f3954104150 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3940009990 tx=0x7f3940009ca0 comp rx=0 tx=0).stop 2024-04-25T18:58:41.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.470+0000 7f3959e00700 1 -- 172.21.15.45:0/1711532166 shutdown_connections 2024-04-25T18:58:41.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.470+0000 7f3959e00700 1 --2- 172.21.15.45:0/1711532166 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3954104690 0x7f395410af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:41.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.470+0000 7f3959e00700 1 --2- 172.21.15.45:0/1711532166 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954102af0 0x7f3954102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:41.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.470+0000 7f3959e00700 1 --2- 172.21.15.45:0/1711532166 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3954103cf0 0x7f3954104150 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:41.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.470+0000 7f3959e00700 1 -- 172.21.15.45:0/1711532166 >> 172.21.15.45:0/1711532166 conn(0x7f39540fe090 msgr2=0x7f39541004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:41.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.470+0000 7f3959e00700 1 -- 172.21.15.45:0/1711532166 shutdown_connections 2024-04-25T18:58:41.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.470+0000 7f3959e00700 1 -- 172.21.15.45:0/1711532166 wait complete. 2024-04-25T18:58:41.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.471+0000 7f3959e00700 1 Processor -- start 2024-04-25T18:58:41.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.472+0000 7f3959e00700 1 -- start start 2024-04-25T18:58:41.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.472+0000 7f3959e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3954102af0 0x7f3954198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:41.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.472+0000 7f3959e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3954103cf0 0x7f3954199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:41.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.472+0000 7f3959e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954104690 0x7f39541a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:41.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.472+0000 7f3959e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f395410e5b0 con 0x7f3954104690 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.472+0000 7f3959e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f395410e430 con 0x7f3954102af0 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.472+0000 7f3959e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f395410e730 con 0x7f3954103cf0 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3954102af0 0x7f3954198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3954103cf0 0x7f3954199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3953400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954104690 0x7f39541a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3954102af0 0x7f3954198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:45900/0 (socket says 172.21.15.45:45900) 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3954103cf0 0x7f3954199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:38284/0 (socket says 172.21.15.45:38284) 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952a00700 1 -- 172.21.15.45:0/3490268234 learned_addr learned my addr 172.21.15.45:0/3490268234 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952a00700 1 -- 172.21.15.45:0/3490268234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3954103cf0 msgr2=0x7f3954199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952a00700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3954103cf0 0x7f3954199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952a00700 1 -- 172.21.15.45:0/3490268234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954104690 msgr2=0x7f39541a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952a00700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954104690 0x7f39541a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952a00700 1 -- 172.21.15.45:0/3490268234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39541a0890 con 0x7f3954102af0 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3952000700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3954103cf0 0x7f3954199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:41.476 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.473+0000 7f3953400700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954104690 0x7f39541a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:41.477 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.474+0000 7f3952a00700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3954102af0 0x7f3954198c40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f394400e970 tx=0x7f394400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:41.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.474+0000 7f3958a00700 1 -- 172.21.15.45:0/3490268234 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944004220 con 0x7f3954102af0 2024-04-25T18:58:41.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.474+0000 7f3958a00700 1 -- 172.21.15.45:0/3490268234 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39440043c0 con 0x7f3954102af0 2024-04-25T18:58:41.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.474+0000 7f3958a00700 1 -- 172.21.15.45:0/3490268234 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3944003ce0 con 0x7f3954102af0 2024-04-25T18:58:41.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.474+0000 7f3959e00700 1 -- 172.21.15.45:0/3490268234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39541a0b80 con 0x7f3954102af0 2024-04-25T18:58:41.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.474+0000 7f3959e00700 1 -- 172.21.15.45:0/3490268234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39541a1070 con 0x7f3954102af0 2024-04-25T18:58:41.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.475+0000 7f3959e00700 1 -- 172.21.15.45:0/3490268234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3954069730 con 0x7f3954102af0 2024-04-25T18:58:41.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.477+0000 7f3958a00700 1 -- 172.21.15.45:0/3490268234 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f394400c7e0 con 0x7f3954102af0 2024-04-25T18:58:41.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.479+0000 7f3958a00700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f39380a3c40 0x7f39380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.479+0000 7f3958a00700 1 -- 172.21.15.45:0/3490268234 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f39440d0790 con 0x7f3954102af0 2024-04-25T18:58:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.479+0000 7f3952000700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f39380a3c40 0x7f39380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:41.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.480+0000 7f3952000700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f39380a3c40 0x7f39380a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3940000c00 tx=0x7f394001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:41.483 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.481+0000 7f3958a00700 1 -- 172.21.15.45:0/3490268234 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f394409d180 con 0x7f3954102af0 2024-04-25T18:58:41.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:41 smithi045 ceph-mon[27776]: pgmap v426: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:41.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.763+0000 7f3959e00700 1 -- 172.21.15.45:0/3490268234 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f395402e1d0 con 0x7f3954102af0 2024-04-25T18:58:41.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.764+0000 7f3958a00700 1 -- 172.21.15.45:0/3490268234 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3038 (secure 0 0 0) 0x7f394409ca80 con 0x7f3954102af0 2024-04-25T18:58:41.767 INFO:teuthology.orchestra.run.smithi045.stdout:e25 2024-04-25T18:58:41.767 INFO:teuthology.orchestra.run.smithi045.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T18:58:41.767 INFO:teuthology.orchestra.run.smithi045.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T18:58:41.767 INFO:teuthology.orchestra.run.smithi045.stdout:legacy client fscid: 1 2024-04-25T18:58:41.767 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:58:41.767 INFO:teuthology.orchestra.run.smithi045.stdout:Filesystem 'cephfs' (1) 2024-04-25T18:58:41.767 INFO:teuthology.orchestra.run.smithi045.stdout:fs_name cephfs 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 25 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-04-25T18:55:26.025536+0000 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-04-25T18:55:50.062944+0000 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:tableserver 0 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:root 0 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:session_timeout 300 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:session_autoclose 300 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:max_file_size 1099511627776 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:max_xattr_size 65536 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:required_client_features {} 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure 0 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:last_failure_osd_epoch 0 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:max_mds 3 2024-04-25T18:58:41.768 INFO:teuthology.orchestra.run.smithi045.stdout:in 0,1,2 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:up {0=24524,1=24512,2=24505} 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:failed 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:damaged 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:stopped 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:data_pools [3] 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:metadata_pool 2 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:inline_data disabled 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:balancer 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:bal_rank_mask -1 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:standby_count_wanted 0 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:qdb_cluster leader: 24524 members: 24524,24505,24512 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.b{0:24524} state up:active seq 7 addr [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.c{0:24493} state up:standby-replay seq 1 addr [v2:172.21.15.79:6836/3290580293,v1:172.21.15.79:6837/3290580293] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.a{1:24512} state up:active seq 11 addr [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.769 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.h{1:24485} state up:standby-replay seq 1 addr [v2:172.21.15.62:6836/2985890701,v1:172.21.15.62:6837/2985890701] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.i{2:24505} state up:active seq 14 addr [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.f{2:24481} state up:standby-replay seq 1 addr [v2:172.21.15.79:6834/1201787807,v1:172.21.15.79:6835/1201787807] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout:Standby daemons: 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.j{-1:14364} state up:standby seq 1 addr [v2:172.21.15.45:6834/4023030153,v1:172.21.15.45:6835/4023030153] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.g{-1:14400} state up:standby seq 1 addr [v2:172.21.15.45:6838/3080448067,v1:172.21.15.45:6839/3080448067] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.k{-1:24434} state up:standby seq 1 addr [v2:172.21.15.62:6832/1072807299,v1:172.21.15.62:6833/1072807299] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.l{-1:24445} state up:standby seq 1 addr [v2:172.21.15.79:6832/990407858,v1:172.21.15.79:6833/990407858] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.d{-1:24452} state up:standby seq 1 addr [v2:172.21.15.45:6836/496487622,v1:172.21.15.45:6837/496487622] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.770 INFO:teuthology.orchestra.run.smithi045.stdout:[mds.e{-1:24464} state up:standby seq 1 addr [v2:172.21.15.62:6834/3037892802,v1:172.21.15.62:6835/3037892802] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T18:58:41.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.768+0000 7f3959e00700 1 -- 172.21.15.45:0/3490268234 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f39380a3c40 msgr2=0x7f39380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:41.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.768+0000 7f3959e00700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f39380a3c40 0x7f39380a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3940000c00 tx=0x7f394001a040 comp rx=0 tx=0).stop 2024-04-25T18:58:41.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.768+0000 7f3959e00700 1 -- 172.21.15.45:0/3490268234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3954102af0 msgr2=0x7f3954198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:41.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.768+0000 7f3959e00700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3954102af0 0x7f3954198c40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f394400e970 tx=0x7f394400ec80 comp rx=0 tx=0).stop 2024-04-25T18:58:41.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.769+0000 7f3959e00700 1 -- 172.21.15.45:0/3490268234 shutdown_connections 2024-04-25T18:58:41.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.769+0000 7f3959e00700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f39380a3c40 0x7f39380a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:41.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.769+0000 7f3959e00700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3954102af0 0x7f3954198c40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:41.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.769+0000 7f3959e00700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3954104690 0x7f39541a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:41.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.769+0000 7f3959e00700 1 --2- 172.21.15.45:0/3490268234 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3954103cf0 0x7f3954199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:41.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.769+0000 7f3959e00700 1 -- 172.21.15.45:0/3490268234 >> 172.21.15.45:0/3490268234 conn(0x7f39540fe090 msgr2=0x7f3954107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:41.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.769+0000 7f3959e00700 1 -- 172.21.15.45:0/3490268234 shutdown_connections 2024-04-25T18:58:41.772 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:41.769+0000 7f3959e00700 1 -- 172.21.15.45:0/3490268234 wait complete. 2024-04-25T18:58:41.772 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-25T18:58:41.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:41 smithi062 ceph-mon[32844]: pgmap v426: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:41.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:41 smithi079 ceph-mon[33002]: pgmap v426: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:42.529 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T18:58:42.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:42 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3490268234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T18:58:42.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:42 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3490268234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T18:58:42.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:42 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3490268234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T18:58:42.997 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:43.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:43 smithi062 ceph-mon[32844]: pgmap v427: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:43.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:43 smithi079 ceph-mon[33002]: pgmap v427: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:43.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:43 smithi045 ceph-mon[27776]: pgmap v427: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:45.222 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.220+0000 7f67dac00700 1 -- 172.21.15.45:0/2393554240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67d410a980 msgr2=0x7f67d410ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:45.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.220+0000 7f67dac00700 1 --2- 172.21.15.45:0/2393554240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67d410a980 0x7f67d410ce10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009f90 tx=0x7f67cc00b330 comp rx=0 tx=0).stop 2024-04-25T18:58:45.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.220+0000 7f67dac00700 1 -- 172.21.15.45:0/2393554240 shutdown_connections 2024-04-25T18:58:45.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.220+0000 7f67dac00700 1 --2- 172.21.15.45:0/2393554240 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67d410a980 0x7f67d410ce10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:45.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.220+0000 7f67dac00700 1 --2- 172.21.15.45:0/2393554240 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d4076a10 0x7f67d4074e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:45.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.220+0000 7f67dac00700 1 --2- 172.21.15.45:0/2393554240 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d4075440 0x7f67d40758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:45.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.220+0000 7f67dac00700 1 -- 172.21.15.45:0/2393554240 >> 172.21.15.45:0/2393554240 conn(0x7f67d40fe040 msgr2=0x7f67d4100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:45.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.220+0000 7f67dac00700 1 -- 172.21.15.45:0/2393554240 shutdown_connections 2024-04-25T18:58:45.223 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.220+0000 7f67dac00700 1 -- 172.21.15.45:0/2393554240 wait complete. 2024-04-25T18:58:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.221+0000 7f67dac00700 1 Processor -- start 2024-04-25T18:58:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.221+0000 7f67dac00700 1 -- start start 2024-04-25T18:58:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67dac00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d4075440 0x7f67d419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67dac00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67d4076a10 0x7f67d419b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67dac00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d410a980 0x7f67d41a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67dac00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f67d4110770 con 0x7f67d4075440 2024-04-25T18:58:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67dac00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f67d41105f0 con 0x7f67d410a980 2024-04-25T18:58:45.224 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67dac00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f67d41108f0 con 0x7f67d4076a10 2024-04-25T18:58:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67d3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d4075440 0x7f67d419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67d8c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d410a980 0x7f67d41a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67d3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d4075440 0x7f67d419add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39030/0 (socket says 172.21.15.45:39030) 2024-04-25T18:58:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67d3e00700 1 -- 172.21.15.45:0/3043522571 learned_addr learned my addr 172.21.15.45:0/3043522571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:45.225 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.222+0000 7f67cb400700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67d4076a10 0x7f67d419b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67cb400700 1 -- 172.21.15.45:0/3043522571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d410a980 msgr2=0x7f67d41a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67cb400700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d410a980 0x7f67d41a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67cb400700 1 -- 172.21.15.45:0/3043522571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d4075440 msgr2=0x7f67d419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67cb400700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d4075440 0x7f67d419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67cb400700 1 -- 172.21.15.45:0/3043522571 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d41a2a30 con 0x7f67d4076a10 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67d3e00700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d4075440 0x7f67d419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67cb400700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67d4076a10 0x7f67d419b310 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00ea20 tx=0x7f67bc00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67d1600700 1 -- 172.21.15.45:0/3043522571 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc00cbd0 con 0x7f67d4076a10 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67d1600700 1 -- 172.21.15.45:0/3043522571 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67bc00ee60 con 0x7f67d4076a10 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67d1600700 1 -- 172.21.15.45:0/3043522571 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc005020 con 0x7f67d4076a10 2024-04-25T18:58:45.226 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.223+0000 7f67dac00700 1 -- 172.21.15.45:0/3043522571 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d41a2d20 con 0x7f67d4076a10 2024-04-25T18:58:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.224+0000 7f67dac00700 1 -- 172.21.15.45:0/3043522571 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d41a31b0 con 0x7f67d4076a10 2024-04-25T18:58:45.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.225+0000 7f67dac00700 1 -- 172.21.15.45:0/3043522571 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d4069730 con 0x7f67d4076a10 2024-04-25T18:58:45.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.227+0000 7f67d1600700 1 -- 172.21.15.45:0/3043522571 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f67bc0051c0 con 0x7f67d4076a10 2024-04-25T18:58:45.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.228+0000 7f67d1600700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f67b80a3c40 0x7f67b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.228+0000 7f67d1600700 1 -- 172.21.15.45:0/3043522571 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f67bc0d03b0 con 0x7f67d4076a10 2024-04-25T18:58:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.229+0000 7f67d3e00700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f67b80a3c40 0x7f67b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:45.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.229+0000 7f67d3e00700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f67b80a3c40 0x7f67b80a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f67c4005950 tx=0x7f67c4009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:45.234 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.233+0000 7f67d1600700 1 -- 172.21.15.45:0/3043522571 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67bc09cd10 con 0x7f67d4076a10 2024-04-25T18:58:45.474 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.472+0000 7f67dac00700 1 -- 172.21.15.45:0/3043522571 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f67d4063650 con 0x7f67d4076a10 2024-04-25T18:58:45.475 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.473+0000 7f67d1600700 1 -- 172.21.15.45:0/3043522571 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v94) ==== 56+0+5307 (secure 0 0 0) 0x7f67bc09c610 con 0x7f67d4076a10 2024-04-25T18:58:45.476 INFO:teuthology.orchestra.run.smithi045.stdout:epoch 94 2024-04-25T18:58:45.476 INFO:teuthology.orchestra.run.smithi045.stdout:fsid ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T18:58:45.476 INFO:teuthology.orchestra.run.smithi045.stdout:created 2024-04-25T18:46:09.147410+0000 2024-04-25T18:58:45.476 INFO:teuthology.orchestra.run.smithi045.stdout:modified 2024-04-25T18:56:17.531513+0000 2024-04-25T18:58:45.476 INFO:teuthology.orchestra.run.smithi045.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T18:58:45.476 INFO:teuthology.orchestra.run.smithi045.stdout:crush_version 26 2024-04-25T18:58:45.476 INFO:teuthology.orchestra.run.smithi045.stdout:full_ratio 0.95 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:backfillfull_ratio 0.9 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:nearfull_ratio 0.85 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:require_min_compat_client luminous 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:min_compat_client luminous 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:require_osd_release squid 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:stretch_mode_enabled false 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 92 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 92 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:max_osd 12 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:osd.0 up in weight 1 up_from 8 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6802/4137498586,v1:172.21.15.45:6803/4137498586] [v2:172.21.15.45:6804/4137498586,v1:172.21.15.45:6805/4137498586] exists,up ee46df06-8c1b-43ba-a8d7-c8c7a23e8995 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:osd.1 up in weight 1 up_from 13 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6800/237724307,v1:172.21.15.62:6801/237724307] [v2:172.21.15.62:6802/237724307,v1:172.21.15.62:6803/237724307] exists,up 550faceb-9f8f-4beb-8dd2-e0c6b4968c5e 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6800/2930472364,v1:172.21.15.79:6801/2930472364] [v2:172.21.15.79:6802/2930472364,v1:172.21.15.79:6803/2930472364] exists,up 30f605cc-9a0c-4410-8d6d-299c2413482b 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:osd.3 up in weight 1 up_from 27 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6810/3944994121,v1:172.21.15.45:6811/3944994121] [v2:172.21.15.45:6812/3944994121,v1:172.21.15.45:6813/3944994121] exists,up 2cab3d66-da4b-4960-bf80-8d198d0eac8d 2024-04-25T18:58:45.477 INFO:teuthology.orchestra.run.smithi045.stdout:osd.4 up in weight 1 up_from 35 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6808/2453949627,v1:172.21.15.62:6809/2453949627] [v2:172.21.15.62:6810/2453949627,v1:172.21.15.62:6811/2453949627] exists,up ac287cf8-20dd-477f-8877-966a1d4e269b 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:osd.5 up in weight 1 up_from 40 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6808/188682646,v1:172.21.15.79:6809/188682646] [v2:172.21.15.79:6810/188682646,v1:172.21.15.79:6811/188682646] exists,up c258a3e7-ef41-4ad5-a9b0-689eb2761171 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:osd.6 up in weight 1 up_from 48 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6818/401127899,v1:172.21.15.45:6819/401127899] [v2:172.21.15.45:6820/401127899,v1:172.21.15.45:6821/401127899] exists,up 9b1c752c-c246-4c8c-adee-e8948e1c2978 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:osd.7 up in weight 1 up_from 54 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6816/3708428772,v1:172.21.15.62:6817/3708428772] [v2:172.21.15.62:6818/3708428772,v1:172.21.15.62:6819/3708428772] exists,up eb5a094c-22a9-4e39-9014-57094699afec 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:osd.8 up in weight 1 up_from 62 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6816/2481950968,v1:172.21.15.79:6817/2481950968] [v2:172.21.15.79:6818/2481950968,v1:172.21.15.79:6819/2481950968] exists,up b9e1829d-cafc-47cf-ba9b-1dc60e0c6421 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:osd.9 up in weight 1 up_from 70 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6826/2318794689,v1:172.21.15.45:6827/2318794689] [v2:172.21.15.45:6828/2318794689,v1:172.21.15.45:6829/2318794689] exists,up f783937a-cdd7-4310-a6b4-00cf934db047 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:osd.10 up in weight 1 up_from 76 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.62:6824/2698516716,v1:172.21.15.62:6825/2698516716] [v2:172.21.15.62:6826/2698516716,v1:172.21.15.62:6827/2698516716] exists,up bf2abd19-2916-4e72-ace4-4fc3a72ef06c 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:osd.11 up in weight 1 up_from 84 up_thru 93 down_at 0 last_clean_interval [0,0) [v2:172.21.15.79:6824/1965921727,v1:172.21.15.79:6825/1965921727] [v2:172.21.15.79:6826/1965921727,v1:172.21.15.79:6827/1965921727] exists,up 197c940c-1050-4bff-b88c-86b6a4da0cf5 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.13 [2,11] 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 2.1c [8,11] 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.19 [3,8] 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.1b [4,8] 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.1e [5,6] 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.23 [4,7] 2024-04-25T18:58:45.478 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.28 [5,6] 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.34 [4,9] 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.38 [3,9] 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:pg_upmap_items 3.3c [4,9] 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1117554794 expires 2024-04-26T18:47:16.094556+0000 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/3065212608 expires 2024-04-26T18:47:16.094556+0000 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/2122448921 expires 2024-04-26T18:46:42.822772+0000 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1563847530 expires 2024-04-26T18:47:16.094556+0000 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/817529633 expires 2024-04-26T18:47:16.094556+0000 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/1502518510 expires 2024-04-26T18:46:42.822772+0000 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6800/1404432740 expires 2024-04-26T18:46:42.822772+0000 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/3065212608 expires 2024-04-26T18:47:16.094556+0000 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:0/3157265170 expires 2024-04-26T18:46:42.822772+0000 2024-04-25T18:58:45.479 INFO:teuthology.orchestra.run.smithi045.stdout:blocklist 172.21.15.45:6801/1404432740 expires 2024-04-26T18:46:42.822772+0000 2024-04-25T18:58:45.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.477+0000 7f67dac00700 1 -- 172.21.15.45:0/3043522571 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f67b80a3c40 msgr2=0x7f67b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:45.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.477+0000 7f67dac00700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f67b80a3c40 0x7f67b80a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f67c4005950 tx=0x7f67c4009310 comp rx=0 tx=0).stop 2024-04-25T18:58:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.477+0000 7f67dac00700 1 -- 172.21.15.45:0/3043522571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67d4076a10 msgr2=0x7f67d419b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.477+0000 7f67dac00700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67d4076a10 0x7f67d419b310 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f67bc00ea20 tx=0x7f67bc00ed30 comp rx=0 tx=0).stop 2024-04-25T18:58:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.478+0000 7f67dac00700 1 -- 172.21.15.45:0/3043522571 shutdown_connections 2024-04-25T18:58:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.478+0000 7f67dac00700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f67b80a3c40 0x7f67b80a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.478+0000 7f67dac00700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f67d410a980 0x7f67d41a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.478+0000 7f67dac00700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f67d4075440 0x7f67d419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.478+0000 7f67dac00700 1 --2- 172.21.15.45:0/3043522571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f67d4076a10 0x7f67d419b310 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.478+0000 7f67dac00700 1 -- 172.21.15.45:0/3043522571 >> 172.21.15.45:0/3043522571 conn(0x7f67d40fe040 msgr2=0x7f67d41068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.478+0000 7f67dac00700 1 -- 172.21.15.45:0/3043522571 shutdown_connections 2024-04-25T18:58:45.481 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:45.478+0000 7f67dac00700 1 -- 172.21.15.45:0/3043522571 wait complete. 2024-04-25T18:58:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:45 smithi062 ceph-mon[32844]: pgmap v428: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:45.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:45 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:58:45.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:45 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3043522571' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T18:58:45.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:45 smithi045 ceph-mon[27776]: pgmap v428: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:45.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:45 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:58:45.903 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:45 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3043522571' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T18:58:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:45 smithi079 ceph-mon[33002]: pgmap v428: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:45 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:58:45.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:45 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3043522571' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T18:58:46.277 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T18:58:46.288 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:58:46.765 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:47.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:47.203 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:58:47.204 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:47 smithi045 ceph-mon[27776]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:47.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:58:47.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:47 smithi062 ceph-mon[32844]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:47.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:58:47.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:58:47.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:47 smithi079 ceph-mon[33002]: from='mgr.14152 172.21.15.45:0/2350258367' entity='mgr.x' 2024-04-25T18:58:48.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:48 smithi045 ceph-mon[27776]: pgmap v429: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:48.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:48 smithi062 ceph-mon[32844]: pgmap v429: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:48.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:48 smithi079 ceph-mon[33002]: pgmap v429: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:48.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.965+0000 7fc0afe00700 1 -- 172.21.15.45:0/180916071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b010cfb0 msgr2=0x7fc0b010f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:48.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.965+0000 7fc0afe00700 1 --2- 172.21.15.45:0/180916071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b010cfb0 0x7fc0b010f470 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800b0a0 tx=0x7fc0a800b3b0 comp rx=0 tx=0).stop 2024-04-25T18:58:48.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.965+0000 7fc0afe00700 1 -- 172.21.15.45:0/180916071 shutdown_connections 2024-04-25T18:58:48.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.965+0000 7fc0afe00700 1 --2- 172.21.15.45:0/180916071 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b010cfb0 0x7fc0b010f470 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:48.968 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.965+0000 7fc0afe00700 1 --2- 172.21.15.45:0/180916071 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0b0103b80 0x7fc0b0103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:48.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.966+0000 7fc0afe00700 1 --2- 172.21.15.45:0/180916071 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b01044a0 0x7fc0b010c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:48.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.966+0000 7fc0afe00700 1 -- 172.21.15.45:0/180916071 >> 172.21.15.45:0/180916071 conn(0x7fc0b00fd3d0 msgr2=0x7fc0b00ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:48.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.966+0000 7fc0afe00700 1 -- 172.21.15.45:0/180916071 shutdown_connections 2024-04-25T18:58:48.969 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.966+0000 7fc0afe00700 1 -- 172.21.15.45:0/180916071 wait complete. 2024-04-25T18:58:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.967+0000 7fc0afe00700 1 Processor -- start 2024-04-25T18:58:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.967+0000 7fc0afe00700 1 -- start start 2024-04-25T18:58:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.967+0000 7fc0afe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b0103b80 0x7fc0b019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.967+0000 7fc0afe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0b01044a0 0x7fc0b019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.967+0000 7fc0afe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b010cfb0 0x7fc0b019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.967+0000 7fc0afe00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0112bf0 con 0x7fc0b01044a0 2024-04-25T18:58:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.967+0000 7fc0afe00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0112a70 con 0x7fc0b0103b80 2024-04-25T18:58:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.967+0000 7fc0afe00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b0112d70 con 0x7fc0b010cfb0 2024-04-25T18:58:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.967+0000 7fc0ae000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0b01044a0 0x7fc0b019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:48.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0aea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b0103b80 0x7fc0b019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0ae000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0b01044a0 0x7fc0b019b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39046/0 (socket says 172.21.15.45:39046) 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0ae000700 1 -- 172.21.15.45:0/634249251 learned_addr learned my addr 172.21.15.45:0/634249251 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0af400700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b010cfb0 0x7fc0b019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0ae000700 1 -- 172.21.15.45:0/634249251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b010cfb0 msgr2=0x7fc0b019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0ae000700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b010cfb0 0x7fc0b019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0ae000700 1 -- 172.21.15.45:0/634249251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b0103b80 msgr2=0x7fc0b019b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0ae000700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b0103b80 0x7fc0b019b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0ae000700 1 -- 172.21.15.45:0/634249251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b01a02c0 con 0x7fc0b01044a0 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0af400700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b010cfb0 0x7fc0b019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.968+0000 7fc0ae000700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0b01044a0 0x7fc0b019b7f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00e9c0 tx=0x7fc09c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.969+0000 7fc0a7400700 1 -- 172.21.15.45:0/634249251 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c004a30 con 0x7fc0b01044a0 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.969+0000 7fc0a7400700 1 -- 172.21.15.45:0/634249251 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc09c00cc70 con 0x7fc0b01044a0 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.969+0000 7fc0afe00700 1 -- 172.21.15.45:0/634249251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b01a05b0 con 0x7fc0b01044a0 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.969+0000 7fc0a7400700 1 -- 172.21.15.45:0/634249251 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c0054f0 con 0x7fc0b01044a0 2024-04-25T18:58:48.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.969+0000 7fc0afe00700 1 -- 172.21.15.45:0/634249251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b01a7e30 con 0x7fc0b01044a0 2024-04-25T18:58:48.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.970+0000 7fc0afe00700 1 -- 172.21.15.45:0/634249251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0b0069730 con 0x7fc0b01044a0 2024-04-25T18:58:48.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.972+0000 7fc0a7400700 1 -- 172.21.15.45:0/634249251 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc09c00c7c0 con 0x7fc0b01044a0 2024-04-25T18:58:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.973+0000 7fc0a7400700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc0940a3cc0 0x7fc0940a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:48.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.974+0000 7fc0a7400700 1 -- 172.21.15.45:0/634249251 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fc09c0d0640 con 0x7fc0b01044a0 2024-04-25T18:58:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.974+0000 7fc0aea00700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc0940a3cc0 0x7fc0940a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:48.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.974+0000 7fc0aea00700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc0940a3cc0 0x7fc0940a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009c60 tx=0x7fc0a0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:48.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:48.976+0000 7fc0a7400700 1 -- 172.21.15.45:0/634249251 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc09c053ff0 con 0x7fc0b01044a0 2024-04-25T18:58:49.246 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.243+0000 7fc0afe00700 1 -- 172.21.15.45:0/634249251 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc0b01a80c0 con 0x7fc0b01044a0 2024-04-25T18:58:49.247 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.245+0000 7fc0a7400700 1 -- 172.21.15.45:0/634249251 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fc09c09cfa0 con 0x7fc0b01044a0 2024-04-25T18:58:49.248 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:58:49.248 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:55:50.062944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24512]},"id":1}]} 2024-04-25T18:58:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.248+0000 7fc0afe00700 1 -- 172.21.15.45:0/634249251 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc0940a3cc0 msgr2=0x7fc0940a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.248+0000 7fc0afe00700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc0940a3cc0 0x7fc0940a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009c60 tx=0x7fc0a0009260 comp rx=0 tx=0).stop 2024-04-25T18:58:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.248+0000 7fc0afe00700 1 -- 172.21.15.45:0/634249251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0b01044a0 msgr2=0x7fc0b019b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:49.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.248+0000 7fc0afe00700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0b01044a0 0x7fc0b019b7f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00e9c0 tx=0x7fc09c00ecd0 comp rx=0 tx=0).stop 2024-04-25T18:58:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.248+0000 7fc0afe00700 1 -- 172.21.15.45:0/634249251 shutdown_connections 2024-04-25T18:58:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.248+0000 7fc0afe00700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fc0940a3cc0 0x7fc0940a6180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.248+0000 7fc0afe00700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc0b0103b80 0x7fc0b019b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.248+0000 7fc0afe00700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc0b010cfb0 0x7fc0b019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.248+0000 7fc0afe00700 1 --2- 172.21.15.45:0/634249251 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc0b01044a0 0x7fc0b019b7f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.248+0000 7fc0afe00700 1 -- 172.21.15.45:0/634249251 >> 172.21.15.45:0/634249251 conn(0x7fc0b00fd3d0 msgr2=0x7fc0b00ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.249+0000 7fc0afe00700 1 -- 172.21.15.45:0/634249251 shutdown_connections 2024-04-25T18:58:49.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:49.249+0000 7fc0afe00700 1 -- 172.21.15.45:0/634249251 wait complete. 2024-04-25T18:58:49.252 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-25T18:58:50.097 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:58:50.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:50 smithi062 ceph-mon[32844]: pgmap v430: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:50.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:50 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/634249251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:50 smithi079 ceph-mon[33002]: pgmap v430: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:50 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/634249251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:50.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:50 smithi045 ceph-mon[27776]: pgmap v430: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:50.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:50 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/634249251' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:50.563 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:52.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:52 smithi045 ceph-mon[27776]: pgmap v431: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:52.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:52 smithi062 ceph-mon[32844]: pgmap v431: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:52.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:52 smithi079 ceph-mon[33002]: pgmap v431: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:52.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.729+0000 7fa9c8600700 1 -- 172.21.15.45:0/3003631540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9c0109310 msgr2=0x7fa9c01096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:52.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.729+0000 7fa9c8600700 1 --2- 172.21.15.45:0/3003631540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9c0109310 0x7fa9c01096f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa9b40099c0 tx=0x7fa9b4009cd0 comp rx=0 tx=0).stop 2024-04-25T18:58:52.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.730+0000 7fa9c8600700 1 -- 172.21.15.45:0/3003631540 shutdown_connections 2024-04-25T18:58:52.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.730+0000 7fa9c8600700 1 --2- 172.21.15.45:0/3003631540 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c010e3a0 0x7fa9c0115f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:52.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.730+0000 7fa9c8600700 1 --2- 172.21.15.45:0/3003631540 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9c0109310 0x7fa9c01096f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:52.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.730+0000 7fa9c8600700 1 --2- 172.21.15.45:0/3003631540 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c0109cc0 0x7fa9c010dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:52.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.730+0000 7fa9c8600700 1 -- 172.21.15.45:0/3003631540 >> 172.21.15.45:0/3003631540 conn(0x7fa9c0076210 msgr2=0x7fa9c0078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:52.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.730+0000 7fa9c8600700 1 -- 172.21.15.45:0/3003631540 shutdown_connections 2024-04-25T18:58:52.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.730+0000 7fa9c8600700 1 -- 172.21.15.45:0/3003631540 wait complete. 2024-04-25T18:58:52.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.731+0000 7fa9c8600700 1 Processor -- start 2024-04-25T18:58:52.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.731+0000 7fa9c8600700 1 -- start start 2024-04-25T18:58:52.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c8600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0109310 0x7fa9c01a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:52.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c8600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c0109cc0 0x7fa9c01a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:52.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c8600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9c010e3a0 0x7fa9c019f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:52.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c8600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0119280 con 0x7fa9c0109cc0 2024-04-25T18:58:52.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c8600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0119100 con 0x7fa9c0109310 2024-04-25T18:58:52.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c8600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0119400 con 0x7fa9c010e3a0 2024-04-25T18:58:52.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c5c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0109310 0x7fa9c01a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:52.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c5c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0109310 0x7fa9c01a53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:44144/0 (socket says 172.21.15.45:44144) 2024-04-25T18:58:52.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c5c00700 1 -- 172.21.15.45:0/808855308 learned_addr learned my addr 172.21.15.45:0/808855308 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:52.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c6600700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9c010e3a0 0x7fa9c019f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:52.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.732+0000 7fa9c5200700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c0109cc0 0x7fa9c01a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:52.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.733+0000 7fa9c5c00700 1 -- 172.21.15.45:0/808855308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9c010e3a0 msgr2=0x7fa9c019f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:52.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.733+0000 7fa9c5c00700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9c010e3a0 0x7fa9c019f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:52.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.733+0000 7fa9c5c00700 1 -- 172.21.15.45:0/808855308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c0109cc0 msgr2=0x7fa9c01a5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:52.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.733+0000 7fa9c5c00700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c0109cc0 0x7fa9c01a5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:52.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.733+0000 7fa9c5c00700 1 -- 172.21.15.45:0/808855308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c019fd00 con 0x7fa9c0109310 2024-04-25T18:58:52.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.733+0000 7fa9c5c00700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0109310 0x7fa9c01a53e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4006010 tx=0x7fa9b4005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:52.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.733+0000 7fa9be000700 1 -- 172.21.15.45:0/808855308 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4026070 con 0x7fa9c0109310 2024-04-25T18:58:52.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.734+0000 7fa9c8600700 1 -- 172.21.15.45:0/808855308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c019ff90 con 0x7fa9c0109310 2024-04-25T18:58:52.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.734+0000 7fa9be000700 1 -- 172.21.15.45:0/808855308 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b4005680 con 0x7fa9c0109310 2024-04-25T18:58:52.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.734+0000 7fa9be000700 1 -- 172.21.15.45:0/808855308 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400fb20 con 0x7fa9c0109310 2024-04-25T18:58:52.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.734+0000 7fa9c8600700 1 -- 172.21.15.45:0/808855308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c01ac2b0 con 0x7fa9c0109310 2024-04-25T18:58:52.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.735+0000 7fa9c8600700 1 -- 172.21.15.45:0/808855308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c01a0ad0 con 0x7fa9c0109310 2024-04-25T18:58:52.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.737+0000 7fa9be000700 1 -- 172.21.15.45:0/808855308 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa9b4029070 con 0x7fa9c0109310 2024-04-25T18:58:52.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.738+0000 7fa9be000700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa9a80a3c40 0x7fa9a80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:52.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.738+0000 7fa9be000700 1 -- 172.21.15.45:0/808855308 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fa9b40d1370 con 0x7fa9c0109310 2024-04-25T18:58:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.738+0000 7fa9c5200700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa9a80a3c40 0x7fa9a80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:52.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.739+0000 7fa9c5200700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa9a80a3c40 0x7fa9a80a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0078860 tx=0x7fa9ac009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:52.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:52.741+0000 7fa9be000700 1 -- 172.21.15.45:0/808855308 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9b4030020 con 0x7fa9c0109310 2024-04-25T18:58:53.021 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.019+0000 7fa9c8600700 1 -- 172.21.15.45:0/808855308 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9c0069710 con 0x7fa9c0109310 2024-04-25T18:58:53.022 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.020+0000 7fa9be000700 1 -- 172.21.15.45:0/808855308 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11852 (secure 0 0 0) 0x7fa9b4013070 con 0x7fa9c0109310 2024-04-25T18:58:53.023 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:58:53.023 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:55:50.062944+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24512]},"id":1}]} 2024-04-25T18:58:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 -- 172.21.15.45:0/808855308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa9a80a3c40 msgr2=0x7fa9a80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa9a80a3c40 0x7fa9a80a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0078860 tx=0x7fa9ac009260 comp rx=0 tx=0).stop 2024-04-25T18:58:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 -- 172.21.15.45:0/808855308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0109310 msgr2=0x7fa9c01a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:53.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0109310 0x7fa9c01a53e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4006010 tx=0x7fa9b4005280 comp rx=0 tx=0).stop 2024-04-25T18:58:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 -- 172.21.15.45:0/808855308 shutdown_connections 2024-04-25T18:58:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fa9a80a3c40 0x7fa9a80a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa9c0109310 0x7fa9c01a53e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9c010e3a0 0x7fa9c019f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 --2- 172.21.15.45:0/808855308 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa9c0109cc0 0x7fa9c01a5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 -- 172.21.15.45:0/808855308 >> 172.21.15.45:0/808855308 conn(0x7fa9c0076210 msgr2=0x7fa9c010d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 -- 172.21.15.45:0/808855308 shutdown_connections 2024-04-25T18:58:53.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:53.023+0000 7fa9c8600700 1 -- 172.21.15.45:0/808855308 wait complete. 2024-04-25T18:58:53.027 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 25 2024-04-25T18:58:53.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:53 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/808855308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:53.404 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:53 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/808855308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:53.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:53 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/808855308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:58:53.814 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:58:54.276 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:54.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:54 smithi062 ceph-mon[32844]: pgmap v432: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:54.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:54 smithi079 ceph-mon[33002]: pgmap v432: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:54.434 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:54 smithi045 ceph-mon[27776]: pgmap v432: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:58:56.116 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:56 smithi045 ceph-mon[27776]: pgmap v433: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:56.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:56 smithi062 ceph-mon[32844]: pgmap v433: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.407+0000 7f566b200700 1 -- 172.21.15.45:0/3804461648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5664113640 msgr2=0x7f5664115ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.407+0000 7f566b200700 1 --2- 172.21.15.45:0/3804461648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5664113640 0x7f5664115ae0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f565c00b0a0 tx=0x7f565c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:58:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.407+0000 7f566b200700 1 -- 172.21.15.45:0/3804461648 shutdown_connections 2024-04-25T18:58:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.407+0000 7f566b200700 1 --2- 172.21.15.45:0/3804461648 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5664101290 0x7f5664101670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.407+0000 7f566b200700 1 --2- 172.21.15.45:0/3804461648 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5664113640 0x7f5664115ae0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.407+0000 7f566b200700 1 --2- 172.21.15.45:0/3804461648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5664101c40 0x7f5664112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:56.409 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.407+0000 7f566b200700 1 -- 172.21.15.45:0/3804461648 >> 172.21.15.45:0/3804461648 conn(0x7f5664077fb0 msgr2=0x7f56640ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:56.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.407+0000 7f566b200700 1 -- 172.21.15.45:0/3804461648 shutdown_connections 2024-04-25T18:58:56.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.407+0000 7f566b200700 1 -- 172.21.15.45:0/3804461648 wait complete. 2024-04-25T18:58:56.410 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.408+0000 7f566b200700 1 Processor -- start 2024-04-25T18:58:56.411 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.409+0000 7f566b200700 1 -- start start 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.409+0000 7f566b200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5664101290 0x7f566419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.409+0000 7f566b200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5664101c40 0x7f566419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.409+0000 7f566b200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5664113640 0x7f56641a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.409+0000 7f566b200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5664119250 con 0x7f5664113640 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.409+0000 7f566b200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f56641190d0 con 0x7f5664101290 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.409+0000 7f566b200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56641193d0 con 0x7f5664101c40 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.409+0000 7f5663400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5664101c40 0x7f566419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5663400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5664101c40 0x7f566419fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32850/0 (socket says 172.21.15.45:32850) 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5663400700 1 -- 172.21.15.45:0/3826934321 learned_addr learned my addr 172.21.15.45:0/3826934321 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5663e00700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5664101290 0x7f566419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:56.412 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5669200700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5664113640 0x7f56641a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5663400700 1 -- 172.21.15.45:0/3826934321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5664101290 msgr2=0x7f566419f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5663400700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5664101290 0x7f566419f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5663400700 1 -- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5664113640 msgr2=0x7f56641a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5663400700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5664113640 0x7f56641a3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5663400700 1 -- 172.21.15.45:0/3826934321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56641a46c0 con 0x7f5664101c40 2024-04-25T18:58:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5669200700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5664113640 0x7f56641a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:58:56.413 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.410+0000 7f5663400700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5664101c40 0x7f566419fc40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f565000ea50 tx=0x7f565000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.411+0000 7f5660c00700 1 -- 172.21.15.45:0/3826934321 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565000cbd0 con 0x7f5664101c40 2024-04-25T18:58:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.411+0000 7f566b200700 1 -- 172.21.15.45:0/3826934321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56641a49b0 con 0x7f5664101c40 2024-04-25T18:58:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.411+0000 7f5660c00700 1 -- 172.21.15.45:0/3826934321 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565000cd70 con 0x7f5664101c40 2024-04-25T18:58:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.411+0000 7f5660c00700 1 -- 172.21.15.45:0/3826934321 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650018750 con 0x7f5664101c40 2024-04-25T18:58:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.411+0000 7f566b200700 1 -- 172.21.15.45:0/3826934321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56641ac230 con 0x7f5664101c40 2024-04-25T18:58:56.414 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.412+0000 7f566b200700 1 -- 172.21.15.45:0/3826934321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5664102040 con 0x7f5664101c40 2024-04-25T18:58:56.416 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.414+0000 7f5660c00700 1 -- 172.21.15.45:0/3826934321 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f565001f020 con 0x7f5664101c40 2024-04-25T18:58:56.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.415+0000 7f5660c00700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56480a3c40 0x7f56480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:58:56.417 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.416+0000 7f5660c00700 1 -- 172.21.15.45:0/3826934321 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f56500d0f60 con 0x7f5664101c40 2024-04-25T18:58:56.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.416+0000 7f5663e00700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56480a3c40 0x7f56480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:58:56.418 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.416+0000 7f5663e00700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56480a3c40 0x7f56480a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5654009680 tx=0x7f5654009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:58:56.420 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.418+0000 7f5660c00700 1 -- 172.21.15.45:0/3826934321 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5650026030 con 0x7f5664101c40 2024-04-25T18:58:56.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:56 smithi079 ceph-mon[33002]: pgmap v433: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:58:56.652 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.650+0000 7f566b200700 1 -- 172.21.15.45:0/3826934321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f566402df80 con 0x7f5664101c40 2024-04-25T18:58:56.654 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.652+0000 7f5660c00700 1 -- 172.21.15.45:0/3826934321 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+22654 (secure 0 0 0) 0x7f565009d8c0 con 0x7f5664101c40 2024-04-25T18:58:56.655 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:58:56.655 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":94,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T18:56:17.531513+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:55:17.591864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:55:22.070218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.655+0000 7f566b200700 1 -- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56480a3c40 msgr2=0x7f56480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.655+0000 7f566b200700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56480a3c40 0x7f56480a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5654009680 tx=0x7f5654009260 comp rx=0 tx=0).stop 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.655+0000 7f566b200700 1 -- 172.21.15.45:0/3826934321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5664101c40 msgr2=0x7f566419fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.655+0000 7f566b200700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5664101c40 0x7f566419fc40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f565000ea50 tx=0x7f565000ed60 comp rx=0 tx=0).stop 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.655+0000 7f566b200700 1 -- 172.21.15.45:0/3826934321 shutdown_connections 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.655+0000 7f566b200700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f56480a3c40 0x7f56480a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.655+0000 7f566b200700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5664101290 0x7f566419f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.655+0000 7f566b200700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5664113640 0x7f56641a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.655+0000 7f566b200700 1 --2- 172.21.15.45:0/3826934321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5664101c40 0x7f566419fc40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.655+0000 7f566b200700 1 -- 172.21.15.45:0/3826934321 >> 172.21.15.45:0/3826934321 conn(0x7f5664077fb0 msgr2=0x7f5664111fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.656+0000 7f566b200700 1 -- 172.21.15.45:0/3826934321 shutdown_connections 2024-04-25T18:58:56.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:58:56.656+0000 7f566b200700 1 -- 172.21.15.45:0/3826934321 wait complete. 2024-04-25T18:58:57.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:57 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3826934321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:57.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:57 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3826934321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:57.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:57 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3826934321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:58:57.448 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T18:55:26.025536+0000', 'modified': '2024-04-25T18:55:50.062944+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24512, 'mds_2': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6835/1201787807', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 1201787807}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 1201787807}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/2985890701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 2985890701}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 2985890701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6837/3290580293', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 3290580293}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 3290580293}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.79:6839/235958219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 235958219}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 235958219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.45:6841/2490735053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6840', 'nonce': 2490735053}, {'type': 'v1', 'addr': '172.21.15.45:6841', 'nonce': 2490735053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.62:6839/3288398937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 3288398937}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 3288398937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24524, 24505, 24512]} 2024-04-25T18:58:57.449 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T18:58:57.449 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T18:58:57.941 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:58:58.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:58:58 smithi062 ceph-mon[32844]: pgmap v434: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:58.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:58:58 smithi079 ceph-mon[33002]: pgmap v434: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:58:58.448 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:58:58 smithi045 ceph-mon[27776]: pgmap v434: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:59:00.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.247+0000 7f13fc600700 1 -- 172.21.15.45:0/256164153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4104ec0 msgr2=0x7f13f41052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:00.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.247+0000 7f13fc600700 1 --2- 172.21.15.45:0/256164153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4104ec0 0x7f13f41052a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f13e80099c0 tx=0x7f13e8009cd0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.250 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.247+0000 7f13fc600700 1 -- 172.21.15.45:0/256164153 shutdown_connections 2024-04-25T18:59:00.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.247+0000 7f13fc600700 1 --2- 172.21.15.45:0/256164153 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f4109fa0 0x7f13f4111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.247+0000 7f13fc600700 1 --2- 172.21.15.45:0/256164153 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4104ec0 0x7f13f41052a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.247+0000 7f13fc600700 1 --2- 172.21.15.45:0/256164153 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f4105870 0x7f13f41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.247+0000 7f13fc600700 1 -- 172.21.15.45:0/256164153 >> 172.21.15.45:0/256164153 conn(0x7f13f4100750 msgr2=0x7f13f4102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:00.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.248+0000 7f13fc600700 1 -- 172.21.15.45:0/256164153 shutdown_connections 2024-04-25T18:59:00.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.248+0000 7f13fc600700 1 -- 172.21.15.45:0/256164153 wait complete. 2024-04-25T18:59:00.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.249+0000 7f13fc600700 1 Processor -- start 2024-04-25T18:59:00.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.249+0000 7f13fc600700 1 -- start start 2024-04-25T18:59:00.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.249+0000 7f13fc600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f4104ec0 0x7f13f419b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.249+0000 7f13fc600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f4105870 0x7f13f419b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.249+0000 7f13fc600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4109fa0 0x7f13f419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.249+0000 7f13fc600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4114e30 con 0x7f13f4109fa0 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.249+0000 7f13fc600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4114cb0 con 0x7f13f4105870 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.249+0000 7f13fc600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f13f4114fb0 con 0x7f13f4104ec0 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13f9c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f4104ec0 0x7f13f419b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13fa600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4109fa0 0x7f13f419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13f9c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f4104ec0 0x7f13f419b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32874/0 (socket says 172.21.15.45:32874) 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13f9c00700 1 -- 172.21.15.45:0/1303044690 learned_addr learned my addr 172.21.15.45:0/1303044690 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13f9200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f4105870 0x7f13f419b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13f9c00700 1 -- 172.21.15.45:0/1303044690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f4105870 msgr2=0x7f13f419b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13f9c00700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f4105870 0x7f13f419b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13f9c00700 1 -- 172.21.15.45:0/1303044690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4109fa0 msgr2=0x7f13f419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13f9c00700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4109fa0 0x7f13f419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.253 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13f9c00700 1 -- 172.21.15.45:0/1303044690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f41a02c0 con 0x7f13f4104ec0 2024-04-25T18:59:00.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13fa600700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4109fa0 0x7f13f419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:00.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.250+0000 7f13f9c00700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f4104ec0 0x7f13f419b300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f13e800b580 tx=0x7f13e80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:00.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.251+0000 7f13f2000700 1 -- 172.21.15.45:0/1303044690 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8027070 con 0x7f13f4104ec0 2024-04-25T18:59:00.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.251+0000 7f13fc600700 1 -- 172.21.15.45:0/1303044690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f41a0550 con 0x7f13f4104ec0 2024-04-25T18:59:00.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.251+0000 7f13f2000700 1 -- 172.21.15.45:0/1303044690 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e8003f20 con 0x7f13f4104ec0 2024-04-25T18:59:00.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.251+0000 7f13f2000700 1 -- 172.21.15.45:0/1303044690 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e8022d10 con 0x7f13f4104ec0 2024-04-25T18:59:00.254 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.251+0000 7f13fc600700 1 -- 172.21.15.45:0/1303044690 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f41a7e30 con 0x7f13f4104ec0 2024-04-25T18:59:00.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.254+0000 7f13f2000700 1 -- 172.21.15.45:0/1303044690 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f13e8022470 con 0x7f13f4104ec0 2024-04-25T18:59:00.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.256+0000 7f13f2000700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13dc0a3cc0 0x7f13dc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.256+0000 7f13f2000700 1 -- 172.21.15.45:0/1303044690 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f13e80d0fe0 con 0x7f13f4104ec0 2024-04-25T18:59:00.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.256+0000 7f13f9200700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13dc0a3cc0 0x7f13dc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.256+0000 7f1406ef9b80 1 Processor -- start 2024-04-25T18:59:00.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.256+0000 7f1406ef9b80 1 -- start start 2024-04-25T18:59:00.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f13f9200700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13dc0a3cc0 0x7f13dc0a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f13f4101c50 tx=0x7f13e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:00.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f142f0 0x562d20f146d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 0x562d20f1f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f1fab0 0x562d20f21ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562d20e67e40 con 0x562d20f14c10 2024-04-25T18:59:00.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562d20e0e6e0 con 0x562d20f1fab0 2024-04-25T18:59:00.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562d20e63e00 con 0x562d20f142f0 2024-04-25T18:59:00.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f13fa600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f1fab0 0x562d20f21ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f13f9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 0x562d20f1f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f13f9c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f142f0 0x562d20f146d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f13fa600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f1fab0 0x562d20f21ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:57892/0 (socket says 172.21.15.45:57892) 2024-04-25T18:59:00.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.257+0000 7f13fa600700 1 -- 172.21.15.45:0/923482959 learned_addr learned my addr 172.21.15.45:0/923482959 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.258+0000 7f13f9c00700 1 -- 172.21.15.45:0/923482959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f1fab0 msgr2=0x562d20f21ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.258+0000 7f13f9c00700 1 --2- 172.21.15.45:0/923482959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f1fab0 0x562d20f21ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.258+0000 7f13f9c00700 1 -- 172.21.15.45:0/923482959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 msgr2=0x562d20f1f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.258+0000 7f13f9c00700 1 --2- 172.21.15.45:0/923482959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 0x562d20f1f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.258+0000 7f13f9c00700 1 -- 172.21.15.45:0/923482959 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d20e04630 con 0x562d20f142f0 2024-04-25T18:59:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.258+0000 7f13fa600700 1 --2- 172.21.15.45:0/923482959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f1fab0 0x562d20f21ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.258+0000 7f13f9c00700 1 --2- 172.21.15.45:0/923482959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f142f0 0x562d20f146d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f13e809d010 tx=0x7f13e8022810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ab1bd99eb69ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.258+0000 7f13e6a00700 1 -- 172.21.15.45:0/923482959 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e809d310 con 0x562d20f142f0 2024-04-25T18:59:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.258+0000 7f13e6a00700 1 -- 172.21.15.45:0/923482959 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e80a16c0 con 0x562d20f142f0 2024-04-25T18:59:00.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.258+0000 7f13e6a00700 1 -- 172.21.15.45:0/923482959 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e809c760 con 0x562d20f142f0 2024-04-25T18:59:00.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.259+0000 7f1406ef9b80 1 -- 172.21.15.45:0/923482959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f142f0 msgr2=0x562d20f146d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:00.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.259+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/923482959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f142f0 0x562d20f146d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f13e809d010 tx=0x7f13e8022810 comp rx=0 tx=0).stop 2024-04-25T18:59:00.262 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.259+0000 7f1406ef9b80 1 -- 172.21.15.45:0/923482959 shutdown_connections 2024-04-25T18:59:00.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.259+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/923482959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f1fab0 0x562d20f21ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.259+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/923482959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f142f0 0x562d20f146d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.259+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/923482959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 0x562d20f1f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.259+0000 7f1406ef9b80 1 -- 172.21.15.45:0/923482959 >> 172.21.15.45:0/923482959 conn(0x562d20e43550 msgr2=0x562d20f10790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:00.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.260+0000 7f1406ef9b80 1 -- 172.21.15.45:0/923482959 shutdown_connections 2024-04-25T18:59:00.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.260+0000 7f1406ef9b80 1 -- 172.21.15.45:0/923482959 wait complete. 2024-04-25T18:59:00.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.261+0000 7f1406ef9b80 1 -- start start 2024-04-25T18:59:00.263 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.261+0000 7f1406ef9b80 1 Processor -- start 2024-04-25T18:59:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.261+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 0x562d20f1f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.261+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 0x562d20f1c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.261+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f1fab0 0x562d20f1cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.261+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562d20e67e40 con 0x562d20f14c10 2024-04-25T18:59:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.261+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562d20e21810 con 0x562d20f142f0 2024-04-25T18:59:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.261+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562d20e63e00 con 0x562d20f1fab0 2024-04-25T18:59:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.261+0000 7f1406ef9b80 10 client.0 _ll_register_callbacks cb 0x562d20e006d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13f9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 0x562d20f1c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13f9c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 0x562d20f1f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.265 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13f9200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 0x562d20f1c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36254/0 (socket says 172.21.15.45:36254) 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13f9200700 1 -- 172.21.15.45:0/3242703959 learned_addr learned my addr 172.21.15.45:0/3242703959 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13fa600700 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f1fab0 0x562d20f1cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13f9c00700 1 -- 172.21.15.45:0/3242703959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f1fab0 msgr2=0x562d20f1cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13f9c00700 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f1fab0 0x562d20f1cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13f9c00700 1 -- 172.21.15.45:0/3242703959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 msgr2=0x562d20f1c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13f9c00700 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 0x562d20f1c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13f9c00700 1 -- 172.21.15.45:0/3242703959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d20db5880 con 0x562d20f142f0 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.262+0000 7f13f9200700 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 0x562d20f1c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.263+0000 7f13f9c00700 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 0x562d20f1f290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f13e8099ce0 tx=0x7f13e80a5db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34f06ab85451950 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.263+0000 7f13db400700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.263+0000 7f13db400700 1 -- 172.21.15.45:0/3242703959 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e80ba020 con 0x562d20f142f0 2024-04-25T18:59:00.266 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.263+0000 7f13db400700 1 -- 172.21.15.45:0/3242703959 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f13e80040d0 con 0x562d20f142f0 2024-04-25T18:59:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.263+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3242703959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e80a16c0 con 0x562d20f142f0 2024-04-25T18:59:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.263+0000 7f13db400700 1 -- 172.21.15.45:0/3242703959 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13e80a5ef0 con 0x562d20f142f0 2024-04-25T18:59:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.263+0000 7f1406ef9b80 10 client.24986 fetch_fsmap learned FSMap version 25 2024-04-25T18:59:00.267 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.263+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3242703959 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d20dfeaa0 con 0x562d20f142f0 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.263+0000 7f13db400700 1 -- 172.21.15.45:0/3242703959 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e80a16c0 con 0x562d20f142f0 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.264+0000 7f13db400700 1 -- 172.21.15.45:0/3242703959 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f13e80990f0 con 0x562d20f142f0 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.264+0000 7f1406ef9b80 10 client.24986 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.264+0000 7f1406ef9b80 10 client.24986 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.264+0000 7f1406ef9b80 20 client.24986 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.264+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x562d20f2ed90 0x562d20f30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.264+0000 7f1406ef9b80 4 client.24986 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.264+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3242703959 --> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d20e75630 con 0x562d20f2ed90 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.264+0000 7f13f9200700 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x562d20f2ed90 0x562d20f30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.265+0000 7f13f9200700 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x562d20f2ed90 0x562d20f30200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d735b6e4b2217a70 server_cookie=9f21b586bfa4fccc in_seq=0 out_seq=0 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.265+0000 7f13db400700 10 client.24986 ms_handle_connect on v2:172.21.15.79:6838/235958219 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f13db400700 1 -- 172.21.15.45:0/3242703959 <== mds.2 v2:172.21.15.79:6838/235958219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562d20e75630 con 0x562d20f2ed90 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f13db400700 10 client.24986 handle_command_reply: tid=0 2024-04-25T18:59:00.269 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 client.24986 shutdown 2024-04-25T18:59:00.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3242703959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 msgr2=0x562d20f1f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:00.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 0x562d20f1f290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f13e8099ce0 tx=0x7f13e80a5db0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3242703959 shutdown_connections 2024-04-25T18:59:00.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x562d20f2ed90 0x562d20f30200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 0x562d20f1f290 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f1fab0 0x562d20f1cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/3242703959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f14c10 0x562d20f1c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f13db400700 0 client.24986 ms_handle_reset on v2:172.21.15.79:6838/235958219 2024-04-25T18:59:00.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3242703959 >> 172.21.15.45:0/3242703959 conn(0x562d20e43550 msgr2=0x562d20f16c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:00.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3242703959 shutdown_connections 2024-04-25T18:59:00.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.266+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3242703959 wait complete. 2024-04-25T18:59:00.271 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.267+0000 7f1406ef9b80 20 client.24986 trim_cache size 0 max 16384 2024-04-25T18:59:00.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.289+0000 7f1406ef9b80 1 Processor -- start 2024-04-25T18:59:00.291 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f1406ef9b80 1 -- start start 2024-04-25T18:59:00.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f142f0 0x562d20f1ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f14c10 0x562d20f39660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.292 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f1fab0 0x562d20f39ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562d20f2e540 con 0x562d20f142f0 2024-04-25T18:59:00.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562d20f2ec10 con 0x562d20f14c10 2024-04-25T18:59:00.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562d20f2f2e0 con 0x562d20f1fab0 2024-04-25T18:59:00.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f13f9c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f142f0 0x562d20f1ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f13f9c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f142f0 0x562d20f1ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36262/0 (socket says 172.21.15.45:36262) 2024-04-25T18:59:00.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f13f9c00700 1 -- 172.21.15.45:0/3130297266 learned_addr learned my addr 172.21.15.45:0/3130297266 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:00.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f13fa600700 1 --2- 172.21.15.45:0/3130297266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f1fab0 0x562d20f39ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.293 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.290+0000 7f13f9200700 1 --2- 172.21.15.45:0/3130297266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f14c10 0x562d20f39660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.291+0000 7f13f9c00700 1 -- 172.21.15.45:0/3130297266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f1fab0 msgr2=0x562d20f39ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:00.294 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.291+0000 7f13f9c00700 1 --2- 172.21.15.45:0/3130297266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f1fab0 0x562d20f39ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.291+0000 7f13f9c00700 1 -- 172.21.15.45:0/3130297266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f14c10 msgr2=0x562d20f39660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.291+0000 7f13f9c00700 1 --2- 172.21.15.45:0/3130297266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f14c10 0x562d20f39660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.291+0000 7f13f9c00700 1 -- 172.21.15.45:0/3130297266 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d20f1ab40 con 0x562d20f142f0 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.291+0000 7f13f9c00700 1 --2- 172.21.15.45:0/3130297266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f142f0 0x562d20f1ca80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f13e809d010 tx=0x7f13e80996e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d72137c631555f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.291+0000 7f13e6000700 1 -- 172.21.15.45:0/3130297266 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e80ba020 con 0x562d20f142f0 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f13e6000700 1 -- 172.21.15.45:0/3130297266 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e807d020 con 0x562d20f142f0 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f13e6000700 1 -- 172.21.15.45:0/3130297266 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e80a1240 con 0x562d20f142f0 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3130297266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f142f0 msgr2=0x562d20f1ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/3130297266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f142f0 0x562d20f1ca80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f13e809d010 tx=0x7f13e80996e0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3130297266 shutdown_connections 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/3130297266 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f1fab0 0x562d20f39ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/3130297266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f142f0 0x562d20f1ca80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/3130297266 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f14c10 0x562d20f39660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3130297266 >> 172.21.15.45:0/3130297266 conn(0x562d20e43550 msgr2=0x562d20f1c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:00.295 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3130297266 shutdown_connections 2024-04-25T18:59:00.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.292+0000 7f1406ef9b80 1 -- 172.21.15.45:0/3130297266 wait complete. 2024-04-25T18:59:00.296 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.293+0000 7f1406ef9b80 1 -- start start 2024-04-25T18:59:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.293+0000 7f1406ef9b80 1 Processor -- start 2024-04-25T18:59:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.293+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 0x562d20f39500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f14c10 0x562d20f39a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.298 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f1406ef9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f1fab0 0x562d20f33890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562d20f30080 con 0x562d20f1fab0 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x562d20f2f9b0 con 0x562d20f142f0 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f1406ef9b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x562d20f30750 con 0x562d20f14c10 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f1406ef9b80 10 client.0 _ll_register_callbacks cb 0x562d20e006d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13fa600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f1fab0 0x562d20f33890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13fa600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f1fab0 0x562d20f33890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36264/0 (socket says 172.21.15.45:36264) 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13fa600700 1 -- 172.21.15.45:0/327440738 learned_addr learned my addr 172.21.15.45:0/327440738 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13f9200700 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f14c10 0x562d20f39a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13f9c00700 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 0x562d20f39500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13fa600700 1 -- 172.21.15.45:0/327440738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f14c10 msgr2=0x562d20f39a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13fa600700 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f14c10 0x562d20f39a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13fa600700 1 -- 172.21.15.45:0/327440738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 msgr2=0x562d20f39500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13fa600700 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 0x562d20f39500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.299 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13fa600700 1 -- 172.21.15.45:0/327440738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d20f340d0 con 0x562d20f1fab0 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13f9200700 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f14c10 0x562d20f39a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.294+0000 7f13f9c00700 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 0x562d20f39500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.295+0000 7f13fa600700 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f1fab0 0x562d20f33890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f13ec004810 tx=0x7f13ec004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e56cfdc71af5ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.295+0000 7f13db400700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.295+0000 7f13db400700 1 -- 172.21.15.45:0/327440738 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec004bb0 con 0x562d20f1fab0 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.295+0000 7f1406ef9b80 1 -- 172.21.15.45:0/327440738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13e807d020 con 0x562d20f1fab0 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.295+0000 7f13db400700 1 -- 172.21.15.45:0/327440738 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f13ec004d50 con 0x562d20f1fab0 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.295+0000 7f13db400700 1 -- 172.21.15.45:0/327440738 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13ec027070 con 0x562d20f1fab0 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.295+0000 7f1406ef9b80 10 client.14871 fetch_fsmap learned FSMap version 25 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.295+0000 7f1406ef9b80 1 -- 172.21.15.45:0/327440738 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562d20f96ad0 con 0x562d20f1fab0 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.296+0000 7f13db400700 1 -- 172.21.15.45:0/327440738 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13e807d020 con 0x562d20f1fab0 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.296+0000 7f13db400700 1 -- 172.21.15.45:0/327440738 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f13ec021cc0 con 0x562d20f1fab0 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.296+0000 7f1406ef9b80 10 client.14871 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:59:00.300 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.296+0000 7f1406ef9b80 10 client.14871 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:59:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.296+0000 7f1406ef9b80 20 client.14871 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.296+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x562d20f9b6e0 0x562d20f9a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.296+0000 7f1406ef9b80 4 client.14871 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:59:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.296+0000 7f1406ef9b80 1 -- 172.21.15.45:0/327440738 --> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] -- command(tid 0: {"prefix": "status"}) -- 0x562d20e6f6d0 con 0x562d20f9b6e0 2024-04-25T18:59:00.301 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.296+0000 7f13f9c00700 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x562d20f9b6e0 0x562d20f9a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f13f9c00700 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x562d20f9b6e0 0x562d20f9a3f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b0e6e3334919963 server_cookie=f7cb119c8ab9fd0c in_seq=0 out_seq=0 2024-04-25T18:59:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f13db400700 10 client.14871 ms_handle_connect on v2:172.21.15.79:6838/235958219 2024-04-25T18:59:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f13db400700 1 -- 172.21.15.45:0/327440738 <== mds.2 v2:172.21.15.79:6838/235958219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x562d20e6f6d0 con 0x562d20f9b6e0 2024-04-25T18:59:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f13db400700 10 client.14871 handle_command_reply: tid=0 2024-04-25T18:59:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f1406ef9b80 1 client.14871 shutdown 2024-04-25T18:59:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f1406ef9b80 1 -- 172.21.15.45:0/327440738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f1fab0 msgr2=0x562d20f33890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f1fab0 0x562d20f33890 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f13ec004810 tx=0x7f13ec004840 comp rx=0 tx=0).stop 2024-04-25T18:59:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f1406ef9b80 1 -- 172.21.15.45:0/327440738 shutdown_connections 2024-04-25T18:59:00.302 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x562d20f9b6e0 0x562d20f9a3f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x562d20f142f0 0x562d20f39500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.297+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562d20f1fab0 0x562d20f33890 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.298+0000 7f1406ef9b80 1 --2- 172.21.15.45:0/327440738 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x562d20f14c10 0x562d20f39a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.298+0000 7f1406ef9b80 1 -- 172.21.15.45:0/327440738 >> 172.21.15.45:0/327440738 conn(0x562d20e43550 msgr2=0x562d20f0f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.298+0000 7f13db400700 0 client.14871 ms_handle_reset on v2:172.21.15.79:6838/235958219 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.298+0000 7f1406ef9b80 1 -- 172.21.15.45:0/327440738 shutdown_connections 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.298+0000 7f1406ef9b80 1 -- 172.21.15.45:0/327440738 wait complete. 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.298+0000 7f1406ef9b80 20 client.14871 trim_cache size 0 max 16384 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:59:00.303 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-04-25T18:59:00.304 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24505, 2024-04-25T18:59:00.304 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T18:59:00.304 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T18:59:00.304 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T18:59:00.304 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 196.92391773200001, 2024-04-25T18:59:00.304 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 25, 2024-04-25T18:59:00.304 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-25T18:59:00.304 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:59:00.304 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 248.175287376 2024-04-25T18:59:00.304 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 -- 172.21.15.45:0/1303044690 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13dc0a3cc0 msgr2=0x7f13dc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13dc0a3cc0 0x7f13dc0a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f13f4101c50 tx=0x7f13e000b410 comp rx=0 tx=0).stop 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 -- 172.21.15.45:0/1303044690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f4104ec0 msgr2=0x7f13f419b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f4104ec0 0x7f13f419b300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f13e800b580 tx=0x7f13e80047b0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 -- 172.21.15.45:0/1303044690 shutdown_connections 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13dc0a3cc0 0x7f13dc0a6180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f13f4109fa0 0x7f13f419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f13f4104ec0 0x7f13f419b300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 --2- 172.21.15.45:0/1303044690 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f13f4105870 0x7f13f419b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 -- 172.21.15.45:0/1303044690 >> 172.21.15.45:0/1303044690 conn(0x7f13f4100750 msgr2=0x7f13f4102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:00.305 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 -- 172.21.15.45:0/1303044690 shutdown_connections 2024-04-25T18:59:00.306 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:00.299+0000 7f13fc600700 1 -- 172.21.15.45:0/1303044690 wait complete. 2024-04-25T18:59:00.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:00 smithi062 ceph-mon[32844]: pgmap v435: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:59:00.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:00 smithi079 ceph-mon[33002]: pgmap v435: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:59:00.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:00 smithi045 ceph-mon[27776]: pgmap v435: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:59:01.202 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.a status 2024-04-25T18:59:01.677 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:02.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:02 smithi062 ceph-mon[32844]: pgmap v436: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:59:02.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:02 smithi079 ceph-mon[33002]: pgmap v436: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:59:02.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:02 smithi045 ceph-mon[27776]: pgmap v436: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:59:03.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.834+0000 7f66e7200700 1 -- 172.21.15.45:0/1156810368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66e0077890 msgr2=0x7f66e01133c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.834+0000 7f66e7200700 1 --2- 172.21.15.45:0/1156810368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66e0077890 0x7f66e01133c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009990 tx=0x7f66cc009ca0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.835+0000 7f66e7200700 1 -- 172.21.15.45:0/1156810368 shutdown_connections 2024-04-25T18:59:03.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.835+0000 7f66e7200700 1 --2- 172.21.15.45:0/1156810368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0113900 0x7f66e0115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.835+0000 7f66e7200700 1 --2- 172.21.15.45:0/1156810368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e0076f70 0x7f66e0077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.835+0000 7f66e7200700 1 --2- 172.21.15.45:0/1156810368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66e0077890 0x7f66e01133c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.835+0000 7f66e7200700 1 -- 172.21.15.45:0/1156810368 >> 172.21.15.45:0/1156810368 conn(0x7f66e0100750 msgr2=0x7f66e0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.835+0000 7f66e7200700 1 -- 172.21.15.45:0/1156810368 shutdown_connections 2024-04-25T18:59:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.835+0000 7f66e7200700 1 -- 172.21.15.45:0/1156810368 wait complete. 2024-04-25T18:59:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.836+0000 7f66e7200700 1 Processor -- start 2024-04-25T18:59:03.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.836+0000 7f66e7200700 1 -- start start 2024-04-25T18:59:03.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66e7200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e0076f70 0x7f66e01a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66e7200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66e0077890 0x7f66e01a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66e7200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0113900 0x7f66e019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66e7200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01192e0 con 0x7f66e0076f70 2024-04-25T18:59:03.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66e7200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0119160 con 0x7f66e0113900 2024-04-25T18:59:03.839 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66e7200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0119460 con 0x7f66e0077890 2024-04-25T18:59:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66df400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66e0077890 0x7f66e01a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66df400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66e0077890 0x7f66e01a5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32926/0 (socket says 172.21.15.45:32926) 2024-04-25T18:59:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66df400700 1 -- 172.21.15.45:0/2261661800 learned_addr learned my addr 172.21.15.45:0/2261661800 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66dfe00700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e0076f70 0x7f66e01a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.837+0000 7f66e5200700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0113900 0x7f66e019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.838+0000 7f66df400700 1 -- 172.21.15.45:0/2261661800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0113900 msgr2=0x7f66e019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.838+0000 7f66df400700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0113900 0x7f66e019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.838+0000 7f66df400700 1 -- 172.21.15.45:0/2261661800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e0076f70 msgr2=0x7f66e01a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.838+0000 7f66df400700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e0076f70 0x7f66e01a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.838+0000 7f66df400700 1 -- 172.21.15.45:0/2261661800 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e019fd60 con 0x7f66e0077890 2024-04-25T18:59:03.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.838+0000 7f66df400700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66e0077890 0x7f66e01a5930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f66cc005910 tx=0x7f66cc00b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:03.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.839+0000 7f66dcc00700 1 -- 172.21.15.45:0/2261661800 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc027070 con 0x7f66e0077890 2024-04-25T18:59:03.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.839+0000 7f66dcc00700 1 -- 172.21.15.45:0/2261661800 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66cc022470 con 0x7f66e0077890 2024-04-25T18:59:03.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.839+0000 7f66e7200700 1 -- 172.21.15.45:0/2261661800 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e019fff0 con 0x7f66e0077890 2024-04-25T18:59:03.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.839+0000 7f66dcc00700 1 -- 172.21.15.45:0/2261661800 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc017650 con 0x7f66e0077890 2024-04-25T18:59:03.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.839+0000 7f66e7200700 1 -- 172.21.15.45:0/2261661800 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e01ac2c0 con 0x7f66e0077890 2024-04-25T18:59:03.844 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.842+0000 7f66dcc00700 1 -- 172.21.15.45:0/2261661800 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f66cc02a070 con 0x7f66e0077890 2024-04-25T18:59:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.844+0000 7f66f1b26b80 1 Processor -- start 2024-04-25T18:59:03.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.844+0000 7f66f1b26b80 1 -- start start 2024-04-25T18:59:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf460fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf46100a0 0x555bf46099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4609ef0 0x555bf460a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555bf455be40 con 0x555bf46100a0 2024-04-25T18:59:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555bf45026e0 con 0x555bf4609ef0 2024-04-25T18:59:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555bf4557e00 con 0x555bf460f780 2024-04-25T18:59:03.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66dcc00700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f66c40a3c40 0x7f66c40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66e5200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf460fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66dcc00700 1 -- 172.21.15.45:0/2261661800 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f66cc0d10a0 con 0x7f66e0077890 2024-04-25T18:59:03.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66dfe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf46100a0 0x555bf46099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66e5200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf460fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32938/0 (socket says 172.21.15.45:32938) 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66e5200700 1 -- 172.21.15.45:0/743266931 learned_addr learned my addr 172.21.15.45:0/743266931 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66e5200700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f66c40a3c40 0x7f66c40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.845+0000 7f66df400700 1 --2- 172.21.15.45:0/743266931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4609ef0 0x555bf460a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.846+0000 7f66e5200700 1 -- 172.21.15.45:0/743266931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4609ef0 msgr2=0x555bf460a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.846+0000 7f66e5200700 1 --2- 172.21.15.45:0/743266931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4609ef0 0x555bf460a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.846+0000 7f66e5200700 1 -- 172.21.15.45:0/743266931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf46100a0 msgr2=0x555bf46099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.846+0000 7f66e5200700 1 --2- 172.21.15.45:0/743266931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf46100a0 0x555bf46099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.846+0000 7f66e5200700 1 -- 172.21.15.45:0/743266931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf44f8630 con 0x555bf460f780 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.846+0000 7f66dfe00700 1 --2- 172.21.15.45:0/743266931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf46100a0 0x555bf46099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.846+0000 7f66e5200700 1 --2- 172.21.15.45:0/743266931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf460fb60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66d8008f80 tx=0x7f66d801a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ada670295d8467cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:03.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.846+0000 7f66e5200700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f66c40a3c40 0x7f66c40a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f66d801aa10 tx=0x7f66d800e000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.847+0000 7f66d5600700 1 -- 172.21.15.45:0/743266931 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d80091d0 con 0x555bf460f780 2024-04-25T18:59:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.847+0000 7f66d5600700 1 -- 172.21.15.45:0/743266931 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d80088d0 con 0x555bf460f780 2024-04-25T18:59:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.847+0000 7f66d5600700 1 -- 172.21.15.45:0/743266931 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8003a00 con 0x555bf460f780 2024-04-25T18:59:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.847+0000 7f66f1b26b80 1 -- 172.21.15.45:0/743266931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 msgr2=0x555bf460fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.847+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/743266931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf460fb60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f66d8008f80 tx=0x7f66d801a6c0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.847+0000 7f66f1b26b80 1 -- 172.21.15.45:0/743266931 shutdown_connections 2024-04-25T18:59:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.847+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/743266931 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4609ef0 0x555bf460a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.847+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/743266931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf460fb60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.847+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/743266931 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf46100a0 0x555bf46099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.848+0000 7f66f1b26b80 1 -- 172.21.15.45:0/743266931 >> 172.21.15.45:0/743266931 conn(0x555bf4537f00 msgr2=0x555bf46048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.848+0000 7f66f1b26b80 1 -- 172.21.15.45:0/743266931 shutdown_connections 2024-04-25T18:59:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.848+0000 7f66f1b26b80 1 -- 172.21.15.45:0/743266931 wait complete. 2024-04-25T18:59:03.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.848+0000 7f66f1b26b80 1 -- start start 2024-04-25T18:59:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.849+0000 7f66f1b26b80 1 Processor -- start 2024-04-25T18:59:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.849+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4609ef0 0x555bf4615dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.849+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf460f780 0x555bf46130d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.849+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf4613610 0x555bf4613a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.849+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555bf46168c0 con 0x555bf460f780 2024-04-25T18:59:03.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.849+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555bf4515810 con 0x555bf4609ef0 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.849+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555bf4616a40 con 0x555bf4613610 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.849+0000 7f66f1b26b80 10 client.0 _ll_register_callbacks cb 0x555bf44f46d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66df400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf460f780 0x555bf46130d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66df400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf460f780 0x555bf46130d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36302/0 (socket says 172.21.15.45:36302) 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66e5200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf4613610 0x555bf4613a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66df400700 1 -- 172.21.15.45:0/256760047 learned_addr learned my addr 172.21.15.45:0/256760047 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66dfe00700 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4609ef0 0x555bf4615dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66e5200700 1 -- 172.21.15.45:0/256760047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4609ef0 msgr2=0x555bf4615dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66e5200700 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4609ef0 0x555bf4615dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66e5200700 1 -- 172.21.15.45:0/256760047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf460f780 msgr2=0x555bf46130d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66e5200700 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf460f780 0x555bf46130d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66e5200700 1 -- 172.21.15.45:0/256760047 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf44a9880 con 0x555bf4613610 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66df400700 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf460f780 0x555bf46130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66e5200700 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf4613610 0x555bf4613a90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f66d8004d90 tx=0x7f66d8004e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66a517af26d9171e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:03.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.850+0000 7f66ca000700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T18:59:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.851+0000 7f66ca000700 1 -- 172.21.15.45:0/256760047 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d80091d0 con 0x555bf4613610 2024-04-25T18:59:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.851+0000 7f66ca000700 1 -- 172.21.15.45:0/256760047 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f66d8028040 con 0x555bf4613610 2024-04-25T18:59:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.851+0000 7f66f1b26b80 1 -- 172.21.15.45:0/256760047 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d80088d0 con 0x555bf4613610 2024-04-25T18:59:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.851+0000 7f66ca000700 1 -- 172.21.15.45:0/256760047 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f66d801bd00 con 0x555bf4613610 2024-04-25T18:59:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.851+0000 7f66f1b26b80 10 client.25009 fetch_fsmap learned FSMap version 25 2024-04-25T18:59:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.851+0000 7f66f1b26b80 1 -- 172.21.15.45:0/256760047 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bf44f2aa0 con 0x555bf4613610 2024-04-25T18:59:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.851+0000 7f66ca000700 1 -- 172.21.15.45:0/256760047 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d80088d0 con 0x555bf4613610 2024-04-25T18:59:03.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.852+0000 7f66ca000700 1 -- 172.21.15.45:0/256760047 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f66d8007690 con 0x555bf4613610 2024-04-25T18:59:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.852+0000 7f66f1b26b80 10 client.25009 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:59:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.852+0000 7f66f1b26b80 10 client.25009 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T18:59:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.852+0000 7f66f1b26b80 20 client.25009 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.852+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x555bf4623710 0x555bf4624b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.852+0000 7f66f1b26b80 4 client.25009 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:59:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.852+0000 7f66f1b26b80 1 -- 172.21.15.45:0/256760047 --> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bf4569630 con 0x555bf4623710 2024-04-25T18:59:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.852+0000 7f66dfe00700 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x555bf4623710 0x555bf4624b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.853+0000 7f66dfe00700 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x555bf4623710 0x555bf4624b80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9713094869432ef7 server_cookie=5e04a4f98dc5054e in_seq=0 out_seq=0 2024-04-25T18:59:03.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.853+0000 7f66ca000700 10 client.25009 ms_handle_connect on v2:172.21.15.45:6840/2490735053 2024-04-25T18:59:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66ca000700 1 -- 172.21.15.45:0/256760047 <== mds.1 v2:172.21.15.45:6840/2490735053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555bf4569630 con 0x555bf4623710 2024-04-25T18:59:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66ca000700 10 client.25009 handle_command_reply: tid=0 2024-04-25T18:59:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 client.25009 shutdown 2024-04-25T18:59:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 -- 172.21.15.45:0/256760047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf4613610 msgr2=0x555bf4613a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf4613610 0x555bf4613a90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f66d8004d90 tx=0x7f66d8004e90 comp rx=0 tx=0).stop 2024-04-25T18:59:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 -- 172.21.15.45:0/256760047 shutdown_connections 2024-04-25T18:59:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x555bf4623710 0x555bf4624b80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4609ef0 0x555bf4615dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf4613610 0x555bf4613a90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66ca000700 0 client.25009 ms_handle_reset on v2:172.21.15.45:6840/2490735053 2024-04-25T18:59:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/256760047 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf460f780 0x555bf46130d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 -- 172.21.15.45:0/256760047 >> 172.21.15.45:0/256760047 conn(0x555bf4537f00 msgr2=0x555bf4603e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 -- 172.21.15.45:0/256760047 shutdown_connections 2024-04-25T18:59:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.854+0000 7f66f1b26b80 1 -- 172.21.15.45:0/256760047 wait complete. 2024-04-25T18:59:03.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.855+0000 7f66f1b26b80 20 client.25009 trim_cache size 0 max 16384 2024-04-25T18:59:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.877+0000 7f66f1b26b80 1 Processor -- start 2024-04-25T18:59:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.877+0000 7f66f1b26b80 1 -- start start 2024-04-25T18:59:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 0x555bf46099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf46045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 0x555bf462dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555bf4622ec0 con 0x555bf4609ef0 2024-04-25T18:59:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555bf4623590 con 0x555bf4613610 2024-04-25T18:59:03.880 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555bf4623c60 con 0x555bf460f780 2024-04-25T18:59:03.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66df400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf46045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66dfe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 0x555bf46099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.881 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66df400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf46045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32960/0 (socket says 172.21.15.45:32960) 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66df400700 1 -- 172.21.15.45:0/2614298680 learned_addr learned my addr 172.21.15.45:0/2614298680 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.878+0000 7f66e5200700 1 --2- 172.21.15.45:0/2614298680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 0x555bf462dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.879+0000 7f66df400700 1 -- 172.21.15.45:0/2614298680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 msgr2=0x555bf462dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.879+0000 7f66df400700 1 --2- 172.21.15.45:0/2614298680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 0x555bf462dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.879+0000 7f66df400700 1 -- 172.21.15.45:0/2614298680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 msgr2=0x555bf46099b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.879+0000 7f66df400700 1 --2- 172.21.15.45:0/2614298680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 0x555bf46099b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.879+0000 7f66df400700 1 -- 172.21.15.45:0/2614298680 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bf4606080 con 0x555bf460f780 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.879+0000 7f66df400700 1 --2- 172.21.15.45:0/2614298680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf46045d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f66cc099850 tx=0x7f66cc00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cc4faefdd5a1b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.879+0000 7f66d4c00700 1 -- 172.21.15.45:0/2614298680 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc09daf0 con 0x555bf460f780 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.879+0000 7f66d4c00700 1 -- 172.21.15.45:0/2614298680 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66cc0712f0 con 0x555bf460f780 2024-04-25T18:59:03.882 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.879+0000 7f66d4c00700 1 -- 172.21.15.45:0/2614298680 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc09d650 con 0x555bf460f780 2024-04-25T18:59:03.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.880+0000 7f66f1b26b80 1 -- 172.21.15.45:0/2614298680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 msgr2=0x555bf46045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.880+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/2614298680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf46045d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f66cc099850 tx=0x7f66cc00bf60 comp rx=0 tx=0).stop 2024-04-25T18:59:03.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.880+0000 7f66f1b26b80 1 -- 172.21.15.45:0/2614298680 shutdown_connections 2024-04-25T18:59:03.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.880+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/2614298680 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 0x555bf462dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.880+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/2614298680 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 0x555bf46099b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.883 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.880+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/2614298680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf46045d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.880+0000 7f66f1b26b80 1 -- 172.21.15.45:0/2614298680 >> 172.21.15.45:0/2614298680 conn(0x555bf4537f00 msgr2=0x555bf4611a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:03.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.880+0000 7f66f1b26b80 1 -- 172.21.15.45:0/2614298680 shutdown_connections 2024-04-25T18:59:03.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.880+0000 7f66f1b26b80 1 -- 172.21.15.45:0/2614298680 wait complete. 2024-04-25T18:59:03.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.881+0000 7f66f1b26b80 1 -- start start 2024-04-25T18:59:03.884 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.881+0000 7f66f1b26b80 1 Processor -- start 2024-04-25T18:59:03.885 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 0x555bf462e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf4628fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66f1b26b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 0x555bf4629550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555bf4624a00 con 0x555bf4609ef0 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555bf4624330 con 0x555bf4613610 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66f1b26b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555bf46250d0 con 0x555bf460f780 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66f1b26b80 10 client.0 _ll_register_callbacks cb 0x555bf44f46d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66dfe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 0x555bf462e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66e5200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 0x555bf4629550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66dfe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 0x555bf462e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36314/0 (socket says 172.21.15.45:36314) 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66dfe00700 1 -- 172.21.15.45:0/271039965 learned_addr learned my addr 172.21.15.45:0/271039965 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66df400700 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf4628fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66e5200700 1 -- 172.21.15.45:0/271039965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 msgr2=0x555bf4628fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:03.886 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66e5200700 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf4628fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66e5200700 1 -- 172.21.15.45:0/271039965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 msgr2=0x555bf462e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66e5200700 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 0x555bf462e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66e5200700 1 -- 172.21.15.45:0/271039965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bf4625ae0 con 0x555bf4613610 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.882+0000 7f66dfe00700 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 0x555bf462e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.883+0000 7f66e5200700 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 0x555bf4629550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f66d8008780 tx=0x7f66d801a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919de012e14edf67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.883+0000 7f66ca000700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.883+0000 7f66ca000700 1 -- 172.21.15.45:0/271039965 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8007be0 con 0x555bf4613610 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.883+0000 7f66ca000700 1 -- 172.21.15.45:0/271039965 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f66d8002030 con 0x555bf4613610 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.883+0000 7f66f1b26b80 1 -- 172.21.15.45:0/271039965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66cc0712f0 con 0x555bf4613610 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.883+0000 7f66ca000700 1 -- 172.21.15.45:0/271039965 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f66d8044070 con 0x555bf4613610 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.883+0000 7f66f1b26b80 10 client.25010 fetch_fsmap learned FSMap version 25 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.883+0000 7f66f1b26b80 1 -- 172.21.15.45:0/271039965 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555bf468b750 con 0x555bf4613610 2024-04-25T18:59:03.887 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.884+0000 7f66ca000700 1 -- 172.21.15.45:0/271039965 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66cc0712f0 con 0x555bf4613610 2024-04-25T18:59:03.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.884+0000 7f66ca000700 1 -- 172.21.15.45:0/271039965 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f66d8007d80 con 0x555bf4613610 2024-04-25T18:59:03.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.884+0000 7f66f1b26b80 10 client.25010 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:59:03.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.884+0000 7f66f1b26b80 10 client.25010 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T18:59:03.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.884+0000 7f66f1b26b80 20 client.25010 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:03.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.884+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x555bf46902a0 0x555bf468efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:03.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.884+0000 7f66f1b26b80 4 client.25010 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:59:03.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.884+0000 7f66f1b26b80 1 -- 172.21.15.45:0/271039965 --> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] -- command(tid 0: {"prefix": "status"}) -- 0x555bf45636d0 con 0x555bf46902a0 2024-04-25T18:59:03.889 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.885+0000 7f66dfe00700 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x555bf46902a0 0x555bf468efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.885+0000 7f66dfe00700 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x555bf46902a0 0x555bf468efb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a74330cfbb7bbba server_cookie=7f78ef27bb5b36c in_seq=0 out_seq=0 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.885+0000 7f66ca000700 10 client.25010 ms_handle_connect on v2:172.21.15.45:6840/2490735053 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.885+0000 7f66ca000700 1 -- 172.21.15.45:0/271039965 <== mds.1 v2:172.21.15.45:6840/2490735053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555bf45636d0 con 0x555bf46902a0 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.885+0000 7f66ca000700 10 client.25010 handle_command_reply: tid=0 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.885+0000 7f66f1b26b80 1 client.25010 shutdown 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 1 -- 172.21.15.45:0/271039965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 msgr2=0x555bf4629550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 0x555bf4629550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f66d8008780 tx=0x7f66d801a6c0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 1 -- 172.21.15.45:0/271039965 shutdown_connections 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x555bf46902a0 0x555bf468efb0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555bf4613610 0x555bf4629550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555bf4609ef0 0x555bf462e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66ca000700 0 client.25010 ms_handle_reset on v2:172.21.15.45:6840/2490735053 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 1 --2- 172.21.15.45:0/271039965 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555bf460f780 0x555bf4628fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 1 -- 172.21.15.45:0/271039965 >> 172.21.15.45:0/271039965 conn(0x555bf4537f00 msgr2=0x555bf4610490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:03.890 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 1 -- 172.21.15.45:0/271039965 shutdown_connections 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 1 -- 172.21.15.45:0/271039965 wait complete. 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.886+0000 7f66f1b26b80 20 client.25010 trim_cache size 0 max 16384 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24512, 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 201.521678933, 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 25, 2024-04-25T18:59:03.891 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-25T18:59:03.892 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:59:03.892 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 241.395838693 2024-04-25T18:59:03.892 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T18:59:03.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 -- 172.21.15.45:0/2261661800 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f66c40a3c40 msgr2=0x7f66c40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f66c40a3c40 0x7f66c40a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f66d801aa10 tx=0x7f66d800e000 comp rx=0 tx=0).stop 2024-04-25T18:59:03.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 -- 172.21.15.45:0/2261661800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66e0077890 msgr2=0x7f66e01a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:03.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66e0077890 0x7f66e01a5930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f66cc005910 tx=0x7f66cc00b670 comp rx=0 tx=0).stop 2024-04-25T18:59:03.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 -- 172.21.15.45:0/2261661800 shutdown_connections 2024-04-25T18:59:03.892 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f66c40a3c40 0x7f66c40a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66e0113900 0x7f66e019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66e0076f70 0x7f66e01a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 --2- 172.21.15.45:0/2261661800 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66e0077890 0x7f66e01a5930 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 -- 172.21.15.45:0/2261661800 >> 172.21.15.45:0/2261661800 conn(0x7f66e0100750 msgr2=0x7f66e0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 -- 172.21.15.45:0/2261661800 shutdown_connections 2024-04-25T18:59:03.893 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:03.887+0000 7f66e7200700 1 -- 172.21.15.45:0/2261661800 wait complete. 2024-04-25T18:59:04.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:04 smithi062 ceph-mon[32844]: pgmap v437: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:59:04.432 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:04 smithi045 ceph-mon[27776]: pgmap v437: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:59:04.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:04 smithi079 ceph-mon[33002]: pgmap v437: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T18:59:04.820 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T18:59:05.300 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:06.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:06 smithi079 ceph-mon[33002]: pgmap v438: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:59:06.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:06 smithi045 ceph-mon[27776]: pgmap v438: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:59:06.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:06 smithi062 ceph-mon[32844]: pgmap v438: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T18:59:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.503+0000 7f273b400700 1 -- 172.21.15.45:0/1006210184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2734107990 msgr2=0x7f2734107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.503+0000 7f273b400700 1 --2- 172.21.15.45:0/1006210184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2734107990 0x7f2734107d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2728009990 tx=0x7f2728009ca0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.503+0000 7f273b400700 1 -- 172.21.15.45:0/1006210184 shutdown_connections 2024-04-25T18:59:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.503+0000 7f273b400700 1 --2- 172.21.15.45:0/1006210184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2734107990 0x7f2734107d70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.503+0000 7f273b400700 1 --2- 172.21.15.45:0/1006210184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2734102300 0x7f273410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.503+0000 7f273b400700 1 --2- 172.21.15.45:0/1006210184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2734101940 0x7f2734101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.503+0000 7f273b400700 1 -- 172.21.15.45:0/1006210184 >> 172.21.15.45:0/1006210184 conn(0x7f27340788f0 msgr2=0x7f2734078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:07.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.503+0000 7f273b400700 1 -- 172.21.15.45:0/1006210184 shutdown_connections 2024-04-25T18:59:07.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.503+0000 7f273b400700 1 -- 172.21.15.45:0/1006210184 wait complete. 2024-04-25T18:59:07.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.504+0000 7f273b400700 1 Processor -- start 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f273b400700 1 -- start start 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f273b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2734101940 0x7f2734197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f273b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2734102300 0x7f2734198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f273b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2734107990 0x7f2734198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f273b400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2734111b00 con 0x7f2734102300 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f273b400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2734111980 con 0x7f2734101940 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f273b400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2734111c80 con 0x7f2734107990 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f2733e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2734102300 0x7f2734198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f2733e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2734102300 0x7f2734198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55172/0 (socket says 172.21.15.45:55172) 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f2739400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2734107990 0x7f2734198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f2733e00700 1 -- 172.21.15.45:0/975761773 learned_addr learned my addr 172.21.15.45:0/975761773 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.505+0000 7f2738a00700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2734101940 0x7f2734197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.506+0000 7f2739400700 1 -- 172.21.15.45:0/975761773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2734101940 msgr2=0x7f2734197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.506+0000 7f2739400700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2734101940 0x7f2734197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.506+0000 7f2739400700 1 -- 172.21.15.45:0/975761773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2734102300 msgr2=0x7f2734198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.506+0000 7f2739400700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2734102300 0x7f2734198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.506+0000 7f2739400700 1 -- 172.21.15.45:0/975761773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27341a7660 con 0x7f2734107990 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.506+0000 7f2738a00700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2734101940 0x7f2734197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.506+0000 7f2739400700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2734107990 0x7f2734198870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f272c00efc0 tx=0x7f272c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.506+0000 7f2731600700 1 -- 172.21.15.45:0/975761773 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c0092e0 con 0x7f2734107990 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.507+0000 7f2731600700 1 -- 172.21.15.45:0/975761773 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f272c004bb0 con 0x7f2734107990 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.507+0000 7f273b400700 1 -- 172.21.15.45:0/975761773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27341a78f0 con 0x7f2734107990 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.507+0000 7f2731600700 1 -- 172.21.15.45:0/975761773 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272c004020 con 0x7f2734107990 2024-04-25T18:59:07.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.507+0000 7f273b400700 1 -- 172.21.15.45:0/975761773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27341a7e80 con 0x7f2734107990 2024-04-25T18:59:07.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.510+0000 7f2731600700 1 -- 172.21.15.45:0/975761773 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f272c01b030 con 0x7f2734107990 2024-04-25T18:59:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.511+0000 7f2745e68b80 1 Processor -- start 2024-04-25T18:59:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.511+0000 7f2731600700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f271c0a3cc0 0x7f271c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2731600700 1 -- 172.21.15.45:0/975761773 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f272c0cfdf0 con 0x7f2734107990 2024-04-25T18:59:07.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2745e68b80 1 -- start start 2024-04-25T18:59:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2733e00700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f271c0a3cc0 0x7f271c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784609780 0x55f784609b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460a0a0 0x55f7846039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 0x55f784604330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2745e68b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f784555e40 con 0x55f78460a0a0 2024-04-25T18:59:07.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2745e68b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f7844fc6e0 con 0x55f784603ef0 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2745e68b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f784551e00 con 0x55f784609780 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2738a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784609780 0x55f784609b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2738a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784609780 0x55f784609b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:54802/0 (socket says 172.21.15.45:54802) 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2738a00700 1 -- 172.21.15.45:0/4136766299 learned_addr learned my addr 172.21.15.45:0/4136766299 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.512+0000 7f2739400700 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 0x55f784604330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2733e00700 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460a0a0 0x55f7846039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2738a00700 1 -- 172.21.15.45:0/4136766299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 msgr2=0x55f784604330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2738a00700 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 0x55f784604330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2738a00700 1 -- 172.21.15.45:0/4136766299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460a0a0 msgr2=0x55f7846039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2738a00700 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460a0a0 0x55f7846039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2738a00700 1 -- 172.21.15.45:0/4136766299 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7844f2630 con 0x55f784609780 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2739400700 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 0x55f784604330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2733e00700 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460a0a0 0x55f7846039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T18:59:07.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2733e00700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f271c0a3cc0 0x7f271c0a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2734199f50 tx=0x7f2720009980 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:07.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2738a00700 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784609780 0x55f784609b60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2728009990 tx=0x7f2728005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f88b83133c70538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:07.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2726000700 1 -- 172.21.15.45:0/4136766299 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728027070 con 0x55f784609780 2024-04-25T18:59:07.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2726000700 1 -- 172.21.15.45:0/4136766299 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2728003680 con 0x55f784609780 2024-04-25T18:59:07.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.513+0000 7f2726000700 1 -- 172.21.15.45:0/4136766299 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272800f6c0 con 0x55f784609780 2024-04-25T18:59:07.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.514+0000 7f2745e68b80 1 -- 172.21.15.45:0/4136766299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784609780 msgr2=0x55f784609b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.514+0000 7f2745e68b80 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784609780 0x55f784609b60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2728009990 tx=0x7f2728005960 comp rx=0 tx=0).stop 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.514+0000 7f2745e68b80 1 -- 172.21.15.45:0/4136766299 shutdown_connections 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.514+0000 7f2745e68b80 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 0x55f784604330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.514+0000 7f2745e68b80 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784609780 0x55f784609b60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.514+0000 7f2745e68b80 1 --2- 172.21.15.45:0/4136766299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460a0a0 0x55f7846039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.514+0000 7f2745e68b80 1 -- 172.21.15.45:0/4136766299 >> 172.21.15.45:0/4136766299 conn(0x55f784531f00 msgr2=0x55f7845fe8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.514+0000 7f2745e68b80 1 -- 172.21.15.45:0/4136766299 shutdown_connections 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.514+0000 7f2745e68b80 1 -- 172.21.15.45:0/4136766299 wait complete. 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.515+0000 7f2745e68b80 1 -- start start 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.515+0000 7f2745e68b80 1 Processor -- start 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 0x55f78460fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784609780 0x55f78460d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460d5e0 0x55f78460da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2745e68b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f7846108c0 con 0x55f78460d5e0 2024-04-25T18:59:07.519 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2745e68b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f78450f810 con 0x55f784609780 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2745e68b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f784610a40 con 0x55f784603ef0 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2745e68b80 10 client.0 _ll_register_callbacks cb 0x55f7844ee6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2733e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784609780 0x55f78460d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2739400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460d5e0 0x55f78460da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2733e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784609780 0x55f78460d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:33174/0 (socket says 172.21.15.45:33174) 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2733e00700 1 -- 172.21.15.45:0/384461586 learned_addr learned my addr 172.21.15.45:0/384461586 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.516+0000 7f2738a00700 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 0x55f78460fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f2733e00700 1 -- 172.21.15.45:0/384461586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 msgr2=0x55f78460fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f2733e00700 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 0x55f78460fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f2733e00700 1 -- 172.21.15.45:0/384461586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460d5e0 msgr2=0x55f78460da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f2733e00700 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460d5e0 0x55f78460da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f2733e00700 1 -- 172.21.15.45:0/384461586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7844a3880 con 0x55f784609780 2024-04-25T18:59:07.520 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f2738a00700 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 0x55f78460fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f2733e00700 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784609780 0x55f78460d0a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2720009820 tx=0x7f27200047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66ec622836908961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f271aa00700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T18:59:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f271aa00700 1 -- 172.21.15.45:0/384461586 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720023070 con 0x55f784609780 2024-04-25T18:59:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f271aa00700 1 -- 172.21.15.45:0/384461586 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f272001f800 con 0x55f784609780 2024-04-25T18:59:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.517+0000 7f2745e68b80 1 -- 172.21.15.45:0/384461586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2728003680 con 0x55f784609780 2024-04-25T18:59:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.518+0000 7f271aa00700 1 -- 172.21.15.45:0/384461586 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f272001fa00 con 0x55f784609780 2024-04-25T18:59:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.518+0000 7f2745e68b80 10 client.25022 fetch_fsmap learned FSMap version 25 2024-04-25T18:59:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.518+0000 7f2745e68b80 1 -- 172.21.15.45:0/384461586 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f7844ecaa0 con 0x55f784609780 2024-04-25T18:59:07.522 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.518+0000 7f271aa00700 1 -- 172.21.15.45:0/384461586 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2728003680 con 0x55f784609780 2024-04-25T18:59:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.518+0000 7f271aa00700 1 -- 172.21.15.45:0/384461586 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2720013ce0 con 0x55f784609780 2024-04-25T18:59:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.519+0000 7f2745e68b80 10 client.25022 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:59:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.519+0000 7f2745e68b80 10 client.25022 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T18:59:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.519+0000 7f2745e68b80 20 client.25022 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.519+0000 7f2745e68b80 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f78461daa0 0x55f78461ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.519+0000 7f2745e68b80 4 client.25022 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:59:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.519+0000 7f2745e68b80 1 -- 172.21.15.45:0/384461586 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f784563630 con 0x55f78461daa0 2024-04-25T18:59:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.519+0000 7f2738a00700 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f78461daa0 0x55f78461ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.520+0000 7f2738a00700 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f78461daa0 0x55f78461ef10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=233eef01e47f8262 server_cookie=885c14b70939f950 in_seq=0 out_seq=0 2024-04-25T18:59:07.523 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.520+0000 7f271aa00700 10 client.25022 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f271aa00700 1 -- 172.21.15.45:0/384461586 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f784563630 con 0x55f78461daa0 2024-04-25T18:59:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f271aa00700 10 client.25022 handle_command_reply: tid=0 2024-04-25T18:59:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 client.25022 shutdown 2024-04-25T18:59:07.524 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 -- 172.21.15.45:0/384461586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784609780 msgr2=0x55f78460d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784609780 0x55f78460d0a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2720009820 tx=0x7f27200047c0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 -- 172.21.15.45:0/384461586 shutdown_connections 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f78461daa0 0x55f78461ef10 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f78460d5e0 0x55f78460da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 0x55f78460fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 --2- 172.21.15.45:0/384461586 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784609780 0x55f78460d0a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f271aa00700 0 client.25022 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 -- 172.21.15.45:0/384461586 >> 172.21.15.45:0/384461586 conn(0x55f784531f00 msgr2=0x55f7845fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 -- 172.21.15.45:0/384461586 shutdown_connections 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.521+0000 7f2745e68b80 1 -- 172.21.15.45:0/384461586 wait complete. 2024-04-25T18:59:07.525 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.522+0000 7f2745e68b80 20 client.25022 trim_cache size 0 max 16384 2024-04-25T18:59:07.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.544+0000 7f2745e68b80 1 Processor -- start 2024-04-25T18:59:07.546 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2745e68b80 1 -- start start 2024-04-25T18:59:07.547 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 0x55f7846039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f78460d5e0 0x55f78460a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2745e68b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f78461d250 con 0x55f784609780 2024-04-25T18:59:07.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2745e68b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f78461d920 con 0x55f78460d5e0 2024-04-25T18:59:07.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2745e68b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f78461dff0 con 0x55f784603ef0 2024-04-25T18:59:07.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2733e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2733e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55186/0 (socket says 172.21.15.45:55186) 2024-04-25T18:59:07.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2733e00700 1 -- 172.21.15.45:0/2110265544 learned_addr learned my addr 172.21.15.45:0/2110265544 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:07.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2739400700 1 --2- 172.21.15.45:0/2110265544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f78460d5e0 0x55f78460a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.545+0000 7f2738a00700 1 --2- 172.21.15.45:0/2110265544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 0x55f7846039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.546+0000 7f2733e00700 1 -- 172.21.15.45:0/2110265544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 msgr2=0x55f7846039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.546+0000 7f2733e00700 1 --2- 172.21.15.45:0/2110265544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 0x55f7846039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.546+0000 7f2733e00700 1 -- 172.21.15.45:0/2110265544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f78460d5e0 msgr2=0x55f78460a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:07.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.546+0000 7f2733e00700 1 --2- 172.21.15.45:0/2110265544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f78460d5e0 0x55f78460a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.546+0000 7f2733e00700 1 -- 172.21.15.45:0/2110265544 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7845ff720 con 0x55f784609780 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.546+0000 7f2733e00700 1 --2- 172.21.15.45:0/2110265544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe5d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2720033420 tx=0x7f2720005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae0618bf6c93077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.546+0000 7f2725600700 1 -- 172.21.15.45:0/2110265544 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720023040 con 0x55f784609780 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.546+0000 7f2725600700 1 -- 172.21.15.45:0/2110265544 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2720012110 con 0x55f784609780 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.546+0000 7f2725600700 1 -- 172.21.15.45:0/2110265544 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720028910 con 0x55f784609780 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.547+0000 7f2745e68b80 1 -- 172.21.15.45:0/2110265544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 msgr2=0x55f7845fe5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.547+0000 7f2745e68b80 1 --2- 172.21.15.45:0/2110265544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe5d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2720033420 tx=0x7f2720005d00 comp rx=0 tx=0).stop 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.547+0000 7f2745e68b80 1 -- 172.21.15.45:0/2110265544 shutdown_connections 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.547+0000 7f2745e68b80 1 --2- 172.21.15.45:0/2110265544 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f78460d5e0 0x55f78460a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.547+0000 7f2745e68b80 1 --2- 172.21.15.45:0/2110265544 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f784603ef0 0x55f7846039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.547+0000 7f2745e68b80 1 --2- 172.21.15.45:0/2110265544 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe5d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.547+0000 7f2745e68b80 1 -- 172.21.15.45:0/2110265544 >> 172.21.15.45:0/2110265544 conn(0x55f784531f00 msgr2=0x55f7846042d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.547+0000 7f2745e68b80 1 -- 172.21.15.45:0/2110265544 shutdown_connections 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.547+0000 7f2745e68b80 1 -- 172.21.15.45:0/2110265544 wait complete. 2024-04-25T18:59:07.556 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.548+0000 7f2745e68b80 1 -- start start 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.548+0000 7f2745e68b80 1 Processor -- start 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 0x55f7845fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2745e68b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f78460d5e0 0x55f7845fe620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2745e68b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f78461ed90 con 0x55f784609780 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2745e68b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f78461e6c0 con 0x55f784603ef0 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2745e68b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f78461f460 con 0x55f78460d5e0 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2745e68b80 10 client.0 _ll_register_callbacks cb 0x7f27200107d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2733e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2733e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55202/0 (socket says 172.21.15.45:55202) 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2733e00700 1 -- 172.21.15.45:0/2103542970 learned_addr learned my addr 172.21.15.45:0/2103542970 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2738a00700 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 0x55f7845fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2739400700 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f78460d5e0 0x55f7845fe620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2733e00700 1 -- 172.21.15.45:0/2103542970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f78460d5e0 msgr2=0x55f7845fe620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:07.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2733e00700 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f78460d5e0 0x55f7845fe620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2733e00700 1 -- 172.21.15.45:0/2103542970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 msgr2=0x55f7845fdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.549+0000 7f2733e00700 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 0x55f7845fdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.550+0000 7f2733e00700 1 -- 172.21.15.45:0/2103542970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f784626700 con 0x55f784609780 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.550+0000 7f2733e00700 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe0e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f272001f9d0 tx=0x7f2720013be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c25f0562cf42b71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.550+0000 7f271aa00700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.550+0000 7f271aa00700 1 -- 172.21.15.45:0/2103542970 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720013e60 con 0x55f784609780 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.550+0000 7f271aa00700 1 -- 172.21.15.45:0/2103542970 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f2720005260 con 0x55f784609780 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.550+0000 7f2745e68b80 1 -- 172.21.15.45:0/2103542970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2720012110 con 0x55f784609780 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.550+0000 7f271aa00700 1 -- 172.21.15.45:0/2103542970 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2720023070 con 0x55f784609780 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.550+0000 7f2745e68b80 10 client.14904 fetch_fsmap learned FSMap version 25 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.551+0000 7f2745e68b80 1 -- 172.21.15.45:0/2103542970 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f784684f80 con 0x55f784609780 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.551+0000 7f271aa00700 1 -- 172.21.15.45:0/2103542970 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2720012110 con 0x55f784609780 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.551+0000 7f271aa00700 1 -- 172.21.15.45:0/2103542970 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2720045030 con 0x55f784609780 2024-04-25T18:59:07.558 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.551+0000 7f2745e68b80 10 client.14904 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.551+0000 7f2745e68b80 10 client.14904 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.551+0000 7f2745e68b80 20 client.14904 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.551+0000 7f2745e68b80 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f784689710 0x55f784688420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.551+0000 7f2745e68b80 4 client.14904 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.551+0000 7f2745e68b80 1 -- 172.21.15.45:0/2103542970 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "status"}) -- 0x55f78455d6d0 con 0x55f784689710 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.552+0000 7f2738a00700 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f784689710 0x55f784688420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.552+0000 7f2738a00700 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f784689710 0x55f784688420 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5aac23424c02ab0 server_cookie=1c551b6abda20ca5 in_seq=0 out_seq=0 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.552+0000 7f271aa00700 10 client.14904 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.552+0000 7f271aa00700 1 -- 172.21.15.45:0/2103542970 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f78455d6d0 con 0x55f784689710 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.552+0000 7f271aa00700 10 client.14904 handle_command_reply: tid=0 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.552+0000 7f2745e68b80 1 client.14904 shutdown 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 1 -- 172.21.15.45:0/2103542970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 msgr2=0x55f7845fe0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe0e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f272001f9d0 tx=0x7f2720013be0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 1 -- 172.21.15.45:0/2103542970 shutdown_connections 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f784689710 0x55f784688420 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f784603ef0 0x55f7845fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f78460d5e0 0x55f7845fe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 1 --2- 172.21.15.45:0/2103542970 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f784609780 0x55f7845fe0e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 1 -- 172.21.15.45:0/2103542970 >> 172.21.15.45:0/2103542970 conn(0x55f784531f00 msgr2=0x55f78460a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f271aa00700 0 client.14904 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 1 -- 172.21.15.45:0/2103542970 shutdown_connections 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 1 -- 172.21.15.45:0/2103542970 wait complete. 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.553+0000 7f2745e68b80 20 client.14904 trim_cache size 0 max 16384 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T18:59:07.560 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24524, 2024-04-25T18:59:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T18:59:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T18:59:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T18:59:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 221.313546823, 2024-04-25T18:59:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 25, 2024-04-25T18:59:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 94, 2024-04-25T18:59:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T18:59:07.561 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 242.14276889199999 2024-04-25T18:59:07.561 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 -- 172.21.15.45:0/975761773 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f271c0a3cc0 msgr2=0x7f271c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f271c0a3cc0 0x7f271c0a6180 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2734199f50 tx=0x7f2720009980 comp rx=0 tx=0).stop 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 -- 172.21.15.45:0/975761773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2734107990 msgr2=0x7f2734198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2734107990 0x7f2734198870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f272c00efc0 tx=0x7f272c00c3a0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 -- 172.21.15.45:0/975761773 shutdown_connections 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f271c0a3cc0 0x7f271c0a6180 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2734101940 0x7f2734197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2734107990 0x7f2734198870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 --2- 172.21.15.45:0/975761773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2734102300 0x7f2734198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 -- 172.21.15.45:0/975761773 >> 172.21.15.45:0/975761773 conn(0x7f27340788f0 msgr2=0x7f27341007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 -- 172.21.15.45:0/975761773 shutdown_connections 2024-04-25T18:59:07.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:07.554+0000 7f273b400700 1 -- 172.21.15.45:0/975761773 wait complete. 2024-04-25T18:59:08.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:08 smithi045 ceph-mon[27776]: pgmap v439: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:59:08.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:08 smithi079 ceph-mon[33002]: pgmap v439: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:59:08.477 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T18:59:08.487 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T18:59:08.488 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T18:59:08.488 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T18:59:08.488 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T18:59:08.489 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T18:59:08.489 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T18:59:08.489 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T18:59:08.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:08 smithi062 ceph-mon[32844]: pgmap v439: 97 pgs: 97 active+clean; 645 KiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T18:59:08.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.695+0000 7f20b7400640 1 -- 172.21.15.45:0/3825691777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b01053f0 msgr2=0x7f20b0105870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:08.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.695+0000 7f20b7400640 1 --2- 172.21.15.45:0/3825691777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b01053f0 0x7f20b0105870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009900 tx=0x7f20a002a410 comp rx=0 tx=0).stop 2024-04-25T18:59:08.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.695+0000 7f20b7400640 1 -- 172.21.15.45:0/3825691777 shutdown_connections 2024-04-25T18:59:08.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.695+0000 7f20b7400640 1 --2- 172.21.15.45:0/3825691777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0105db0 0x7f20b010c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:08.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.695+0000 7f20b7400640 1 --2- 172.21.15.45:0/3825691777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b01053f0 0x7f20b0105870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:08.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.695+0000 7f20b7400640 1 --2- 172.21.15.45:0/3825691777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b0104200 0x7f20b0104620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:08.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.696+0000 7f20b7400640 1 -- 172.21.15.45:0/3825691777 >> 172.21.15.45:0/3825691777 conn(0x7f20b00ff770 msgr2=0x7f20b0101bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:08.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.696+0000 7f20b7400640 1 -- 172.21.15.45:0/3825691777 shutdown_connections 2024-04-25T18:59:08.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.696+0000 7f20b7400640 1 -- 172.21.15.45:0/3825691777 wait complete. 2024-04-25T18:59:08.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b7400640 1 Processor -- start 2024-04-25T18:59:08.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b7400640 1 -- start start 2024-04-25T18:59:08.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b7400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b0104200 0x7f20b019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:08.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b01053f0 0x7f20b019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:08.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0105db0 0x7f20b01a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:08.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b7400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20b010fd40 con 0x7f20b0104200 2024-04-25T18:59:08.701 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b7400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f20b010fbc0 con 0x7f20b0105db0 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b7400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f20b010fec0 con 0x7f20b01053f0 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b0104200 0x7f20b019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b0104200 0x7f20b019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55218/0 (socket says 172.21.15.45:55218) 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20b4a00640 1 -- 172.21.15.45:0/2674352173 learned_addr learned my addr 172.21.15.45:0/2674352173 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.697+0000 7f20afe00640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b01053f0 0x7f20b019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20b5400640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0105db0 0x7f20b01a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20b4a00640 1 -- 172.21.15.45:0/2674352173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b01053f0 msgr2=0x7f20b019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20b4a00640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b01053f0 0x7f20b019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20b4a00640 1 -- 172.21.15.45:0/2674352173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0105db0 msgr2=0x7f20b01a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20b4a00640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0105db0 0x7f20b01a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20b4a00640 1 -- 172.21.15.45:0/2674352173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b01a2040 con 0x7f20b0104200 2024-04-25T18:59:08.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20b5400640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0105db0 0x7f20b01a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:59:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20b4a00640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b0104200 0x7f20b019a420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20a400e9c0 tx=0x7f20a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20ad600640 1 -- 172.21.15.45:0/2674352173 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a400cd80 con 0x7f20b0104200 2024-04-25T18:59:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20ad600640 1 -- 172.21.15.45:0/2674352173 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20a40035f0 con 0x7f20b0104200 2024-04-25T18:59:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20ad600640 1 -- 172.21.15.45:0/2674352173 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4010690 con 0x7f20b0104200 2024-04-25T18:59:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.698+0000 7f20b7400640 1 -- 172.21.15.45:0/2674352173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b01a2330 con 0x7f20b0104200 2024-04-25T18:59:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.699+0000 7f20b7400640 1 -- 172.21.15.45:0/2674352173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b01a2800 con 0x7f20b0104200 2024-04-25T18:59:08.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.700+0000 7f20b7400640 1 -- 172.21.15.45:0/2674352173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b010b590 con 0x7f20b0104200 2024-04-25T18:59:08.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.703+0000 7f20ad600640 1 -- 172.21.15.45:0/2674352173 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f20a4004000 con 0x7f20b0104200 2024-04-25T18:59:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.703+0000 7f20ad600640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f20980a3ba0 0x7f20980a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.703+0000 7f20ad600640 1 -- 172.21.15.45:0/2674352173 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f20a40d0a20 con 0x7f20b0104200 2024-04-25T18:59:08.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.704+0000 7f20afe00640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f20980a3ba0 0x7f20980a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:08.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.704+0000 7f20afe00640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f20980a3ba0 0x7f20980a6060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f20a0000c00 tx=0x7f20a00314a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:08.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.708+0000 7f20ad600640 1 -- 172.21.15.45:0/2674352173 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20a409d380 con 0x7f20b0104200 2024-04-25T18:59:08.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:08.969+0000 7f20b7400640 1 -- 172.21.15.45:0/2674352173 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f20b0062b50 con 0x7f20b0104200 2024-04-25T18:59:09.138 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.136+0000 7f20ad600640 1 -- 172.21.15.45:0/2674352173 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f20a409cc80 con 0x7f20b0104200 2024-04-25T18:59:09.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.139+0000 7f20b7400640 1 -- 172.21.15.45:0/2674352173 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f20980a3ba0 msgr2=0x7f20980a6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:09.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.140+0000 7f20b7400640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f20980a3ba0 0x7f20980a6060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f20a0000c00 tx=0x7f20a00314a0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.140+0000 7f20b7400640 1 -- 172.21.15.45:0/2674352173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b0104200 msgr2=0x7f20b019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:09.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.140+0000 7f20b7400640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b0104200 0x7f20b019a420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20a400e9c0 tx=0x7f20a400ee80 comp rx=0 tx=0).stop 2024-04-25T18:59:09.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.140+0000 7f20b7400640 1 -- 172.21.15.45:0/2674352173 shutdown_connections 2024-04-25T18:59:09.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.140+0000 7f20b7400640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f20b0105db0 0x7f20b01a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.140+0000 7f20b7400640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f20980a3ba0 0x7f20980a6060 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.140+0000 7f20b7400640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f20b01053f0 0x7f20b019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.140+0000 7f20b7400640 1 --2- 172.21.15.45:0/2674352173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20b0104200 0x7f20b019a420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.140+0000 7f20b7400640 1 -- 172.21.15.45:0/2674352173 >> 172.21.15.45:0/2674352173 conn(0x7f20b00ff770 msgr2=0x7f20b0108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:09.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.140+0000 7f20b7400640 1 -- 172.21.15.45:0/2674352173 shutdown_connections 2024-04-25T18:59:09.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.141+0000 7f20b7400640 1 -- 172.21.15.45:0/2674352173 wait complete. 2024-04-25T18:59:09.171 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T18:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.359+0000 7fdee8c00640 1 -- 172.21.15.45:0/2854068032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee41041e0 msgr2=0x7fdee4104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.359+0000 7fdee8c00640 1 --2- 172.21.15.45:0/2854068032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee41041e0 0x7fdee4104600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fded4009ca0 tx=0x7fded402a8c0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.359+0000 7fdee8c00640 1 -- 172.21.15.45:0/2854068032 shutdown_connections 2024-04-25T18:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.359+0000 7fdee8c00640 1 --2- 172.21.15.45:0/2854068032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdee4105d90 0x7fdee410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.359+0000 7fdee8c00640 1 --2- 172.21.15.45:0/2854068032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdee41053d0 0x7fdee4105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.359+0000 7fdee8c00640 1 --2- 172.21.15.45:0/2854068032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee41041e0 0x7fdee4104600 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.359+0000 7fdee8c00640 1 -- 172.21.15.45:0/2854068032 >> 172.21.15.45:0/2854068032 conn(0x7fdee40ff770 msgr2=0x7fdee4101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.359+0000 7fdee8c00640 1 -- 172.21.15.45:0/2854068032 shutdown_connections 2024-04-25T18:59:09.361 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.359+0000 7fdee8c00640 1 -- 172.21.15.45:0/2854068032 wait complete. 2024-04-25T18:59:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.360+0000 7fdee8c00640 1 Processor -- start 2024-04-25T18:59:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee8c00640 1 -- start start 2024-04-25T18:59:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee8c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdee41041e0 0x7fdee419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee8c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdee41053d0 0x7fdee419a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee8c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee4105d90 0x7fdee41a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee8c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdee410fd70 con 0x7fdee41053d0 2024-04-25T18:59:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee8c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdee410fbf0 con 0x7fdee4105d90 2024-04-25T18:59:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee8c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdee410fef0 con 0x7fdee41041e0 2024-04-25T18:59:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee1e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdee41041e0 0x7fdee419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:09.363 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee1e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdee41041e0 0x7fdee419a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:54854/0 (socket says 172.21.15.45:54854) 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee1e00640 1 -- 172.21.15.45:0/2807684328 learned_addr learned my addr 172.21.15.45:0/2807684328 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fdee2800640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee4105d90 0x7fdee41a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.361+0000 7fded9400640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdee41053d0 0x7fdee419a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdee1e00640 1 -- 172.21.15.45:0/2807684328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee4105d90 msgr2=0x7fdee41a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdee1e00640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee4105d90 0x7fdee41a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdee1e00640 1 -- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdee41053d0 msgr2=0x7fdee419a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdee1e00640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdee41053d0 0x7fdee419a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdee1e00640 1 -- 172.21.15.45:0/2807684328 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee41a2040 con 0x7fdee41041e0 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdee2800640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee4105d90 0x7fdee41a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdee1e00640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdee41041e0 0x7fdee419a420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fded4005e30 tx=0x7fded40317e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdedb400640 1 -- 172.21.15.45:0/2807684328 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4003bf0 con 0x7fdee41041e0 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdedb400640 1 -- 172.21.15.45:0/2807684328 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded4003d90 con 0x7fdee41041e0 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdee8c00640 1 -- 172.21.15.45:0/2807684328 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee41a22d0 con 0x7fdee41041e0 2024-04-25T18:59:09.364 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdedb400640 1 -- 172.21.15.45:0/2807684328 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded402fc50 con 0x7fdee41041e0 2024-04-25T18:59:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.362+0000 7fdee8c00640 1 -- 172.21.15.45:0/2807684328 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee41a2820 con 0x7fdee41041e0 2024-04-25T18:59:09.365 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.364+0000 7fdee8c00640 1 -- 172.21.15.45:0/2807684328 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee410b550 con 0x7fdee41041e0 2024-04-25T18:59:09.366 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.365+0000 7fdedb400640 1 -- 172.21.15.45:0/2807684328 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fded4004040 con 0x7fdee41041e0 2024-04-25T18:59:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.367+0000 7fdedb400640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdec80a3bf0 0x7fdec80a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.367+0000 7fded9400640 1 -- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdec80a3bf0 msgr2=0x7fdec80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/943541413 2024-04-25T18:59:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.367+0000 7fdedb400640 1 -- 172.21.15.45:0/2807684328 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fded40f1830 con 0x7fdee41041e0 2024-04-25T18:59:09.368 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.367+0000 7fded9400640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdec80a3bf0 0x7fdec80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:59:09.371 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.371+0000 7fdedb400640 1 -- 172.21.15.45:0/2807684328 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fded40be220 con 0x7fdee41041e0 2024-04-25T18:59:09.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:09 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2674352173' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T18:59:09.434 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: ignoring --setuser ceph since I am not root 2024-04-25T18:59:09.434 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: ignoring --setgroup ceph since I am not root 2024-04-25T18:59:09.434 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:09 smithi079 ceph-mgr[34876]: -- 172.21.15.79:0/3988361221 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559751d285a0 con 0x559751cfd000 2024-04-25T18:59:09.434 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:09.318+0000 7f9871640200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:59:09.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:09 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: ignoring --setuser ceph since I am not root 2024-04-25T18:59:09.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:09 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: ignoring --setgroup ceph since I am not root 2024-04-25T18:59:09.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:09 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:09.461+0000 7f93379ba200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:59:09.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:09 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2674352173' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T18:59:09.521 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:09 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2674352173' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T18:59:09.521 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: ignoring --setuser ceph since I am not root 2024-04-25T18:59:09.521 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: ignoring --setgroup ceph since I am not root 2024-04-25T18:59:09.521 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:09 smithi062 ceph-mgr[34246]: -- 172.21.15.62:0/1275933713 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55666203e5a0 con 0x556662012800 2024-04-25T18:59:09.521 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:09.329+0000 7f4ff5087200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T18:59:09.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.568+0000 7fded9400640 1 -- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdec80a3bf0 msgr2=0x7fdec80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/943541413 2024-04-25T18:59:09.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.568+0000 7fded9400640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdec80a3bf0 0x7fdec80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:59:09.584 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.584+0000 7fdee8c00640 1 -- 172.21.15.45:0/2807684328 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fdee402d5c0 con 0x7fdee41041e0 2024-04-25T18:59:09.593 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.592+0000 7fdedb400640 1 -- 172.21.15.45:0/2807684328 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7fded40bdb20 con 0x7fdee41041e0 2024-04-25T18:59:09.596 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 -- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdec80a3bf0 msgr2=0x7fdec80a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdec80a3bf0 0x7fdec80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 -- 172.21.15.45:0/2807684328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdee41041e0 msgr2=0x7fdee419a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdee41041e0 0x7fdee419a420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fded4005e30 tx=0x7fded40317e0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 -- 172.21.15.45:0/2807684328 shutdown_connections 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdee4105d90 0x7fdee41a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdee41053d0 0x7fdee419a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fdec80a3bf0 0x7fdec80a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 --2- 172.21.15.45:0/2807684328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdee41041e0 0x7fdee419a420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 -- 172.21.15.45:0/2807684328 >> 172.21.15.45:0/2807684328 conn(0x7fdee40ff770 msgr2=0x7fdee4108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.595+0000 7fdee8c00640 1 -- 172.21.15.45:0/2807684328 shutdown_connections 2024-04-25T18:59:09.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.596+0000 7fdee8c00640 1 -- 172.21.15.45:0/2807684328 wait complete. 2024-04-25T18:59:09.614 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T18:59:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.804+0000 7f3358400640 1 -- 172.21.15.45:0/1695858989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f33501041e0 msgr2=0x7f3350104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.804+0000 7f3358400640 1 --2- 172.21.15.45:0/1695858989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f33501041e0 0x7f3350104600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3344009900 tx=0x7f334402a3f0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.804+0000 7f3358400640 1 -- 172.21.15.45:0/1695858989 shutdown_connections 2024-04-25T18:59:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.804+0000 7f3358400640 1 --2- 172.21.15.45:0/1695858989 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3350105d90 0x7f335010c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.804+0000 7f3358400640 1 --2- 172.21.15.45:0/1695858989 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33501053d0 0x7f3350105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.804+0000 7f3358400640 1 --2- 172.21.15.45:0/1695858989 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f33501041e0 0x7f3350104600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.804+0000 7f3358400640 1 -- 172.21.15.45:0/1695858989 >> 172.21.15.45:0/1695858989 conn(0x7f33500ff770 msgr2=0x7f3350101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.804+0000 7f3358400640 1 -- 172.21.15.45:0/1695858989 shutdown_connections 2024-04-25T18:59:09.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.804+0000 7f3358400640 1 -- 172.21.15.45:0/1695858989 wait complete. 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.805+0000 7f3358400640 1 Processor -- start 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.805+0000 7f3358400640 1 -- start start 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.805+0000 7f3358400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33501041e0 0x7f335019a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.805+0000 7f3358400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33501053d0 0x7f335019a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.805+0000 7f3358400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350105d90 0x7f33501a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.805+0000 7f3358400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f335010fd40 con 0x7f33501041e0 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.805+0000 7f3358400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f335010fbc0 con 0x7f33501053d0 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.805+0000 7f3358400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f335010fec0 con 0x7f3350105d90 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33501041e0 0x7f335019a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33501041e0 0x7f335019a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55278/0 (socket says 172.21.15.45:55278) 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355a00640 1 -- 172.21.15.45:0/2629883826 learned_addr learned my addr 172.21.15.45:0/2629883826 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:09.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355000640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33501053d0 0x7f335019a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3356400640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350105d90 0x7f33501a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355a00640 1 -- 172.21.15.45:0/2629883826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350105d90 msgr2=0x7f33501a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355a00640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350105d90 0x7f33501a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355a00640 1 -- 172.21.15.45:0/2629883826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33501053d0 msgr2=0x7f335019a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355a00640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33501053d0 0x7f335019a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355a00640 1 -- 172.21.15.45:0/2629883826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33501a2040 con 0x7f33501041e0 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355000640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33501053d0 0x7f335019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.806+0000 7f3355a00640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33501041e0 0x7f335019a420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3344031500 tx=0x7f3344003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.807+0000 7f334e000640 1 -- 172.21.15.45:0/2629883826 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334403f070 con 0x7f33501041e0 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.807+0000 7f334e000640 1 -- 172.21.15.45:0/2629883826 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3344031c50 con 0x7f33501041e0 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.807+0000 7f3358400640 1 -- 172.21.15.45:0/2629883826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33501a22d0 con 0x7f33501041e0 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.807+0000 7f334e000640 1 -- 172.21.15.45:0/2629883826 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3344035030 con 0x7f33501041e0 2024-04-25T18:59:09.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.807+0000 7f3358400640 1 -- 172.21.15.45:0/2629883826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33501a27c0 con 0x7f33501041e0 2024-04-25T18:59:09.810 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.808+0000 7f3358400640 1 -- 172.21.15.45:0/2629883826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f335010b580 con 0x7f33501041e0 2024-04-25T18:59:09.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.810+0000 7f334e000640 1 -- 172.21.15.45:0/2629883826 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f3344042070 con 0x7f33501041e0 2024-04-25T18:59:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.811+0000 7f334e000640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f33380a3bf0 0x7f33380a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.812+0000 7f3355000640 1 -- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f33380a3bf0 msgr2=0x7f33380a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/943541413 2024-04-25T18:59:09.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.812+0000 7f334e000640 1 -- 172.21.15.45:0/2629883826 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f33440f0b70 con 0x7f33501041e0 2024-04-25T18:59:09.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.812+0000 7f3355000640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f33380a3bf0 0x7f33380a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:59:09.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:09.820+0000 7f334e000640 1 -- 172.21.15.45:0/2629883826 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f33440bd530 con 0x7f33501041e0 2024-04-25T18:59:09.863 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:09.518+0000 7f4ff5087200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:59:09.863 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:09.624+0000 7f4ff5087200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:59:09.906 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:09.500+0000 7f9871640200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:59:09.906 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:09.602+0000 7f9871640200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:59:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.013+0000 7f3355000640 1 -- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f33380a3bf0 msgr2=0x7f33380a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/943541413 2024-04-25T18:59:10.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.013+0000 7f3355000640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f33380a3bf0 0x7f33380a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:59:10.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.024+0000 7f3358400640 1 -- 172.21.15.45:0/2629883826 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f335002d5c0 con 0x7f33501041e0 2024-04-25T18:59:10.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.032+0000 7f334e000640 1 -- 172.21.15.45:0/2629883826 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f33440bce30 con 0x7f33501041e0 2024-04-25T18:59:10.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.034+0000 7f3358400640 1 -- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f33380a3bf0 msgr2=0x7f33380a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:59:10.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.034+0000 7f3358400640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f33380a3bf0 0x7f33380a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:10.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.034+0000 7f3358400640 1 -- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33501041e0 msgr2=0x7f335019a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:10.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.034+0000 7f3358400640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33501041e0 0x7f335019a420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3344031500 tx=0x7f3344003c10 comp rx=0 tx=0).stop 2024-04-25T18:59:10.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.034+0000 7f3358400640 1 -- 172.21.15.45:0/2629883826 shutdown_connections 2024-04-25T18:59:10.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.034+0000 7f3358400640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f33380a3bf0 0x7f33380a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:10.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.034+0000 7f3358400640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350105d90 0x7f33501a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:10.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.034+0000 7f3358400640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f33501053d0 0x7f335019a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:10.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.034+0000 7f3358400640 1 --2- 172.21.15.45:0/2629883826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f33501041e0 0x7f335019a420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:10.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.034+0000 7f3358400640 1 -- 172.21.15.45:0/2629883826 >> 172.21.15.45:0/2629883826 conn(0x7f33500ff770 msgr2=0x7f3350108750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:10.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.035+0000 7f3358400640 1 -- 172.21.15.45:0/2629883826 shutdown_connections 2024-04-25T18:59:10.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:10.035+0000 7f3358400640 1 -- 172.21.15.45:0/2629883826 wait complete. 2024-04-25T18:59:10.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:09 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:09.725+0000 7f93379ba200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:59:10.051 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:09 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:09.867+0000 7f93379ba200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T18:59:10.054 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T18:59:10.078 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T18:59:10.078 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T18:59:10.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:59:10.079 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T18:59:10.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:59:10.130 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-25T18:59:10.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:10 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2674352173' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T18:59:10.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:10 smithi079 ceph-mon[33002]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-04-25T18:59:10.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:10 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' 2024-04-25T18:59:10.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:10 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2629883826' entity='client.admin' 2024-04-25T18:59:10.184 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:09.904+0000 7f9871640200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:59:10.184 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:09.987+0000 7f9871640200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:59:10.210 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-04-25T18:59:10.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T18:59:10.214 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T18:59:10.214 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T18:59:10.214 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T18:59:10.215 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T18:59:10.363 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:09.942+0000 7f4ff5087200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:59:10.363 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:10 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:10.030+0000 7f4ff5087200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:59:10.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:10 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2674352173' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T18:59:10.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:10 smithi062 ceph-mon[32844]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-04-25T18:59:10.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:10 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' 2024-04-25T18:59:10.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:10 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2629883826' entity='client.admin' 2024-04-25T18:59:10.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:10 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2674352173' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T18:59:10.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:10 smithi045 ceph-mon[27776]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-04-25T18:59:10.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:10 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' 2024-04-25T18:59:10.412 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:10 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2629883826' entity='client.admin' 2024-04-25T18:59:10.412 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:10 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:10.288+0000 7f93379ba200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T18:59:10.658 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:10.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:10 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:10.410+0000 7f93379ba200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T18:59:11.434 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:11.146+0000 7f9871640200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:59:11.434 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:11.314+0000 7f9871640200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:59:11.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:11.265+0000 7f4ff5087200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:59:11.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:11.441+0000 7f4ff5087200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:59:11.767 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:11.482+0000 7f9871640200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:59:11.918 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:11.617+0000 7f4ff5087200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:59:12.056 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:11.764+0000 7f9871640200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:59:12.056 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:11.855+0000 7f9871640200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:59:12.056 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:12 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:12.054+0000 7f9871640200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:59:12.213 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:12 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:12.060+0000 7f93379ba200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T18:59:12.222 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:11.916+0000 7f4ff5087200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:59:12.222 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:12 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:12.011+0000 7f4ff5087200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:59:12.434 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:12 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:12.243+0000 7f9871640200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:59:12.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:12 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:12.326+0000 7f93379ba200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T18:59:12.567 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:12 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:12.564+0000 7f93379ba200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T18:59:12.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:12 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:12.220+0000 7f4ff5087200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:59:12.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:12 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:12.420+0000 7f4ff5087200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:59:12.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.764+0000 7f1398800700 1 -- 172.21.15.45:0/1254861217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1390103450 msgr2=0x7f139010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:12.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.764+0000 7f1398800700 1 --2- 172.21.15.45:0/1254861217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1390103450 0x7f139010f8b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f138c00b0a0 tx=0x7f138c00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:59:12.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.764+0000 7f1398800700 1 -- 172.21.15.45:0/1254861217 shutdown_connections 2024-04-25T18:59:12.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.764+0000 7f1398800700 1 --2- 172.21.15.45:0/1254861217 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1390108a50 0x7f1390108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:12.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.764+0000 7f1398800700 1 --2- 172.21.15.45:0/1254861217 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1390103450 0x7f139010f8b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:12.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.764+0000 7f1398800700 1 --2- 172.21.15.45:0/1254861217 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390102a90 0x7f1390102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:12.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.764+0000 7f1398800700 1 -- 172.21.15.45:0/1254861217 >> 172.21.15.45:0/1254861217 conn(0x7f1390079ba0 msgr2=0x7f1390079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:12.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.764+0000 7f1398800700 1 -- 172.21.15.45:0/1254861217 shutdown_connections 2024-04-25T18:59:12.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.764+0000 7f1398800700 1 -- 172.21.15.45:0/1254861217 wait complete. 2024-04-25T18:59:12.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.766+0000 7f1398800700 1 Processor -- start 2024-04-25T18:59:12.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.766+0000 7f1398800700 1 -- start start 2024-04-25T18:59:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.766+0000 7f1398800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1390102a90 0x7f139019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:12.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.766+0000 7f1398800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1390103450 0x7f139019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.766+0000 7f1398800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390108a50 0x7f139019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1398800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1390112be0 con 0x7f1390102a90 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1398800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1390112a60 con 0x7f1390103450 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1398800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1390112d60 con 0x7f1390108a50 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1395400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1390103450 0x7f139019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1395e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1390102a90 0x7f139019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1395400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1390103450 0x7f139019b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:33260/0 (socket says 172.21.15.45:33260) 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1395400700 1 -- 172.21.15.45:0/1342785644 learned_addr learned my addr 172.21.15.45:0/1342785644 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1396800700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390108a50 0x7f139019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1395400700 1 -- 172.21.15.45:0/1342785644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390108a50 msgr2=0x7f139019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1395400700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390108a50 0x7f139019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1395400700 1 -- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1390102a90 msgr2=0x7f139019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1395400700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1390102a90 0x7f139019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:12.769 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.767+0000 7f1395400700 1 -- 172.21.15.45:0/1342785644 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13901a0380 con 0x7f1390103450 2024-04-25T18:59:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.768+0000 7f1395400700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1390103450 0x7f139019b8b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f138000e9a0 tx=0x7f138000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.768+0000 7f138aa00700 1 -- 172.21.15.45:0/1342785644 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138000ca10 con 0x7f1390103450 2024-04-25T18:59:12.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.768+0000 7f138aa00700 1 -- 172.21.15.45:0/1342785644 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f138000cbb0 con 0x7f1390103450 2024-04-25T18:59:12.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.768+0000 7f1398800700 1 -- 172.21.15.45:0/1342785644 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13901a0670 con 0x7f1390103450 2024-04-25T18:59:12.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.768+0000 7f138aa00700 1 -- 172.21.15.45:0/1342785644 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380018680 con 0x7f1390103450 2024-04-25T18:59:12.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.768+0000 7f1398800700 1 -- 172.21.15.45:0/1342785644 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13901a7ef0 con 0x7f1390103450 2024-04-25T18:59:12.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.769+0000 7f1398800700 1 -- 172.21.15.45:0/1342785644 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1390069730 con 0x7f1390103450 2024-04-25T18:59:12.773 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.772+0000 7f138aa00700 1 -- 172.21.15.45:0/1342785644 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f1380010430 con 0x7f1390103450 2024-04-25T18:59:12.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.774+0000 7f138aa00700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13780a3d10 0x7f13780a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:12.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.774+0000 7f1395e00700 1 -- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13780a3d10 msgr2=0x7f13780a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/943541413 2024-04-25T18:59:12.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.774+0000 7f138aa00700 1 -- 172.21.15.45:0/1342785644 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7f13800d0610 con 0x7f1390103450 2024-04-25T18:59:12.775 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.774+0000 7f1395e00700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13780a3d10 0x7f13780a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:59:12.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.778+0000 7f138aa00700 1 -- 172.21.15.45:0/1342785644 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f138009cf70 con 0x7f1390103450 2024-04-25T18:59:12.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.975+0000 7f1395e00700 1 -- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13780a3d10 msgr2=0x7f13780a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/943541413 2024-04-25T18:59:12.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:12.975+0000 7f1395e00700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13780a3d10 0x7f13780a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:59:13.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.052+0000 7f1398800700 1 -- 172.21.15.45:0/1342785644 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f13901a80c0 con 0x7f1390103450 2024-04-25T18:59:13.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.053+0000 7f138aa00700 1 -- 172.21.15.45:0/1342785644 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f138009c870 con 0x7f1390103450 2024-04-25T18:59:13.055 INFO:teuthology.orchestra.run.smithi045.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T18:59:13.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 -- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13780a3d10 msgr2=0x7f13780a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:59:13.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13780a3d10 0x7f13780a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:13.057 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 -- 172.21.15.45:0/1342785644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1390103450 msgr2=0x7f139019b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1390103450 0x7f139019b8b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f138000e9a0 tx=0x7f138000ecb0 comp rx=0 tx=0).stop 2024-04-25T18:59:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 -- 172.21.15.45:0/1342785644 shutdown_connections 2024-04-25T18:59:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7f13780a3d10 0x7f13780a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1390108a50 0x7f139019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1390102a90 0x7f139019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 --2- 172.21.15.45:0/1342785644 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1390103450 0x7f139019b8b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 -- 172.21.15.45:0/1342785644 >> 172.21.15.45:0/1342785644 conn(0x7f1390079ba0 msgr2=0x7f139010df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 -- 172.21.15.45:0/1342785644 shutdown_connections 2024-04-25T18:59:13.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:13.056+0000 7f1398800700 1 -- 172.21.15.45:0/1342785644 wait complete. 2024-04-25T18:59:13.106 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:12 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:12.831+0000 7f9871640200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:59:13.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:13 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1342785644' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T18:59:13.363 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:13.042+0000 7f4ff5087200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:59:13.379 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:12 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:12.970+0000 7f93379ba200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T18:59:13.379 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:13 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:13.110+0000 7f93379ba200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T18:59:13.379 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:13 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1342785644' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T18:59:13.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:13 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1342785644' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T18:59:13.434 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:13.242+0000 7f9871640200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:59:13.434 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:13.335+0000 7f9871640200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:59:13.661 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:13 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:13.377+0000 7f93379ba200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T18:59:13.788 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:13.478+0000 7f4ff5087200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:59:13.788 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:13.578+0000 7f4ff5087200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:59:13.934 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:13.533+0000 7f9871640200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:59:13.963 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:13 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:13.660+0000 7f93379ba200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T18:59:14.012 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T18:59:14.013 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T18:59:14.013 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi045.front.sepia.ceph.com 2024-04-25T18:59:14.013 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T18:59:14.013 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T18:59:14.013 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T18:59:14.013 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T18:59:14.013 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T18:59:14.013 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T18:59:14.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:59:14.013 DEBUG:teuthology.orchestra.run.smithi045:> ip addr 2024-04-25T18:59:14.065 INFO:teuthology.orchestra.run.smithi045.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T18:59:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T18:59:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T18:59:14.065 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 ::1/128 scope host 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a6 brd ff:ff:ff:ff:ff:ff 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f0 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:6b:fd:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp7s0f1 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:be brd ff:ff:ff:ff:ff:ff 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f0 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: inet 172.21.15.45/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T18:59:14.066 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft 41900sec preferred_lft 41900sec 2024-04-25T18:59:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: inet6 fe80::ec4:7aff:febd:12be/64 scope link noprefixroute 2024-04-25T18:59:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: valid_lft forever preferred_lft forever 2024-04-25T18:59:14.067 INFO:teuthology.orchestra.run.smithi045.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T18:59:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: link/ether 0c:c4:7a:bd:12:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T18:59:14.067 INFO:teuthology.orchestra.run.smithi045.stdout: altname enp1s0f1 2024-04-25T18:59:14.067 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T18:59:14.067 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T18:59:14.068 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T18:59:14.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add name ceph-brx type bridge 2024-04-25T18:59:14.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr flush dev ceph-brx 2024-04-25T18:59:14.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set ceph-brx up 2024-04-25T18:59:14.068 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T18:59:14.068 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T18:59:14.113 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:13.785+0000 7f4ff5087200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:59:14.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:59:14.372 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:14.045+0000 7f9871640200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:59:14.372 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:14.133+0000 7f9871640200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:59:14.549 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:59:14.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:59:14.559 DEBUG:teuthology.orchestra.run.smithi045:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T18:59:14.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:14.327+0000 7f4ff5087200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:59:14.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:14.422+0000 7f4ff5087200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:59:14.626 INFO:teuthology.orchestra.run.smithi045.stdout:1 2024-04-25T18:59:14.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:59:14.629 DEBUG:teuthology.orchestra.run.smithi045:> route 2024-04-25T18:59:14.684 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:14.369+0000 7f9871640200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:59:14.684 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:14.499+0000 7f9871640200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:59:14.684 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:14.591+0000 7f9871640200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:59:14.686 INFO:teuthology.orchestra.run.smithi045.stdout:Kernel IP routing table 2024-04-25T18:59:14.686 INFO:teuthology.orchestra.run.smithi045.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T18:59:14.686 INFO:teuthology.orchestra.run.smithi045.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T18:59:14.686 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T18:59:14.686 INFO:teuthology.orchestra.run.smithi045.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T18:59:14.687 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T18:59:14.687 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T18:59:14.687 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T18:59:14.687 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T18:59:14.687 DEBUG:teuthology.orchestra.run.smithi045:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T18:59:14.687 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T18:59:14.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:14 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:14.496+0000 7f93379ba200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T18:59:14.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:59:15.006 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:14.705+0000 7f9871640200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:59:15.029 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:14.673+0000 7f4ff5087200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:59:15.029 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:14.811+0000 7f4ff5087200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:59:15.029 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:14.908+0000 7f4ff5087200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:59:15.029 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:15.027+0000 7f4ff5087200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:59:15.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:15 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:15.095+0000 7f93379ba200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T18:59:15.399 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:15 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:15.233+0000 7f93379ba200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T18:59:15.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:59:15.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:59:15.411 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T18:59:15.433 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:15.003+0000 7f9871640200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:59:15.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:59:15.502 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list-id 2024-04-25T18:59:15.550 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T18:59:15.550 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T18:59:15.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T18:59:15.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T18:59:15.550 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T18:59:15.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:15.339+0000 7f4ff5087200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:59:15.648 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:59:15.671 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:15 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:15.521+0000 7f93379ba200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T18:59:15.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:59:15.725 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T18:59:15.725 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T18:59:15.726 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T18:59:15.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T18:59:15.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T18:59:15.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T18:59:15.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T18:59:15.726 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T18:59:15.726 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T18:59:15.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:59:15.825 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:15.609+0000 7f9871640200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:59:15.825 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 18:59:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z[34872]: 2024-04-25T18:59:15.823+0000 7f9871640200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:59:15.933 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:15 smithi045 ceph-mon[27776]: Standby manager daemon z restarted 2024-04-25T18:59:15.933 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:15 smithi045 ceph-mon[27776]: Standby manager daemon z started 2024-04-25T18:59:15.933 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:15 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:59:15.933 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:15 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:59:15.933 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:15 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:59:15.933 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:15 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:59:15.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:59:16.007 DEBUG:teuthology.orchestra.run.smithi045:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T18:59:16.007 DEBUG:teuthology.orchestra.run.smithi045:> set -e 2024-04-25T18:59:16.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set brx.0 up 2024-04-25T18:59:16.007 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T18:59:16.007 DEBUG:teuthology.orchestra.run.smithi045:> ') 2024-04-25T18:59:16.093 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:59:16.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:59:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:15 smithi079 ceph-mon[33002]: Standby manager daemon z restarted 2024-04-25T18:59:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:15 smithi079 ceph-mon[33002]: Standby manager daemon z started 2024-04-25T18:59:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:15 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:59:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:15 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:59:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:15 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:59:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:15 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:59:16.192 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T18:59:16.192 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T18:59:16.197 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:15.973+0000 7f4ff5087200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:59:16.197 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:15 smithi062 ceph-mon[32844]: Standby manager daemon z restarted 2024-04-25T18:59:16.197 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:15 smithi062 ceph-mon[32844]: Standby manager daemon z started 2024-04-25T18:59:16.198 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:15 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T18:59:16.198 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:15 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:59:16.198 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:15 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T18:59:16.198 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:15 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.79:0/289145507' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:59:16.232 INFO:teuthology.orchestra.run.smithi045.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T18:59:16.232 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T18:59:16.233 DEBUG:teuthology.orchestra.run.smithi045:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T18:59:16.325 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T18:59:16.325 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T18:59:16.326 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/17f67f2f-4f96-4dd1-ba78-d9dd5e9b7635 2024-04-25T18:59:16.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:59:16.326 DEBUG:teuthology.orchestra.run.smithi045:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/17f67f2f-4f96-4dd1-ba78-d9dd5e9b7635 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T18:59:16.467 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T18:59:16.468 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T18:59:16.564 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:16.219+0000 7f93379ba200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T18:59:16.564 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:16.351+0000 7f93379ba200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T18:59:16.613 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 18:59:16 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y[34242]: 2024-04-25T18:59:16.196+0000 7f4ff5087200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:59:16.881 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:16.702+0000 7f93379ba200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T18:59:17.153 INFO:teuthology.orchestra.run.smithi045.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T18:59:17.154 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T18:59:17.154 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: resolved to: "172.21.15.45:3300,172.21.15.62:3300,172.21.15.79:3300" 2024-04-25T18:59:17.154 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: trying mount with new device syntax: 0@ea2833ba-0333-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/17f67f2f-4f96-4dd1-ba78-d9dd5e9b7635 2024-04-25T18:59:17.154 INFO:teuthology.orchestra.run.smithi045.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.45:3300/172.21.15.62:3300/172.21.15.79:3300" will pass to kernel 2024-04-25T18:59:17.157 INFO:teuthology.orchestra.run.smithi045.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T18:59:17.157 INFO:teuthology.orchestra.run.smithi045.stdout: You just mounted a file system that supports labels which does not 2024-04-25T18:59:17.157 INFO:teuthology.orchestra.run.smithi045.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T18:59:17.157 INFO:teuthology.orchestra.run.smithi045.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T18:59:17.157 INFO:teuthology.orchestra.run.smithi045.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T18:59:17.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-mon[27776]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-04-25T18:59:17.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-mon[27776]: Standby manager daemon y restarted 2024-04-25T18:59:17.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-mon[27776]: Standby manager daemon y started 2024-04-25T18:59:17.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:59:17.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:59:17.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:59:17.159 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-mon[27776]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:59:17.159 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:16 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:16.911+0000 7f93379ba200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T18:59:17.159 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:17 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:17.049+0000 7f93379ba200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T18:59:17.162 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T18:59:17.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T18:59:17.162 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T18:59:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:16 smithi079 ceph-mon[33002]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-04-25T18:59:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:16 smithi079 ceph-mon[33002]: Standby manager daemon y restarted 2024-04-25T18:59:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:16 smithi079 ceph-mon[33002]: Standby manager daemon y started 2024-04-25T18:59:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:16 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:59:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:16 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:59:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:16 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:59:17.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:16 smithi079 ceph-mon[33002]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:59:17.241 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T18:59:17.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T18:59:17.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:59:17.354 INFO:teuthology.orchestra.run.smithi045.stdout:client25075 2024-04-25T18:59:17.354 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T18:59:17.354 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T18:59:17.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:16 smithi062 ceph-mon[32844]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-04-25T18:59:17.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:16 smithi062 ceph-mon[32844]: Standby manager daemon y restarted 2024-04-25T18:59:17.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:16 smithi062 ceph-mon[32844]: Standby manager daemon y started 2024-04-25T18:59:17.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:16 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T18:59:17.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:16 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T18:59:17.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:16 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T18:59:17.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:16 smithi062 ceph-mon[32844]: from='mgr.? 172.21.15.62:0/1144272590' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T18:59:17.401 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T18:59:17.463 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:17 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:17.212+0000 7f93379ba200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T18:59:17.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T18:59:17.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:59:17.576 INFO:teuthology.orchestra.run.smithi045.stdout:client25075 2024-04-25T18:59:17.577 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T18:59:17.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T18:59:17.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:59:17.749 INFO:teuthology.orchestra.run.smithi045.stdout:client25075 2024-04-25T18:59:17.750 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ea2833ba-0333-11ef-bc93-c7b262605968.client25075/status') 2024-04-25T18:59:17.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T18:59:17.850 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:17 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:17.604+0000 7f93379ba200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T18:59:17.861 INFO:teuthology.orchestra.run.smithi045.stdout:instance: client.25075 (3)172.21.15.45:0/3258206366 2024-04-25T18:59:17.861 INFO:teuthology.orchestra.run.smithi045.stdout:blocklisted: false 2024-04-25T18:59:17.861 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records in 2024-04-25T18:59:17.862 INFO:teuthology.orchestra.run.smithi045.stderr:0+1 records out 2024-04-25T18:59:17.862 INFO:teuthology.orchestra.run.smithi045.stderr:71 bytes copied, 6.2052e-05 s, 1.1 MB/s 2024-04-25T18:59:17.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T18:59:17.877 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T18:59:17.887 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T18:59:17.887 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T18:59:17.887 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T18:59:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.059+0000 7fb80de00640 1 -- 172.21.15.45:0/1070230583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb808109cd0 msgr2=0x7fb80810c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.059+0000 7fb80de00640 1 --2- 172.21.15.45:0/1070230583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb808109cd0 0x7fb80810c160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb800009be0 tx=0x7fb80002a6c0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.059+0000 7fb80de00640 1 -- 172.21.15.45:0/1070230583 shutdown_connections 2024-04-25T18:59:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.059+0000 7fb80de00640 1 --2- 172.21.15.45:0/1070230583 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb808109cd0 0x7fb80810c160 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.059+0000 7fb80de00640 1 --2- 172.21.15.45:0/1070230583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8080749b0 0x7fb808074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.059+0000 7fb80de00640 1 --2- 172.21.15.45:0/1070230583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb808075f90 0x7fb8080743e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.059+0000 7fb80de00640 1 -- 172.21.15.45:0/1070230583 >> 172.21.15.45:0/1070230583 conn(0x7fb8080fd380 msgr2=0x7fb8080ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.060+0000 7fb80de00640 1 -- 172.21.15.45:0/1070230583 shutdown_connections 2024-04-25T18:59:18.061 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.060+0000 7fb80de00640 1 -- 172.21.15.45:0/1070230583 wait complete. 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.060+0000 7fb80de00640 1 Processor -- start 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb80de00640 1 -- start start 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb80de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8080749b0 0x7fb80819a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb80de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808075f90 0x7fb80819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb80de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb808109cd0 0x7fb8081a1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb80de00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb80810fb40 con 0x7fb808075f90 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb80de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb80810f9c0 con 0x7fb8080749b0 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb80de00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb80810fcc0 con 0x7fb808109cd0 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb807e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808075f90 0x7fb80819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb80ca00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8080749b0 0x7fb80819a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb807e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808075f90 0x7fb80819a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36358/0 (socket says 172.21.15.45:36358) 2024-04-25T18:59:18.063 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb807e00640 1 -- 172.21.15.45:0/2628256460 learned_addr learned my addr 172.21.15.45:0/2628256460 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.061+0000 7fb80d400640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb808109cd0 0x7fb8081a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb807e00640 1 -- 172.21.15.45:0/2628256460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb808109cd0 msgr2=0x7fb8081a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb807e00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb808109cd0 0x7fb8081a1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb807e00640 1 -- 172.21.15.45:0/2628256460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8080749b0 msgr2=0x7fb80819a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb807e00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8080749b0 0x7fb80819a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb807e00640 1 -- 172.21.15.45:0/2628256460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8081a1d90 con 0x7fb808075f90 2024-04-25T18:59:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb80ca00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8080749b0 0x7fb80819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:18.064 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb807e00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808075f90 0x7fb80819a700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400e9c0 tx=0x7fb7f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:18.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb805600640 1 -- 172.21.15.45:0/2628256460 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400ccc0 con 0x7fb808075f90 2024-04-25T18:59:18.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb80de00640 1 -- 172.21.15.45:0/2628256460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8081a2080 con 0x7fb808075f90 2024-04-25T18:59:18.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb805600640 1 -- 172.21.15.45:0/2628256460 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f40054d0 con 0x7fb808075f90 2024-04-25T18:59:18.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.062+0000 7fb805600640 1 -- 172.21.15.45:0/2628256460 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4010430 con 0x7fb808075f90 2024-04-25T18:59:18.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.063+0000 7fb80de00640 1 -- 172.21.15.45:0/2628256460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8081a2620 con 0x7fb808075f90 2024-04-25T18:59:18.065 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.064+0000 7fb80de00640 1 -- 172.21.15.45:0/2628256460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb808108a00 con 0x7fb808075f90 2024-04-25T18:59:18.067 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.066+0000 7fb805600640 1 -- 172.21.15.45:0/2628256460 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 21) ==== 138663+0+0 (secure 0 0 0) 0x7fb7f4004f80 con 0x7fb808075f90 2024-04-25T18:59:18.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.068+0000 7fb805600640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb7f00a3be0 0x7fb7f00a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:18.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.068+0000 7fb805600640 1 -- 172.21.15.45:0/2628256460 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8269+0+0 (secure 0 0 0) 0x7fb7f40d0880 con 0x7fb808075f90 2024-04-25T18:59:18.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.068+0000 7fb80ca00640 1 -- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb7f00a3be0 msgr2=0x7fb7f00a60a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/943541413 2024-04-25T18:59:18.069 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.068+0000 7fb80ca00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb7f00a3be0 0x7fb7f00a60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T18:59:18.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.077+0000 7fb805600640 1 -- 172.21.15.45:0/2628256460 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7f409d240 con 0x7fb808075f90 2024-04-25T18:59:18.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:17 smithi079 ceph-mon[33002]: mgrmap e21: x(active, since 12m), standbys: z, y 2024-04-25T18:59:18.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:17 smithi045 ceph-mon[27776]: mgrmap e21: x(active, since 12m), standbys: z, y 2024-04-25T18:59:18.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.269+0000 7fb80ca00640 1 -- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb7f00a3be0 msgr2=0x7fb7f00a60a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/943541413 2024-04-25T18:59:18.270 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.269+0000 7fb80ca00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb7f00a3be0 0x7fb7f00a60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T18:59:18.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.320+0000 7fb80de00640 1 -- 172.21.15.45:0/2628256460 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fb808062b50 con 0x7fb808075f90 2024-04-25T18:59:18.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:17 smithi062 ceph-mon[32844]: mgrmap e21: x(active, since 12m), standbys: z, y 2024-04-25T18:59:18.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.670+0000 7fb80ca00640 1 -- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb7f00a3be0 msgr2=0x7fb7f00a60a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.45:6800/943541413 2024-04-25T18:59:18.671 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.670+0000 7fb80ca00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb7f00a3be0 0x7fb7f00a60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T18:59:18.713 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:18.441+0000 7f93379ba200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T18:59:18.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.764+0000 7fb805600640 1 -- 172.21.15.45:0/2628256460 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7fb7f40bd070 con 0x7fb808075f90 2024-04-25T18:59:18.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.764+0000 7fb805600640 1 -- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb7f00a3be0 msgr2=0x7fb7f00a60a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T18:59:18.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.764+0000 7fb805600640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb7f00a3be0 0x7fb7f00a60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.924 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.923+0000 7fb805600640 1 -- 172.21.15.45:0/2628256460 <== mon.0 v2:172.21.15.45:3300/0 8 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7fb7f408dc80 con 0x7fb808075f90 2024-04-25T18:59:18.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.927+0000 7fb80de00640 1 -- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808075f90 msgr2=0x7fb80819a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.927+0000 7fb80de00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808075f90 0x7fb80819a700 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400e9c0 tx=0x7fb7f400ee80 comp rx=0 tx=0).stop 2024-04-25T18:59:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.928+0000 7fb80de00640 1 -- 172.21.15.45:0/2628256460 shutdown_connections 2024-04-25T18:59:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.928+0000 7fb80de00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:6800/943541413,v1:172.21.15.45:6801/943541413] conn(0x7fb7f00a3be0 0x7fb7f00a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.928+0000 7fb80de00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb808109cd0 0x7fb8081a1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.928+0000 7fb80de00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb808075f90 0x7fb80819a700 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.928+0000 7fb80de00640 1 --2- 172.21.15.45:0/2628256460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8080749b0 0x7fb80819a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.928+0000 7fb80de00640 1 -- 172.21.15.45:0/2628256460 >> 172.21.15.45:0/2628256460 conn(0x7fb8080fd380 msgr2=0x7fb808105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.928+0000 7fb80de00640 1 -- 172.21.15.45:0/2628256460 shutdown_connections 2024-04-25T18:59:18.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:18.928+0000 7fb80de00640 1 -- 172.21.15.45:0/2628256460 wait complete. 2024-04-25T18:59:18.948 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T18:59:19.016 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:18.742+0000 7f93379ba200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T18:59:19.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2628256460' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T18:59:19.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: Active manager daemon x restarted 2024-04-25T18:59:19.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: Activating manager daemon x 2024-04-25T18:59:19.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T18:59:19.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: mgrmap e22: x(active, starting, since 0.022787s), standbys: z, y 2024-04-25T18:59:19.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:59:19.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:59:19.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:59:19.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:59:19.017 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:59:19.018 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:59:19.019 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: Manager daemon x is now available 2024-04-25T18:59:19.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:59:19.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:59:19.020 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:18 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:59:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.172+0000 7f9486600640 1 -- 172.21.15.45:0/919654140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9480101930 msgr2=0x7f9480101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.172+0000 7f9486600640 1 --2- 172.21.15.45:0/919654140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9480101930 0x7f9480101db0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f946c00d910 tx=0x7f946c02a160 comp rx=0 tx=0).stop 2024-04-25T18:59:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.173+0000 7f9486600640 1 -- 172.21.15.45:0/919654140 shutdown_connections 2024-04-25T18:59:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.173+0000 7f9486600640 1 --2- 172.21.15.45:0/919654140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9480106630 0x7f9480108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.173+0000 7f9486600640 1 --2- 172.21.15.45:0/919654140 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9480101930 0x7f9480101db0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.173+0000 7f9486600640 1 --2- 172.21.15.45:0/919654140 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f94800689c0 0x7f94801013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.173+0000 7f9486600640 1 -- 172.21.15.45:0/919654140 >> 172.21.15.45:0/919654140 conn(0x7f94800fb770 msgr2=0x7f94800fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:19.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.173+0000 7f9486600640 1 -- 172.21.15.45:0/919654140 shutdown_connections 2024-04-25T18:59:19.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.173+0000 7f9486600640 1 -- 172.21.15.45:0/919654140 wait complete. 2024-04-25T18:59:19.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.174+0000 7f9486600640 1 Processor -- start 2024-04-25T18:59:19.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.174+0000 7f9486600640 1 -- start start 2024-04-25T18:59:19.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f9486600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800689c0 0x7f9480196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:19.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f9486600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9480101930 0x7f9480196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:19.176 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f9486600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9480106630 0x7f948019d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f9486600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f948010b960 con 0x7f94800689c0 2024-04-25T18:59:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f9486600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f948010b7e0 con 0x7f9480106630 2024-04-25T18:59:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f9486600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f948010bae0 con 0x7f9480101930 2024-04-25T18:59:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f947f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800689c0 0x7f9480196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f947ea00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9480101930 0x7f9480196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f947ea00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9480101930 0x7f9480196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:59568/0 (socket says 172.21.15.45:59568) 2024-04-25T18:59:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f947f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800689c0 0x7f9480196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36488/0 (socket says 172.21.15.45:36488) 2024-04-25T18:59:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f947ea00640 1 -- 172.21.15.45:0/2687544764 learned_addr learned my addr 172.21.15.45:0/2687544764 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:19.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.175+0000 7f947fe00640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9480106630 0x7f948019d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.176+0000 7f947ea00640 1 -- 172.21.15.45:0/2687544764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9480106630 msgr2=0x7f948019d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.176+0000 7f947ea00640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9480106630 0x7f948019d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.176+0000 7f947ea00640 1 -- 172.21.15.45:0/2687544764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800689c0 msgr2=0x7f9480196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.176+0000 7f947ea00640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800689c0 0x7f9480196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.176+0000 7f947ea00640 1 -- 172.21.15.45:0/2687544764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f948019dc40 con 0x7f9480101930 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.176+0000 7f947f400640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800689c0 0x7f9480196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.176+0000 7f947ea00640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9480101930 0x7f9480196560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f946c009230 tx=0x7f946c00c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.177+0000 7f9477e00640 1 -- 172.21.15.45:0/2687544764 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c03f020 con 0x7f9480101930 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.177+0000 7f9486600640 1 -- 172.21.15.45:0/2687544764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f948019ded0 con 0x7f9480101930 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.177+0000 7f9486600640 1 -- 172.21.15.45:0/2687544764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f948019e330 con 0x7f9480101930 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.177+0000 7f9477e00640 1 -- 172.21.15.45:0/2687544764 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f946c02ad70 con 0x7f9480101930 2024-04-25T18:59:19.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.177+0000 7f9477e00640 1 -- 172.21.15.45:0/2687544764 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c02c380 con 0x7f9480101930 2024-04-25T18:59:19.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.178+0000 7f9486600640 1 -- 172.21.15.45:0/2687544764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94600052c0 con 0x7f9480101930 2024-04-25T18:59:19.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.181+0000 7f9477e00640 1 -- 172.21.15.45:0/2687544764 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f946c042020 con 0x7f9480101930 2024-04-25T18:59:19.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.183+0000 7f9477e00640 1 -- 172.21.15.45:0/2687544764 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f946c0efcd0 con 0x7f9480101930 2024-04-25T18:59:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2628256460' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T18:59:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: Active manager daemon x restarted 2024-04-25T18:59:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: Activating manager daemon x 2024-04-25T18:59:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T18:59:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: mgrmap e22: x(active, starting, since 0.022787s), standbys: z, y 2024-04-25T18:59:19.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:59:19.185 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:59:19.186 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:59:19.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:59:19.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:59:19.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: Manager daemon x is now available 2024-04-25T18:59:19.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:59:19.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:59:19.187 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:18 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:59:19.187 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.184+0000 7f9477e00640 1 -- 172.21.15.45:0/2687544764 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f946c0bc930 con 0x7f9480101930 2024-04-25T18:59:19.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2628256460' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T18:59:19.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: Active manager daemon x restarted 2024-04-25T18:59:19.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: Activating manager daemon x 2024-04-25T18:59:19.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T18:59:19.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: mgrmap e22: x(active, starting, since 0.022787s), standbys: z, y 2024-04-25T18:59:19.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T18:59:19.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T18:59:19.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T18:59:19.365 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: Manager daemon x is now available 2024-04-25T18:59:19.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:59:19.367 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:59:19.367 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:18 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T18:59:19.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.119+0000 7f928ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.119+0000 7f928ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.119+0000 7f928ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.119+0000 7f928ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.119+0000 7f928ca00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.121+0000 7f928a200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.121+0000 7f928a200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.121+0000 7f928a200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.121+0000 7f928a200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.121+0000 7f928a200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.777+0000 7f9477e00640 1 -- 172.21.15.45:0/2687544764 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f946c03a070 con 0x7f9480101930 2024-04-25T18:59:19.779 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.778+0000 7f9477e00640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f94640aaa20 0x7f94640ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:19.780 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.779+0000 7f9477e00640 1 -- 172.21.15.45:0/2687544764 --> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f946c005430 con 0x7f94640aaa20 2024-04-25T18:59:19.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.788+0000 7f947f400640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f94640aaa20 0x7f94640ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:19.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.804+0000 7f947f400640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f94640aaa20 0x7f94640ace10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9480101250 tx=0x7f9470011040 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:19.832 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.832+0000 7f9477e00640 1 -- 172.21.15.45:0/2687544764 <== mgr.25049 v2:172.21.15.45:6800/1345298339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f946c005430 con 0x7f94640aaa20 2024-04-25T18:59:19.833 INFO:teuthology.orchestra.run.smithi045.stdout:{} 2024-04-25T18:59:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 -- 172.21.15.45:0/2687544764 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f94640aaa20 msgr2=0x7f94640ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f94640aaa20 0x7f94640ace10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9480101250 tx=0x7f9470011040 comp rx=0 tx=0).stop 2024-04-25T18:59:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 -- 172.21.15.45:0/2687544764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9480101930 msgr2=0x7f9480196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9480101930 0x7f9480196560 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f946c009230 tx=0x7f946c00c4e0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 -- 172.21.15.45:0/2687544764 shutdown_connections 2024-04-25T18:59:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f94640aaa20 0x7f94640ace10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9480106630 0x7f948019d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9480101930 0x7f9480196560 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 --2- 172.21.15.45:0/2687544764 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f94800689c0 0x7f9480196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 -- 172.21.15.45:0/2687544764 >> 172.21.15.45:0/2687544764 conn(0x7f94800fb770 msgr2=0x7f9480102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 -- 172.21.15.45:0/2687544764 shutdown_connections 2024-04-25T18:59:19.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:19.835+0000 7f9486600640 1 -- 172.21.15.45:0/2687544764 wait complete. 2024-04-25T18:59:19.859 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T18:59:19.870 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T18:59:19.878 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T18:59:19.879 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T18:59:19.879 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T18:59:20.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.069+0000 7f0bab200640 1 -- 172.21.15.45:0/3425548640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bac075f10 msgr2=0x7f0bac074360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:20.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.069+0000 7f0bab200640 1 --2- 172.21.15.45:0/3425548640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bac075f10 0x7f0bac074360 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c009900 tx=0x7f0b9c02a480 comp rx=0 tx=0).stop 2024-04-25T18:59:20.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.069+0000 7f0bab200640 1 -- 172.21.15.45:0/3425548640 shutdown_connections 2024-04-25T18:59:20.070 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.069+0000 7f0bab200640 1 --2- 172.21.15.45:0/3425548640 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0bac109ca0 0x7f0bac10c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.069+0000 7f0bab200640 1 --2- 172.21.15.45:0/3425548640 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac074930 0x7f0bac074d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.069+0000 7f0bab200640 1 --2- 172.21.15.45:0/3425548640 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bac075f10 0x7f0bac074360 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.069+0000 7f0bab200640 1 -- 172.21.15.45:0/3425548640 >> 172.21.15.45:0/3425548640 conn(0x7f0bac0fd360 msgr2=0x7f0bac0ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.069+0000 7f0bab200640 1 -- 172.21.15.45:0/3425548640 shutdown_connections 2024-04-25T18:59:20.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.069+0000 7f0bab200640 1 -- 172.21.15.45:0/3425548640 wait complete. 2024-04-25T18:59:20.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.070+0000 7f0bab200640 1 Processor -- start 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.070+0000 7f0bab200640 1 -- start start 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.070+0000 7f0bab200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac074930 0x7f0bac19a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.070+0000 7f0bab200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0bac075f10 0x7f0bac19a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.070+0000 7f0bab200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bac109ca0 0x7f0bac1a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.070+0000 7f0bab200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac10fb50 con 0x7f0bac109ca0 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.070+0000 7f0bab200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac10f9d0 con 0x7f0bac074930 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.070+0000 7f0bab200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac10fcd0 con 0x7f0bac075f10 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0baa800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bac109ca0 0x7f0bac1a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0ba9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac074930 0x7f0bac19a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0ba9400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0bac075f10 0x7f0bac19a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0baa800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bac109ca0 0x7f0bac1a1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36546/0 (socket says 172.21.15.45:36546) 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0baa800640 1 -- 172.21.15.45:0/4191289046 learned_addr learned my addr 172.21.15.45:0/4191289046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0ba9400640 1 -- 172.21.15.45:0/4191289046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac074930 msgr2=0x7f0bac19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:20.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0ba9400640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac074930 0x7f0bac19a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0ba9400640 1 -- 172.21.15.45:0/4191289046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bac109ca0 msgr2=0x7f0bac1a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0ba9400640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bac109ca0 0x7f0bac1a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0ba9400640 1 -- 172.21.15.45:0/4191289046 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bac1a1e70 con 0x7f0bac075f10 2024-04-25T18:59:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.071+0000 7f0ba9400640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0bac075f10 0x7f0bac19a790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800e9f0 tx=0x7f0b9800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.072+0000 7f0ba2a00640 1 -- 172.21.15.45:0/4191289046 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9800cd80 con 0x7f0bac075f10 2024-04-25T18:59:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.072+0000 7f0bab200640 1 -- 172.21.15.45:0/4191289046 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bac1a2160 con 0x7f0bac075f10 2024-04-25T18:59:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.072+0000 7f0ba2a00640 1 -- 172.21.15.45:0/4191289046 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b980035f0 con 0x7f0bac075f10 2024-04-25T18:59:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.072+0000 7f0ba2a00640 1 -- 172.21.15.45:0/4191289046 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b98010690 con 0x7f0bac075f10 2024-04-25T18:59:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.072+0000 7f0bab200640 1 -- 172.21.15.45:0/4191289046 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bac1a26b0 con 0x7f0bac075f10 2024-04-25T18:59:20.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.073+0000 7f0bab200640 1 -- 172.21.15.45:0/4191289046 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bac108a60 con 0x7f0bac075f10 2024-04-25T18:59:20.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.075+0000 7f0ba2a00640 1 -- 172.21.15.45:0/4191289046 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f0b980037a0 con 0x7f0bac075f10 2024-04-25T18:59:20.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.077+0000 7f0ba2a00640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0b900a3990 0x7f0b900a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:20.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.077+0000 7f0ba2a00640 1 -- 172.21.15.45:0/4191289046 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f0b980d0410 con 0x7f0bac075f10 2024-04-25T18:59:20.078 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.077+0000 7f0ba9e00640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0b900a3990 0x7f0b900a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:20.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.078+0000 7f0ba9e00640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0b900a3990 0x7f0b900a5e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c031500 tx=0x7f0b9c005a20 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:20.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.080+0000 7f0ba2a00640 1 -- 172.21.15.45:0/4191289046 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b9809cf80 con 0x7f0bac075f10 2024-04-25T18:59:20.177 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:19 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:59:20.177 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:19 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:59:20.177 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:19 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2628256460' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T18:59:20.177 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:19 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:59:20.177 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:19 smithi062 ceph-mon[32844]: mgrmap e23: x(active, since 1.03912s), standbys: z, y 2024-04-25T18:59:20.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:19 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:59:20.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:19 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:59:20.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:19 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2628256460' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T18:59:20.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:19 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:59:20.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:19 smithi079 ceph-mon[33002]: mgrmap e23: x(active, since 1.03912s), standbys: z, y 2024-04-25T18:59:20.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:59:20.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T18:59:20.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2628256460' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T18:59:20.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T18:59:20.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-mon[27776]: mgrmap e23: x(active, since 1.03912s), standbys: z, y 2024-04-25T18:59:20.214 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.822+0000 7f9286600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:20.214 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.822+0000 7f9286600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:20.214 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.822+0000 7f9286600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:20.214 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.823+0000 7f9286600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:20.214 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 18:59:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T18:59:19.823+0000 7f9286600700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T18:59:20.316 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.315+0000 7f0bab200640 1 -- 172.21.15.45:0/4191289046 --> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f0bac062390 con 0x7f0b900a3990 2024-04-25T18:59:20.423 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.422+0000 7f0ba2a00640 1 -- 172.21.15.45:0/4191289046 <== mgr.25049 v2:172.21.15.45:6800/1345298339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f0bac062390 con 0x7f0b900a3990 2024-04-25T18:59:20.423 INFO:teuthology.orchestra.run.smithi045.stdout:Schedule set for path / 2024-04-25T18:59:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 -- 172.21.15.45:0/4191289046 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0b900a3990 msgr2=0x7f0b900a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0b900a3990 0x7f0b900a5e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c031500 tx=0x7f0b9c005a20 comp rx=0 tx=0).stop 2024-04-25T18:59:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 -- 172.21.15.45:0/4191289046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0bac075f10 msgr2=0x7f0bac19a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0bac075f10 0x7f0bac19a790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0b9800e9f0 tx=0x7f0b9800eeb0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 -- 172.21.15.45:0/4191289046 shutdown_connections 2024-04-25T18:59:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0b900a3990 0x7f0b900a5e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0bac109ca0 0x7f0bac1a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.426 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0bac075f10 0x7f0bac19a790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 --2- 172.21.15.45:0/4191289046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0bac074930 0x7f0bac19a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 -- 172.21.15.45:0/4191289046 >> 172.21.15.45:0/4191289046 conn(0x7f0bac0fd360 msgr2=0x7f0bac105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:20.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 -- 172.21.15.45:0/4191289046 shutdown_connections 2024-04-25T18:59:20.427 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.425+0000 7f0bab200640 1 -- 172.21.15.45:0/4191289046 wait complete. 2024-04-25T18:59:20.443 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T18:59:20.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.636+0000 7f6af2e00640 1 -- 172.21.15.45:0/1042653777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aec1041e0 msgr2=0x7f6aec104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:20.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.636+0000 7f6af2e00640 1 --2- 172.21.15.45:0/1042653777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aec1041e0 0x7f6aec104600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6adc009860 tx=0x7f6adc02a340 comp rx=0 tx=0).stop 2024-04-25T18:59:20.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.636+0000 7f6af2e00640 1 -- 172.21.15.45:0/1042653777 shutdown_connections 2024-04-25T18:59:20.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.636+0000 7f6af2e00640 1 --2- 172.21.15.45:0/1042653777 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec105d90 0x7f6aec10c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.636+0000 7f6af2e00640 1 --2- 172.21.15.45:0/1042653777 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aec1053d0 0x7f6aec105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.636+0000 7f6af2e00640 1 --2- 172.21.15.45:0/1042653777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aec1041e0 0x7f6aec104600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.636+0000 7f6af2e00640 1 -- 172.21.15.45:0/1042653777 >> 172.21.15.45:0/1042653777 conn(0x7f6aec0ff770 msgr2=0x7f6aec101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:20.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.636+0000 7f6af2e00640 1 -- 172.21.15.45:0/1042653777 shutdown_connections 2024-04-25T18:59:20.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.636+0000 7f6af2e00640 1 -- 172.21.15.45:0/1042653777 wait complete. 2024-04-25T18:59:20.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.637+0000 7f6af2e00640 1 Processor -- start 2024-04-25T18:59:20.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.637+0000 7f6af2e00640 1 -- start start 2024-04-25T18:59:20.639 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6af2e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec1041e0 0x7f6aec19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6af2e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aec1053d0 0x7f6aec19a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6af2e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aec105d90 0x7f6aec1a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6af2e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10fd20 con 0x7f6aec1053d0 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6af2e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10fba0 con 0x7f6aec1041e0 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6af2e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6aec10fea0 con 0x7f6aec105d90 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6af0e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aec105d90 0x7f6aec1a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6aeb400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aec1053d0 0x7f6aec19a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec1041e0 0x7f6aec19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6af0e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aec105d90 0x7f6aec1a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:59630/0 (socket says 172.21.15.45:59630) 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6aebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec1041e0 0x7f6aec19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36056/0 (socket says 172.21.15.45:36056) 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.638+0000 7f6af0e00640 1 -- 172.21.15.45:0/3332680100 learned_addr learned my addr 172.21.15.45:0/3332680100 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6aebe00640 1 -- 172.21.15.45:0/3332680100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aec105d90 msgr2=0x7f6aec1a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6aebe00640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aec105d90 0x7f6aec1a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6aebe00640 1 -- 172.21.15.45:0/3332680100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aec1053d0 msgr2=0x7f6aec19a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:20.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6aebe00640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aec1053d0 0x7f6aec19a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:20.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6aebe00640 1 -- 172.21.15.45:0/3332680100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aec1a2020 con 0x7f6aec1041e0 2024-04-25T18:59:20.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6aebe00640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec1041e0 0x7f6aec19a400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6adc02a860 tx=0x7f6adc032e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:20.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6ae8c00640 1 -- 172.21.15.45:0/3332680100 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc037040 con 0x7f6aec1041e0 2024-04-25T18:59:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6ae8c00640 1 -- 172.21.15.45:0/3332680100 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6adc030c50 con 0x7f6aec1041e0 2024-04-25T18:59:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6af2e00640 1 -- 172.21.15.45:0/3332680100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aec1a22b0 con 0x7f6aec1041e0 2024-04-25T18:59:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6ae8c00640 1 -- 172.21.15.45:0/3332680100 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6adc003850 con 0x7f6aec1041e0 2024-04-25T18:59:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.639+0000 7f6af2e00640 1 -- 172.21.15.45:0/3332680100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aec1a2790 con 0x7f6aec1041e0 2024-04-25T18:59:20.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.640+0000 7f6af2e00640 1 -- 172.21.15.45:0/3332680100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aec10b5d0 con 0x7f6aec1041e0 2024-04-25T18:59:20.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.642+0000 7f6ae8c00640 1 -- 172.21.15.45:0/3332680100 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f6adc043020 con 0x7f6aec1041e0 2024-04-25T18:59:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.644+0000 7f6ae8c00640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6ad00a3a10 0x7f6ad00a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.644+0000 7f6ae8c00640 1 -- 172.21.15.45:0/3332680100 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f6adc0ef340 con 0x7f6aec1041e0 2024-04-25T18:59:20.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.644+0000 7f6aeb400640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6ad00a3a10 0x7f6ad00a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:20.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.645+0000 7f6aeb400640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6ad00a3a10 0x7f6ad00a5ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6aec19b910 tx=0x7f6ad800b390 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:20.649 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.648+0000 7f6ae8c00640 1 -- 172.21.15.45:0/3332680100 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6adc04a060 con 0x7f6aec1041e0 2024-04-25T18:59:20.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.900+0000 7f6af2e00640 1 -- 172.21.15.45:0/3332680100 --> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f6aec0624f0 con 0x7f6ad00a3a10 2024-04-25T18:59:20.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:20.939+0000 7f6ae8c00640 1 -- 172.21.15.45:0/3332680100 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7f6adc0b7e20 con 0x7f6aec1041e0 2024-04-25T18:59:21.008 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.007+0000 7f6ae8c00640 1 -- 172.21.15.45:0/3332680100 <== mgr.25049 v2:172.21.15.45:6800/1345298339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f6aec0624f0 con 0x7f6ad00a3a10 2024-04-25T18:59:21.008 INFO:teuthology.orchestra.run.smithi045.stdout:Retention added to path / 2024-04-25T18:59:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 -- 172.21.15.45:0/3332680100 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6ad00a3a10 msgr2=0x7f6ad00a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6ad00a3a10 0x7f6ad00a5ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6aec19b910 tx=0x7f6ad800b390 comp rx=0 tx=0).stop 2024-04-25T18:59:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 -- 172.21.15.45:0/3332680100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec1041e0 msgr2=0x7f6aec19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec1041e0 0x7f6aec19a400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6adc02a860 tx=0x7f6adc032e90 comp rx=0 tx=0).stop 2024-04-25T18:59:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 -- 172.21.15.45:0/3332680100 shutdown_connections 2024-04-25T18:59:21.011 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6ad00a3a10 0x7f6ad00a5ed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6aec105d90 0x7f6aec1a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6aec1053d0 0x7f6aec19a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 --2- 172.21.15.45:0/3332680100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6aec1041e0 0x7f6aec19a400 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 -- 172.21.15.45:0/3332680100 >> 172.21.15.45:0/3332680100 conn(0x7f6aec0ff770 msgr2=0x7f6aec1087a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 -- 172.21.15.45:0/3332680100 shutdown_connections 2024-04-25T18:59:21.012 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.010+0000 7f6af2e00640 1 -- 172.21.15.45:0/3332680100 wait complete. 2024-04-25T18:59:21.028 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:20 smithi045 ceph-mon[27776]: [25/Apr/2024:18:59:19] ENGINE Bus STARTING 2024-04-25T18:59:21.028 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:20 smithi045 ceph-mon[27776]: [25/Apr/2024:18:59:19] ENGINE Serving on http://172.21.15.45:8765 2024-04-25T18:59:21.028 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:20 smithi045 ceph-mon[27776]: [25/Apr/2024:18:59:19] ENGINE Serving on https://172.21.15.45:7150 2024-04-25T18:59:21.028 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:20 smithi045 ceph-mon[27776]: [25/Apr/2024:18:59:19] ENGINE Bus STARTED 2024-04-25T18:59:21.028 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:20 smithi045 ceph-mon[27776]: from='client.25120 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:21.028 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:20 smithi045 ceph-mon[27776]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:21.030 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T18:59:21.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:20 smithi079 ceph-mon[33002]: [25/Apr/2024:18:59:19] ENGINE Bus STARTING 2024-04-25T18:59:21.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:20 smithi079 ceph-mon[33002]: [25/Apr/2024:18:59:19] ENGINE Serving on http://172.21.15.45:8765 2024-04-25T18:59:21.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:20 smithi079 ceph-mon[33002]: [25/Apr/2024:18:59:19] ENGINE Serving on https://172.21.15.45:7150 2024-04-25T18:59:21.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:20 smithi079 ceph-mon[33002]: [25/Apr/2024:18:59:19] ENGINE Bus STARTED 2024-04-25T18:59:21.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:20 smithi079 ceph-mon[33002]: from='client.25120 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:21.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:20 smithi079 ceph-mon[33002]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.226+0000 7f8ad5200640 1 -- 172.21.15.45:0/3090849576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad01041e0 msgr2=0x7f8ad0104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.226+0000 7f8ad5200640 1 --2- 172.21.15.45:0/3090849576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad01041e0 0x7f8ad0104600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009900 tx=0x7f8ac002a410 comp rx=0 tx=0).stop 2024-04-25T18:59:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.226+0000 7f8ad5200640 1 -- 172.21.15.45:0/3090849576 shutdown_connections 2024-04-25T18:59:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.226+0000 7f8ad5200640 1 --2- 172.21.15.45:0/3090849576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad0105d90 0x7f8ad010c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.226+0000 7f8ad5200640 1 --2- 172.21.15.45:0/3090849576 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ad01053d0 0x7f8ad0105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.226+0000 7f8ad5200640 1 --2- 172.21.15.45:0/3090849576 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad01041e0 0x7f8ad0104600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.226+0000 7f8ad5200640 1 -- 172.21.15.45:0/3090849576 >> 172.21.15.45:0/3090849576 conn(0x7f8ad00ff770 msgr2=0x7f8ad0101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.226+0000 7f8ad5200640 1 -- 172.21.15.45:0/3090849576 shutdown_connections 2024-04-25T18:59:21.228 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.226+0000 7f8ad5200640 1 -- 172.21.15.45:0/3090849576 wait complete. 2024-04-25T18:59:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.227+0000 7f8ad5200640 1 Processor -- start 2024-04-25T18:59:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.227+0000 7f8ad5200640 1 -- start start 2024-04-25T18:59:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.227+0000 7f8ad5200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad01041e0 0x7f8ad019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:21.229 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ad5200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ad01053d0 0x7f8ad019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ad5200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad0105d90 0x7f8ad01a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ad5200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010fd70 con 0x7f8ad01041e0 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ad5200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010fbf0 con 0x7f8ad0105d90 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ad5200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad010fef0 con 0x7f8ad01053d0 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ace800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad0105d90 0x7f8ad01a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ac5400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ad01053d0 0x7f8ad019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ace800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad0105d90 0x7f8ad01a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36080/0 (socket says 172.21.15.45:36080) 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8acde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad01041e0 0x7f8ad019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ace800640 1 -- 172.21.15.45:0/3554007 learned_addr learned my addr 172.21.15.45:0/3554007 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ac5400640 1 -- 172.21.15.45:0/3554007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad0105d90 msgr2=0x7f8ad01a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ac5400640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad0105d90 0x7f8ad01a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ac5400640 1 -- 172.21.15.45:0/3554007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad01041e0 msgr2=0x7f8ad019a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ac5400640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad01041e0 0x7f8ad019a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ac5400640 1 -- 172.21.15.45:0/3554007 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad01a2020 con 0x7f8ad01053d0 2024-04-25T18:59:21.230 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.228+0000 7f8ace800640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad0105d90 0x7f8ad01a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:21.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.229+0000 7f8ac5400640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ad01053d0 0x7f8ad019a940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9f0 tx=0x7f8ab800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:21.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.229+0000 7f8ac7400640 1 -- 172.21.15.45:0/3554007 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab800ccc0 con 0x7f8ad01053d0 2024-04-25T18:59:21.231 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.229+0000 7f8ad5200640 1 -- 172.21.15.45:0/3554007 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad01a2310 con 0x7f8ad01053d0 2024-04-25T18:59:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.229+0000 7f8ac7400640 1 -- 172.21.15.45:0/3554007 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ab80108e0 con 0x7f8ad01053d0 2024-04-25T18:59:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.229+0000 7f8ac7400640 1 -- 172.21.15.45:0/3554007 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8018880 con 0x7f8ad01053d0 2024-04-25T18:59:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.229+0000 7f8ad5200640 1 -- 172.21.15.45:0/3554007 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad01a28b0 con 0x7f8ad01053d0 2024-04-25T18:59:21.232 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.230+0000 7f8ad5200640 1 -- 172.21.15.45:0/3554007 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ad010b550 con 0x7f8ad01053d0 2024-04-25T18:59:21.233 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.233+0000 7f8ac7400640 1 -- 172.21.15.45:0/3554007 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7f8ab8010430 con 0x7f8ad01053d0 2024-04-25T18:59:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.234+0000 7f8ac7400640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ab00a3e20 0x7f8ab00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.234+0000 7f8ac7400640 1 -- 172.21.15.45:0/3554007 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f8ab80d0d80 con 0x7f8ad01053d0 2024-04-25T18:59:21.235 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.234+0000 7f8acde00640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ab00a3e20 0x7f8ab00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:21.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.235+0000 7f8acde00640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ab00a3e20 0x7f8ab00a62e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0005ab0 tx=0x7f8ac0005a20 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:21.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.241+0000 7f8ac7400640 1 -- 172.21.15.45:0/3554007 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ab809cc90 con 0x7f8ad01053d0 2024-04-25T18:59:21.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:20 smithi062 ceph-mon[32844]: [25/Apr/2024:18:59:19] ENGINE Bus STARTING 2024-04-25T18:59:21.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:20 smithi062 ceph-mon[32844]: [25/Apr/2024:18:59:19] ENGINE Serving on http://172.21.15.45:8765 2024-04-25T18:59:21.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:20 smithi062 ceph-mon[32844]: [25/Apr/2024:18:59:19] ENGINE Serving on https://172.21.15.45:7150 2024-04-25T18:59:21.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:20 smithi062 ceph-mon[32844]: [25/Apr/2024:18:59:19] ENGINE Bus STARTED 2024-04-25T18:59:21.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:20 smithi062 ceph-mon[32844]: from='client.25120 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:21.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:20 smithi062 ceph-mon[32844]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:21.482 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.481+0000 7f8ad5200640 1 -- 172.21.15.45:0/3554007 --> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f8ad0062390 con 0x7f8ab00a3e20 2024-04-25T18:59:21.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.491+0000 7f8ac7400640 1 -- 172.21.15.45:0/3554007 <== mgr.25049 v2:172.21.15.45:6800/1345298339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f8ad0062390 con 0x7f8ab00a3e20 2024-04-25T18:59:21.492 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T18:59:20", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T18:59:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 -- 172.21.15.45:0/3554007 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ab00a3e20 msgr2=0x7f8ab00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ab00a3e20 0x7f8ab00a62e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0005ab0 tx=0x7f8ac0005a20 comp rx=0 tx=0).stop 2024-04-25T18:59:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 -- 172.21.15.45:0/3554007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ad01053d0 msgr2=0x7f8ad019a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ad01053d0 0x7f8ad019a940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e9f0 tx=0x7f8ab800eeb0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 -- 172.21.15.45:0/3554007 shutdown_connections 2024-04-25T18:59:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ab00a3e20 0x7f8ab00a62e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8ad0105d90 0x7f8ad01a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ad01053d0 0x7f8ad019a940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 --2- 172.21.15.45:0/3554007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8ad01041e0 0x7f8ad019a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 -- 172.21.15.45:0/3554007 >> 172.21.15.45:0/3554007 conn(0x7f8ad00ff770 msgr2=0x7f8ad0108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 -- 172.21.15.45:0/3554007 shutdown_connections 2024-04-25T18:59:21.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.494+0000 7f8ad5200640 1 -- 172.21.15.45:0/3554007 wait complete. 2024-04-25T18:59:21.513 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T18:59:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.706+0000 7f2800800640 1 -- 172.21.15.45:0/1604302237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27f8075f90 msgr2=0x7f27f80743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.706+0000 7f2800800640 1 --2- 172.21.15.45:0/1604302237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27f8075f90 0x7f27f80743e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f27ec009900 tx=0x7f27ec02a400 comp rx=0 tx=0).stop 2024-04-25T18:59:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.706+0000 7f2800800640 1 -- 172.21.15.45:0/1604302237 shutdown_connections 2024-04-25T18:59:21.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.706+0000 7f2800800640 1 --2- 172.21.15.45:0/1604302237 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f8109e80 0x7f27f810c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.706+0000 7f2800800640 1 --2- 172.21.15.45:0/1604302237 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27f80749b0 0x7f27f8074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.706+0000 7f2800800640 1 --2- 172.21.15.45:0/1604302237 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27f8075f90 0x7f27f80743e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.706+0000 7f2800800640 1 -- 172.21.15.45:0/1604302237 >> 172.21.15.45:0/1604302237 conn(0x7f27f80fd530 msgr2=0x7f27f80ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.706+0000 7f2800800640 1 -- 172.21.15.45:0/1604302237 shutdown_connections 2024-04-25T18:59:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.706+0000 7f2800800640 1 -- 172.21.15.45:0/1604302237 wait complete. 2024-04-25T18:59:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.707+0000 7f2800800640 1 Processor -- start 2024-04-25T18:59:21.708 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f2800800640 1 -- start start 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f2800800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27f80749b0 0x7f27f819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f2800800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f8075f90 0x7f27f819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f2800800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27f8109e80 0x7f27f81a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f2800800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8102230 con 0x7f27f8109e80 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f2800800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81020b0 con 0x7f27f8075f90 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f2800800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81023b0 con 0x7f27f80749b0 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f27fde00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27f80749b0 0x7f27f819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f27fde00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27f80749b0 0x7f27f819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:59666/0 (socket says 172.21.15.45:59666) 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f27fde00640 1 -- 172.21.15.45:0/4264361439 learned_addr learned my addr 172.21.15.45:0/4264361439 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.708+0000 7f27fd400640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f8075f90 0x7f27f819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:21.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.709+0000 7f27fe800640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27f8109e80 0x7f27f81a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:21.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.709+0000 7f27fde00640 1 -- 172.21.15.45:0/4264361439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f8075f90 msgr2=0x7f27f819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:21.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.709+0000 7f27fde00640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f8075f90 0x7f27f819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.709+0000 7f27fde00640 1 -- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27f8109e80 msgr2=0x7f27f81a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:21.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.709+0000 7f27fde00640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27f8109e80 0x7f27f81a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:21.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.709+0000 7f27fde00640 1 -- 172.21.15.45:0/4264361439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f81a6460 con 0x7f27f80749b0 2024-04-25T18:59:21.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.709+0000 7f27fe800640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27f8109e80 0x7f27f81a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:59:21.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.709+0000 7f27fde00640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27f80749b0 0x7f27f819e840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27ec005ab0 tx=0x7f27ec003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:21.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.710+0000 7f27f2a00640 1 -- 172.21.15.45:0/4264361439 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec03f070 con 0x7f27f80749b0 2024-04-25T18:59:21.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.710+0000 7f27f2a00640 1 -- 172.21.15.45:0/4264361439 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27ec031c50 con 0x7f27f80749b0 2024-04-25T18:59:21.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.710+0000 7f2800800640 1 -- 172.21.15.45:0/4264361439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f81a66f0 con 0x7f27f80749b0 2024-04-25T18:59:21.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.710+0000 7f27f2a00640 1 -- 172.21.15.45:0/4264361439 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec035030 con 0x7f27f80749b0 2024-04-25T18:59:21.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.710+0000 7f2800800640 1 -- 172.21.15.45:0/4264361439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f81a6c30 con 0x7f27f80749b0 2024-04-25T18:59:21.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.711+0000 7f2800800640 1 -- 172.21.15.45:0/4264361439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f8105a30 con 0x7f27f80749b0 2024-04-25T18:59:21.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.713+0000 7f27f2a00640 1 -- 172.21.15.45:0/4264361439 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 24) ==== 138817+0+0 (secure 0 0 0) 0x7f27ec042070 con 0x7f27f80749b0 2024-04-25T18:59:21.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.714+0000 7f27f2a00640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f27e00a3e20 0x7f27e00a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:21.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.714+0000 7f27f2a00640 1 -- 172.21.15.45:0/4264361439 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f27ec0f1170 con 0x7f27f80749b0 2024-04-25T18:59:21.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.715+0000 7f27fd400640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f27e00a3e20 0x7f27e00a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:21.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.715+0000 7f27fd400640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f27e00a3e20 0x7f27e00a62e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27f819fd50 tx=0x7f27e8005f90 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:21.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:21.718+0000 7f27f2a00640 1 -- 172.21.15.45:0/4264361439 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27ec0bd640 con 0x7f27f80749b0 2024-04-25T18:59:22.016 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.015+0000 7f2800800640 1 -- 172.21.15.45:0/4264361439 --> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f27f8062390 con 0x7f27e00a3e20 2024-04-25T18:59:22.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.026+0000 7f27f2a00640 1 -- 172.21.15.45:0/4264361439 <== mgr.25049 v2:172.21.15.45:6800/1345298339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f27f8062390 con 0x7f27e00a3e20 2024-04-25T18:59:22.027 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.028+0000 7f2800800640 1 -- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f27e00a3e20 msgr2=0x7f27e00a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.028+0000 7f2800800640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f27e00a3e20 0x7f27e00a62e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27f819fd50 tx=0x7f27e8005f90 comp rx=0 tx=0).stop 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.028+0000 7f2800800640 1 -- 172.21.15.45:0/4264361439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27f80749b0 msgr2=0x7f27f819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.028+0000 7f2800800640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27f80749b0 0x7f27f819e840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f27ec005ab0 tx=0x7f27ec003b90 comp rx=0 tx=0).stop 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.029+0000 7f2800800640 1 -- 172.21.15.45:0/4264361439 shutdown_connections 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.029+0000 7f2800800640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f27e00a3e20 0x7f27e00a62e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.029+0000 7f2800800640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f27f8109e80 0x7f27f81a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.029+0000 7f2800800640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f27f8075f90 0x7f27f819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.029+0000 7f2800800640 1 --2- 172.21.15.45:0/4264361439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f27f80749b0 0x7f27f819e840 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.029+0000 7f2800800640 1 -- 172.21.15.45:0/4264361439 >> 172.21.15.45:0/4264361439 conn(0x7f27f80fd530 msgr2=0x7f27f8102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.029+0000 7f2800800640 1 -- 172.21.15.45:0/4264361439 shutdown_connections 2024-04-25T18:59:22.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:22.029+0000 7f2800800640 1 -- 172.21.15.45:0/4264361439 wait complete. 2024-04-25T18:59:22.048 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T18:59:22.124 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T18:59:22.134 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T18:59:22.143 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T18:59:22.143 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T18:59:22.149 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T18:59:22.161 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T18:59:22.161 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:59:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:21 smithi079 ceph-mon[33002]: from='client.25132 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:21 smithi079 ceph-mon[33002]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:21 smithi079 ceph-mon[33002]: from='client.25130 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:21 smithi079 ceph-mon[33002]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-25T18:59:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:21 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:21 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:21 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:21 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:21 smithi045 ceph-mon[27776]: from='client.25132 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:22.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:21 smithi045 ceph-mon[27776]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:22.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:21 smithi045 ceph-mon[27776]: from='client.25130 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:22.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:21 smithi045 ceph-mon[27776]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-25T18:59:22.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:21 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:21 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:21 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.208 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:21 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:21 smithi062 ceph-mon[32844]: from='client.25132 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:21 smithi062 ceph-mon[32844]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:21 smithi062 ceph-mon[32844]: from='client.25130 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:21 smithi062 ceph-mon[32844]: mgrmap e24: x(active, since 2s), standbys: z, y 2024-04-25T18:59:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:21 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:21 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:21 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:21 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:22.595 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:23.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='client.25150 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:23.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='client.25156 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:23.399 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: Unable to set osd_memory_target on smithi079 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.400 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.401 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:23 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='client.25150 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='client.25156 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-04-25T18:59:23.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: Unable to set osd_memory_target on smithi079 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.436 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:23 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='client.25150 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:23.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='client.25156 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T18:59:23.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: Unable to set osd_memory_target on smithi079 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.466 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.466 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:23.466 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:23 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:24.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:24 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi062 to 176.2M 2024-04-25T18:59:24.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:24 smithi045 ceph-mon[27776]: Unable to set osd_memory_target on smithi062 to 184794624: error parsing value: Value '184794624' is below minimum 939524096 2024-04-25T18:59:24.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:24 smithi045 ceph-mon[27776]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:24.328 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:24 smithi045 ceph-mon[27776]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-25T18:59:24.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:24 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi062 to 176.2M 2024-04-25T18:59:24.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:24 smithi062 ceph-mon[32844]: Unable to set osd_memory_target on smithi062 to 184794624: error parsing value: Value '184794624' is below minimum 939524096 2024-04-25T18:59:24.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:24 smithi062 ceph-mon[32844]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:24.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:24 smithi062 ceph-mon[32844]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-25T18:59:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:24 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi062 to 176.2M 2024-04-25T18:59:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:24 smithi079 ceph-mon[33002]: Unable to set osd_memory_target on smithi062 to 184794624: error parsing value: Value '184794624' is below minimum 939524096 2024-04-25T18:59:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:24 smithi079 ceph-mon[33002]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:24 smithi079 ceph-mon[33002]: mgrmap e25: x(active, since 4s), standbys: z, y 2024-04-25T18:59:24.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.831+0000 7f819ce00700 1 -- 172.21.15.45:0/4251385504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8198103450 msgr2=0x7f819810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:24.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.831+0000 7f819ce00700 1 --2- 172.21.15.45:0/4251385504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8198103450 0x7f819810f8b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f819000b0a0 tx=0x7f819000b3b0 comp rx=0 tx=0).stop 2024-04-25T18:59:24.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.831+0000 7f819ce00700 1 -- 172.21.15.45:0/4251385504 shutdown_connections 2024-04-25T18:59:24.833 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.831+0000 7f819ce00700 1 --2- 172.21.15.45:0/4251385504 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8198108a50 0x7f8198108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:24.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.831+0000 7f819ce00700 1 --2- 172.21.15.45:0/4251385504 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8198103450 0x7f819810f8b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:24.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.831+0000 7f819ce00700 1 --2- 172.21.15.45:0/4251385504 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8198102a90 0x7f8198102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:24.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.831+0000 7f819ce00700 1 -- 172.21.15.45:0/4251385504 >> 172.21.15.45:0/4251385504 conn(0x7f8198079ba0 msgr2=0x7f8198079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:24.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.832+0000 7f819ce00700 1 -- 172.21.15.45:0/4251385504 shutdown_connections 2024-04-25T18:59:24.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.832+0000 7f819ce00700 1 -- 172.21.15.45:0/4251385504 wait complete. 2024-04-25T18:59:24.834 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.833+0000 7f819ce00700 1 Processor -- start 2024-04-25T18:59:24.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.833+0000 7f819ce00700 1 -- start start 2024-04-25T18:59:24.835 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.833+0000 7f819ce00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8198102a90 0x7f8198074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.833+0000 7f819ce00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8198103450 0x7f8198075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.833+0000 7f819ce00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8198108a50 0x7f81980757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.833+0000 7f819ce00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8198112be0 con 0x7f8198102a90 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.833+0000 7f819ce00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8198112a60 con 0x7f8198103450 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.833+0000 7f819ce00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8198112d60 con 0x7f8198108a50 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8195400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8198103450 0x7f8198075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8195400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8198103450 0x7f8198075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:57208/0 (socket says 172.21.15.45:57208) 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8195400700 1 -- 172.21.15.45:0/554904961 learned_addr learned my addr 172.21.15.45:0/554904961 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8196800700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8198108a50 0x7f81980757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8195e00700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8198102a90 0x7f8198074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8195400700 1 -- 172.21.15.45:0/554904961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8198108a50 msgr2=0x7f81980757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8195400700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8198108a50 0x7f81980757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:24.836 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8195400700 1 -- 172.21.15.45:0/554904961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8198102a90 msgr2=0x7f8198074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:24.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8195400700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8198102a90 0x7f8198074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:24.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8195400700 1 -- 172.21.15.45:0/554904961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819819d1e0 con 0x7f8198103450 2024-04-25T18:59:24.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.834+0000 7f8196800700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8198108a50 0x7f81980757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:24.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.835+0000 7f8195400700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8198103450 0x7f8198075280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f818400e9a0 tx=0x7f818400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:24.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.835+0000 7f818ea00700 1 -- 172.21.15.45:0/554904961 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818400cbd0 con 0x7f8198103450 2024-04-25T18:59:24.837 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.835+0000 7f818ea00700 1 -- 172.21.15.45:0/554904961 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818400cd70 con 0x7f8198103450 2024-04-25T18:59:24.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.835+0000 7f819ce00700 1 -- 172.21.15.45:0/554904961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819819d4d0 con 0x7f8198103450 2024-04-25T18:59:24.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.835+0000 7f818ea00700 1 -- 172.21.15.45:0/554904961 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184018680 con 0x7f8198103450 2024-04-25T18:59:24.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.835+0000 7f819ce00700 1 -- 172.21.15.45:0/554904961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819819da60 con 0x7f8198103450 2024-04-25T18:59:24.838 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.837+0000 7f819ce00700 1 -- 172.21.15.45:0/554904961 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8198104180 con 0x7f8198103450 2024-04-25T18:59:24.840 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.839+0000 7f818ea00700 1 -- 172.21.15.45:0/554904961 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f8184010430 con 0x7f8198103450 2024-04-25T18:59:24.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.840+0000 7f818ea00700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f817c0a3f50 0x7f817c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:24.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.840+0000 7f818ea00700 1 -- 172.21.15.45:0/554904961 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f81840d0cf0 con 0x7f8198103450 2024-04-25T18:59:24.841 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.841+0000 7f8195e00700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f817c0a3f50 0x7f817c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:24.842 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.841+0000 7f8195e00700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f817c0a3f50 0x7f817c0a6410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8188009680 tx=0x7f8188009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:24.846 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:24.845+0000 7f818ea00700 1 -- 172.21.15.45:0/554904961 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f818409d0e0 con 0x7f8198103450 2024-04-25T18:59:25.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.088+0000 7f819ce00700 1 -- 172.21.15.45:0/554904961 --> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8198076270 con 0x7f817c0a3f50 2024-04-25T18:59:25.099 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.098+0000 7f818ea00700 1 -- 172.21.15.45:0/554904961 <== mgr.25049 v2:172.21.15.45:6800/1345298339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257783 (secure 0 0 0) 0x7f8198076270 con 0x7f817c0a3f50 2024-04-25T18:59:25.104 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 -- 172.21.15.45:0/554904961 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f817c0a3f50 msgr2=0x7f817c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f817c0a3f50 0x7f817c0a6410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8188009680 tx=0x7f8188009260 comp rx=0 tx=0).stop 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 -- 172.21.15.45:0/554904961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8198103450 msgr2=0x7f8198075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8198103450 0x7f8198075280 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f818400e9a0 tx=0x7f818400ecb0 comp rx=0 tx=0).stop 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 -- 172.21.15.45:0/554904961 shutdown_connections 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f817c0a3f50 0x7f817c0a6410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8198108a50 0x7f81980757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8198102a90 0x7f8198074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 --2- 172.21.15.45:0/554904961 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8198103450 0x7f8198075280 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 -- 172.21.15.45:0/554904961 >> 172.21.15.45:0/554904961 conn(0x7f8198079ba0 msgr2=0x7f8198101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:25.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.103+0000 7f819ce00700 1 -- 172.21.15.45:0/554904961 shutdown_connections 2024-04-25T18:59:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:25.104+0000 7f819ce00700 1 -- 172.21.15.45:0/554904961 wait complete. 2024-04-25T18:59:25.107 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T18:59:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-25T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-25T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:25 smithi045 ceph-mon[27776]: Updating smithi079:/etc/ceph/ceph.conf 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-25T18:59:25.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T18:59:25.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:59:25.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:59:25.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T18:59:25.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-25T18:59:25.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:25 smithi062 ceph-mon[32844]: Updating smithi079:/etc/ceph/ceph.conf 2024-04-25T18:59:25.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:25.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:25.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: Updating smithi062:/etc/ceph/ceph.conf 2024-04-25T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:25 smithi079 ceph-mon[33002]: Updating smithi079:/etc/ceph/ceph.conf 2024-04-25T18:59:26.211 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-25T18:59:24.777155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1192312,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1546,"num_read_kb":938,"num_write":482,"num_write_kb":4386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":873,"ondisk_log_size":873,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394084,"kb_used_data":15780,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124547740,"statfs":{"total":1151940427776,"available":1151536885760,"internally_reserved":0,"allocated":16158720,"data_stored":9505297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.957039"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192112+0000","last_change":"2024-04-25T18:55:23.241030+0000","last_active":"2024-04-25T18:59:19.192112+0000","last_peered":"2024-04-25T18:59:19.192112+0000","last_clean":"2024-04-25T18:59:19.192112+0000","last_became_active":"2024-04-25T18:55:23.240724+0000","last_became_peered":"2024-04-25T18:55:23.240724+0000","last_unstale":"2024-04-25T18:59:19.192112+0000","last_undegraded":"2024-04-25T18:59:19.192112+0000","last_fullsized":"2024-04-25T18:59:19.192112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:37.995475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193313+0000","last_change":"2024-04-25T18:55:23.229875+0000","last_active":"2024-04-25T18:59:19.193313+0000","last_peered":"2024-04-25T18:59:19.193313+0000","last_clean":"2024-04-25T18:59:19.193313+0000","last_became_active":"2024-04-25T18:55:23.229640+0000","last_became_peered":"2024-04-25T18:55:23.229640+0000","last_unstale":"2024-04-25T18:59:19.193313+0000","last_undegraded":"2024-04-25T18:59:19.193313+0000","last_fullsized":"2024-04-25T18:59:19.193313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:10.262376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193164+0000","last_change":"2024-04-25T18:55:23.229821+0000","last_active":"2024-04-25T18:59:19.193164+0000","last_peered":"2024-04-25T18:59:19.193164+0000","last_clean":"2024-04-25T18:59:19.193164+0000","last_became_active":"2024-04-25T18:55:23.229414+0000","last_became_peered":"2024-04-25T18:55:23.229414+0000","last_unstale":"2024-04-25T18:59:19.193164+0000","last_undegraded":"2024-04-25T18:59:19.193164+0000","last_fullsized":"2024-04-25T18:59:19.193164+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:56.063786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907514+0000","last_change":"2024-04-25T18:56:17.560167+0000","last_active":"2024-04-25T18:59:18.907514+0000","last_peered":"2024-04-25T18:59:18.907514+0000","last_clean":"2024-04-25T18:59:18.907514+0000","last_became_active":"2024-04-25T18:56:17.558466+0000","last_became_peered":"2024-04-25T18:56:17.558466+0000","last_unstale":"2024-04-25T18:59:18.907514+0000","last_undegraded":"2024-04-25T18:59:18.907514+0000","last_fullsized":"2024-04-25T18:59:18.907514+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:22.371076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757484+0000","last_change":"2024-04-25T18:55:23.233512+0000","last_active":"2024-04-25T18:59:18.757484+0000","last_peered":"2024-04-25T18:59:18.757484+0000","last_clean":"2024-04-25T18:59:18.757484+0000","last_became_active":"2024-04-25T18:55:23.233158+0000","last_became_peered":"2024-04-25T18:55:23.233158+0000","last_unstale":"2024-04-25T18:59:18.757484+0000","last_undegraded":"2024-04-25T18:59:18.757484+0000","last_fullsized":"2024-04-25T18:59:18.757484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:44.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928760+0000","last_change":"2024-04-25T18:55:23.225921+0000","last_active":"2024-04-25T18:59:18.928760+0000","last_peered":"2024-04-25T18:59:18.928760+0000","last_clean":"2024-04-25T18:59:18.928760+0000","last_became_active":"2024-04-25T18:55:23.225547+0000","last_became_peered":"2024-04-25T18:55:23.225547+0000","last_unstale":"2024-04-25T18:59:18.928760+0000","last_undegraded":"2024-04-25T18:59:18.928760+0000","last_fullsized":"2024-04-25T18:59:18.928760+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:29.828394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758001+0000","last_change":"2024-04-25T18:55:23.217111+0000","last_active":"2024-04-25T18:59:18.758001+0000","last_peered":"2024-04-25T18:59:18.758001+0000","last_clean":"2024-04-25T18:59:18.758001+0000","last_became_active":"2024-04-25T18:55:23.216731+0000","last_became_peered":"2024-04-25T18:55:23.216731+0000","last_unstale":"2024-04-25T18:59:18.758001+0000","last_undegraded":"2024-04-25T18:59:18.758001+0000","last_fullsized":"2024-04-25T18:59:18.758001+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:17.931928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907086+0000","last_change":"2024-04-25T18:56:17.555014+0000","last_active":"2024-04-25T18:59:18.907086+0000","last_peered":"2024-04-25T18:59:18.907086+0000","last_clean":"2024-04-25T18:59:18.907086+0000","last_became_active":"2024-04-25T18:56:17.553863+0000","last_became_peered":"2024-04-25T18:56:17.553863+0000","last_unstale":"2024-04-25T18:59:18.907086+0000","last_undegraded":"2024-04-25T18:59:18.907086+0000","last_fullsized":"2024-04-25T18:59:18.907086+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.240754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132827+0000","last_change":"2024-04-25T18:55:23.218197+0000","last_active":"2024-04-25T18:59:19.132827+0000","last_peered":"2024-04-25T18:59:19.132827+0000","last_clean":"2024-04-25T18:59:19.132827+0000","last_became_active":"2024-04-25T18:55:23.217665+0000","last_became_peered":"2024-04-25T18:55:23.217665+0000","last_unstale":"2024-04-25T18:59:19.132827+0000","last_undegraded":"2024-04-25T18:59:19.132827+0000","last_fullsized":"2024-04-25T18:59:19.132827+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:11.829991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.929205+0000","last_change":"2024-04-25T18:55:23.221533+0000","last_active":"2024-04-25T18:59:18.929205+0000","last_peered":"2024-04-25T18:59:18.929205+0000","last_clean":"2024-04-25T18:59:18.929205+0000","last_became_active":"2024-04-25T18:55:23.220711+0000","last_became_peered":"2024-04-25T18:55:23.220711+0000","last_unstale":"2024-04-25T18:59:18.929205+0000","last_undegraded":"2024-04-25T18:59:18.929205+0000","last_fullsized":"2024-04-25T18:59:18.929205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:45.185166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758645+0000","last_change":"2024-04-25T18:55:23.221329+0000","last_active":"2024-04-25T18:59:18.758645+0000","last_peered":"2024-04-25T18:59:18.758645+0000","last_clean":"2024-04-25T18:59:18.758645+0000","last_became_active":"2024-04-25T18:55:23.220156+0000","last_became_peered":"2024-04-25T18:55:23.220156+0000","last_unstale":"2024-04-25T18:59:18.758645+0000","last_undegraded":"2024-04-25T18:59:18.758645+0000","last_fullsized":"2024-04-25T18:59:18.758645+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:46.981199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907481+0000","last_change":"2024-04-25T18:56:17.558018+0000","last_active":"2024-04-25T18:59:18.907481+0000","last_peered":"2024-04-25T18:59:18.907481+0000","last_clean":"2024-04-25T18:59:18.907481+0000","last_became_active":"2024-04-25T18:56:17.556229+0000","last_became_peered":"2024-04-25T18:56:17.556229+0000","last_unstale":"2024-04-25T18:59:18.907481+0000","last_undegraded":"2024-04-25T18:59:18.907481+0000","last_fullsized":"2024-04-25T18:59:18.907481+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:50.949895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757146+0000","last_change":"2024-04-25T18:55:23.227474+0000","last_active":"2024-04-25T18:59:18.757146+0000","last_peered":"2024-04-25T18:59:18.757146+0000","last_clean":"2024-04-25T18:59:18.757146+0000","last_became_active":"2024-04-25T18:55:23.227068+0000","last_became_peered":"2024-04-25T18:55:23.227068+0000","last_unstale":"2024-04-25T18:59:18.757146+0000","last_undegraded":"2024-04-25T18:59:18.757146+0000","last_fullsized":"2024-04-25T18:59:18.757146+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:13.884833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192323+0000","last_change":"2024-04-25T18:55:23.230572+0000","last_active":"2024-04-25T18:59:19.192323+0000","last_peered":"2024-04-25T18:59:19.192323+0000","last_clean":"2024-04-25T18:59:19.192323+0000","last_became_active":"2024-04-25T18:55:23.229756+0000","last_became_peered":"2024-04-25T18:55:23.229756+0000","last_unstale":"2024-04-25T18:59:19.192323+0000","last_undegraded":"2024-04-25T18:59:19.192323+0000","last_fullsized":"2024-04-25T18:59:19.192323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:15.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192637+0000","last_change":"2024-04-25T18:55:23.227913+0000","last_active":"2024-04-25T18:59:19.192637+0000","last_peered":"2024-04-25T18:59:19.192637+0000","last_clean":"2024-04-25T18:59:19.192637+0000","last_became_active":"2024-04-25T18:55:23.227598+0000","last_became_peered":"2024-04-25T18:55:23.227598+0000","last_unstale":"2024-04-25T18:59:19.192637+0000","last_undegraded":"2024-04-25T18:59:19.192637+0000","last_fullsized":"2024-04-25T18:59:19.192637+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:33.555741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132304+0000","last_change":"2024-04-25T18:55:23.230241+0000","last_active":"2024-04-25T18:59:19.132304+0000","last_peered":"2024-04-25T18:59:19.132304+0000","last_clean":"2024-04-25T18:59:19.132304+0000","last_became_active":"2024-04-25T18:55:23.229898+0000","last_became_peered":"2024-04-25T18:55:23.229898+0000","last_unstale":"2024-04-25T18:59:19.132304+0000","last_undegraded":"2024-04-25T18:59:19.132304+0000","last_fullsized":"2024-04-25T18:59:19.132304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:27.278475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192205+0000","last_change":"2024-04-25T18:55:23.240978+0000","last_active":"2024-04-25T18:59:19.192205+0000","last_peered":"2024-04-25T18:59:19.192205+0000","last_clean":"2024-04-25T18:59:19.192205+0000","last_became_active":"2024-04-25T18:55:23.240702+0000","last_became_peered":"2024-04-25T18:55:23.240702+0000","last_unstale":"2024-04-25T18:59:19.192205+0000","last_undegraded":"2024-04-25T18:59:19.192205+0000","last_fullsized":"2024-04-25T18:59:19.192205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:06.053688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907730+0000","last_change":"2024-04-25T18:55:23.232116+0000","last_active":"2024-04-25T18:59:18.907730+0000","last_peered":"2024-04-25T18:59:18.907730+0000","last_clean":"2024-04-25T18:59:18.907730+0000","last_became_active":"2024-04-25T18:55:23.231586+0000","last_became_peered":"2024-04-25T18:55:23.231586+0000","last_unstale":"2024-04-25T18:59:18.907730+0000","last_undegraded":"2024-04-25T18:59:18.907730+0000","last_fullsized":"2024-04-25T18:59:18.907730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:32.640823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129676+0000","last_change":"2024-04-25T18:55:23.224519+0000","last_active":"2024-04-25T18:59:19.129676+0000","last_peered":"2024-04-25T18:59:19.129676+0000","last_clean":"2024-04-25T18:59:19.129676+0000","last_became_active":"2024-04-25T18:55:23.224145+0000","last_became_peered":"2024-04-25T18:55:23.224145+0000","last_unstale":"2024-04-25T18:59:19.129676+0000","last_undegraded":"2024-04-25T18:59:19.129676+0000","last_fullsized":"2024-04-25T18:59:19.129676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:03.340162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907764+0000","last_change":"2024-04-25T18:55:19.164977+0000","last_active":"2024-04-25T18:59:18.907764+0000","last_peered":"2024-04-25T18:59:18.907764+0000","last_clean":"2024-04-25T18:59:18.907764+0000","last_became_active":"2024-04-25T18:55:19.164316+0000","last_became_peered":"2024-04-25T18:55:19.164316+0000","last_unstale":"2024-04-25T18:59:18.907764+0000","last_undegraded":"2024-04-25T18:59:18.907764+0000","last_fullsized":"2024-04-25T18:59:18.907764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:08.031680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895526+0000","last_change":"2024-04-25T18:56:17.559746+0000","last_active":"2024-04-25T18:59:18.895526+0000","last_peered":"2024-04-25T18:59:18.895526+0000","last_clean":"2024-04-25T18:59:18.895526+0000","last_became_active":"2024-04-25T18:56:17.558761+0000","last_became_peered":"2024-04-25T18:56:17.558761+0000","last_unstale":"2024-04-25T18:59:18.895526+0000","last_undegraded":"2024-04-25T18:59:18.895526+0000","last_fullsized":"2024-04-25T18:59:18.895526+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:43.518760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895716+0000","last_change":"2024-04-25T18:55:19.179943+0000","last_active":"2024-04-25T18:59:18.895716+0000","last_peered":"2024-04-25T18:59:18.895716+0000","last_clean":"2024-04-25T18:59:18.895716+0000","last_became_active":"2024-04-25T18:55:19.179519+0000","last_became_peered":"2024-04-25T18:55:19.179519+0000","last_unstale":"2024-04-25T18:59:18.895716+0000","last_undegraded":"2024-04-25T18:59:18.895716+0000","last_fullsized":"2024-04-25T18:59:18.895716+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:46.365187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193409+0000","last_change":"2024-04-25T18:55:23.237965+0000","last_active":"2024-04-25T18:59:19.193409+0000","last_peered":"2024-04-25T18:59:19.193409+0000","last_clean":"2024-04-25T18:59:19.193409+0000","last_became_active":"2024-04-25T18:55:23.237520+0000","last_became_peered":"2024-04-25T18:55:23.237520+0000","last_unstale":"2024-04-25T18:59:19.193409+0000","last_undegraded":"2024-04-25T18:59:19.193409+0000","last_fullsized":"2024-04-25T18:59:19.193409+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:03.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757872+0000","last_change":"2024-04-25T18:55:19.163965+0000","last_active":"2024-04-25T18:59:18.757872+0000","last_peered":"2024-04-25T18:59:18.757872+0000","last_clean":"2024-04-25T18:59:18.757872+0000","last_became_active":"2024-04-25T18:55:19.163390+0000","last_became_peered":"2024-04-25T18:55:19.163390+0000","last_unstale":"2024-04-25T18:59:18.757872+0000","last_undegraded":"2024-04-25T18:59:18.757872+0000","last_fullsized":"2024-04-25T18:59:18.757872+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:54.183730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132443+0000","last_change":"2024-04-25T18:55:23.227124+0000","last_active":"2024-04-25T18:59:19.132443+0000","last_peered":"2024-04-25T18:59:19.132443+0000","last_clean":"2024-04-25T18:59:19.132443+0000","last_became_active":"2024-04-25T18:55:23.226383+0000","last_became_peered":"2024-04-25T18:55:23.226383+0000","last_unstale":"2024-04-25T18:59:19.132443+0000","last_undegraded":"2024-04-25T18:59:19.132443+0000","last_fullsized":"2024-04-25T18:59:19.132443+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:22.011212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191814+0000","last_change":"2024-04-25T18:56:17.544303+0000","last_active":"2024-04-25T18:59:19.191814+0000","last_peered":"2024-04-25T18:59:19.191814+0000","last_clean":"2024-04-25T18:59:19.191814+0000","last_became_active":"2024-04-25T18:56:17.542002+0000","last_became_peered":"2024-04-25T18:56:17.542002+0000","last_unstale":"2024-04-25T18:59:19.191814+0000","last_undegraded":"2024-04-25T18:59:19.191814+0000","last_fullsized":"2024-04-25T18:59:19.191814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:23.848398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923703+0000","last_change":"2024-04-25T18:55:23.217703+0000","last_active":"2024-04-25T18:59:18.923703+0000","last_peered":"2024-04-25T18:59:18.923703+0000","last_clean":"2024-04-25T18:59:18.923703+0000","last_became_active":"2024-04-25T18:55:23.216767+0000","last_became_peered":"2024-04-25T18:55:23.216767+0000","last_unstale":"2024-04-25T18:59:18.923703+0000","last_undegraded":"2024-04-25T18:59:18.923703+0000","last_fullsized":"2024-04-25T18:59:18.923703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:17.779356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":257,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.741346+0000","last_change":"2024-04-25T18:55:19.164199+0000","last_active":"2024-04-25T18:59:19.741346+0000","last_peered":"2024-04-25T18:59:19.741346+0000","last_clean":"2024-04-25T18:59:19.741346+0000","last_became_active":"2024-04-25T18:55:19.163849+0000","last_became_peered":"2024-04-25T18:55:19.163849+0000","last_unstale":"2024-04-25T18:59:19.741346+0000","last_undegraded":"2024-04-25T18:59:19.741346+0000","last_fullsized":"2024-04-25T18:59:19.741346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:25.234229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":219,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757254+0000","last_change":"2024-04-25T18:55:23.216916+0000","last_active":"2024-04-25T18:59:18.757254+0000","last_peered":"2024-04-25T18:59:18.757254+0000","last_clean":"2024-04-25T18:59:18.757254+0000","last_became_active":"2024-04-25T18:55:23.216249+0000","last_became_peered":"2024-04-25T18:55:23.216249+0000","last_unstale":"2024-04-25T18:59:18.757254+0000","last_undegraded":"2024-04-25T18:59:18.757254+0000","last_fullsized":"2024-04-25T18:59:18.757254+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:21.522318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928712+0000","last_change":"2024-04-25T18:55:19.171004+0000","last_active":"2024-04-25T18:59:18.928712+0000","last_peered":"2024-04-25T18:59:18.928712+0000","last_clean":"2024-04-25T18:59:18.928712+0000","last_became_active":"2024-04-25T18:55:19.170570+0000","last_became_peered":"2024-04-25T18:55:19.170570+0000","last_unstale":"2024-04-25T18:59:18.928712+0000","last_undegraded":"2024-04-25T18:59:18.928712+0000","last_fullsized":"2024-04-25T18:59:18.928712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:15:24.954033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923953+0000","last_change":"2024-04-25T18:56:17.552615+0000","last_active":"2024-04-25T18:59:18.923953+0000","last_peered":"2024-04-25T18:59:18.923953+0000","last_clean":"2024-04-25T18:59:18.923953+0000","last_became_active":"2024-04-25T18:56:17.552154+0000","last_became_peered":"2024-04-25T18:56:17.552154+0000","last_unstale":"2024-04-25T18:59:18.923953+0000","last_undegraded":"2024-04-25T18:59:18.923953+0000","last_fullsized":"2024-04-25T18:59:18.923953+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:01.199859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757300+0000","last_change":"2024-04-25T18:55:19.173398+0000","last_active":"2024-04-25T18:59:18.757300+0000","last_peered":"2024-04-25T18:59:18.757300+0000","last_clean":"2024-04-25T18:59:18.757300+0000","last_became_active":"2024-04-25T18:55:19.171477+0000","last_became_peered":"2024-04-25T18:55:19.171477+0000","last_unstale":"2024-04-25T18:59:18.757300+0000","last_undegraded":"2024-04-25T18:59:18.757300+0000","last_fullsized":"2024-04-25T18:59:18.757300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:35.230596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895264+0000","last_change":"2024-04-25T18:55:23.235209+0000","last_active":"2024-04-25T18:59:18.895264+0000","last_peered":"2024-04-25T18:59:18.895264+0000","last_clean":"2024-04-25T18:59:18.895264+0000","last_became_active":"2024-04-25T18:55:23.231898+0000","last_became_peered":"2024-04-25T18:55:23.231898+0000","last_unstale":"2024-04-25T18:59:18.895264+0000","last_undegraded":"2024-04-25T18:59:18.895264+0000","last_fullsized":"2024-04-25T18:59:18.895264+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.521282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923448+0000","last_change":"2024-04-25T18:55:19.163780+0000","last_active":"2024-04-25T18:59:18.923448+0000","last_peered":"2024-04-25T18:59:18.923448+0000","last_clean":"2024-04-25T18:59:18.923448+0000","last_became_active":"2024-04-25T18:55:19.163197+0000","last_became_peered":"2024-04-25T18:55:19.163197+0000","last_unstale":"2024-04-25T18:59:18.923448+0000","last_undegraded":"2024-04-25T18:59:18.923448+0000","last_fullsized":"2024-04-25T18:59:18.923448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:02:26.519160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907206+0000","last_change":"2024-04-25T18:56:17.556804+0000","last_active":"2024-04-25T18:59:18.907206+0000","last_peered":"2024-04-25T18:59:18.907206+0000","last_clean":"2024-04-25T18:59:18.907206+0000","last_became_active":"2024-04-25T18:56:17.554885+0000","last_became_peered":"2024-04-25T18:56:17.554885+0000","last_unstale":"2024-04-25T18:59:18.907206+0000","last_undegraded":"2024-04-25T18:59:18.907206+0000","last_fullsized":"2024-04-25T18:59:18.907206+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:36.316156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":248,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.130034+0000","last_change":"2024-04-25T18:55:19.170565+0000","last_active":"2024-04-25T18:59:19.130034+0000","last_peered":"2024-04-25T18:59:19.130034+0000","last_clean":"2024-04-25T18:59:19.130034+0000","last_became_active":"2024-04-25T18:55:19.170162+0000","last_became_peered":"2024-04-25T18:55:19.170162+0000","last_unstale":"2024-04-25T18:59:19.130034+0000","last_undegraded":"2024-04-25T18:59:19.130034+0000","last_fullsized":"2024-04-25T18:59:19.130034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:36.606661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":214,"num_read_kb":214,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132749+0000","last_change":"2024-04-25T18:55:23.227436+0000","last_active":"2024-04-25T18:59:19.132749+0000","last_peered":"2024-04-25T18:59:19.132749+0000","last_clean":"2024-04-25T18:59:19.132749+0000","last_became_active":"2024-04-25T18:55:23.227044+0000","last_became_peered":"2024-04-25T18:55:23.227044+0000","last_unstale":"2024-04-25T18:59:19.132749+0000","last_undegraded":"2024-04-25T18:59:19.132749+0000","last_fullsized":"2024-04-25T18:59:19.132749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:34.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758348+0000","last_change":"2024-04-25T18:55:19.167514+0000","last_active":"2024-04-25T18:59:18.758348+0000","last_peered":"2024-04-25T18:59:18.758348+0000","last_clean":"2024-04-25T18:59:18.758348+0000","last_became_active":"2024-04-25T18:55:19.167102+0000","last_became_peered":"2024-04-25T18:55:19.167102+0000","last_unstale":"2024-04-25T18:59:18.758348+0000","last_undegraded":"2024-04-25T18:59:18.758348+0000","last_fullsized":"2024-04-25T18:59:18.758348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:41.472274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193323+0000","last_change":"2024-04-25T18:55:23.237012+0000","last_active":"2024-04-25T18:59:19.193323+0000","last_peered":"2024-04-25T18:59:19.193323+0000","last_clean":"2024-04-25T18:59:19.193323+0000","last_became_active":"2024-04-25T18:55:23.230263+0000","last_became_peered":"2024-04-25T18:55:23.230263+0000","last_unstale":"2024-04-25T18:59:19.193323+0000","last_undegraded":"2024-04-25T18:59:19.193323+0000","last_fullsized":"2024-04-25T18:59:19.193323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:30.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923910+0000","last_change":"2024-04-25T18:55:19.179555+0000","last_active":"2024-04-25T18:59:18.923910+0000","last_peered":"2024-04-25T18:59:18.923910+0000","last_clean":"2024-04-25T18:59:18.923910+0000","last_became_active":"2024-04-25T18:55:19.179190+0000","last_became_peered":"2024-04-25T18:55:19.179190+0000","last_unstale":"2024-04-25T18:59:18.923910+0000","last_undegraded":"2024-04-25T18:59:18.923910+0000","last_fullsized":"2024-04-25T18:59:18.923910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:57.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132483+0000","last_change":"2024-04-25T18:55:23.219978+0000","last_active":"2024-04-25T18:59:19.132483+0000","last_peered":"2024-04-25T18:59:19.132483+0000","last_clean":"2024-04-25T18:59:19.132483+0000","last_became_active":"2024-04-25T18:55:23.219630+0000","last_became_peered":"2024-04-25T18:55:23.219630+0000","last_unstale":"2024-04-25T18:59:19.132483+0000","last_undegraded":"2024-04-25T18:59:19.132483+0000","last_fullsized":"2024-04-25T18:59:19.132483+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:31.294500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"95'39","reported_seq":509,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.741678+0000","last_change":"2024-04-25T18:55:19.164042+0000","last_active":"2024-04-25T18:59:19.741678+0000","last_peered":"2024-04-25T18:59:19.741678+0000","last_clean":"2024-04-25T18:59:19.741678+0000","last_became_active":"2024-04-25T18:55:19.163604+0000","last_became_peered":"2024-04-25T18:55:19.163604+0000","last_unstale":"2024-04-25T18:59:19.741678+0000","last_undegraded":"2024-04-25T18:59:19.741678+0000","last_fullsized":"2024-04-25T18:59:19.741678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:55.030715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71438,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":441,"num_read_kb":290,"num_write":40,"num_write_kb":88,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757632+0000","last_change":"2024-04-25T18:55:23.222702+0000","last_active":"2024-04-25T18:59:18.757632+0000","last_peered":"2024-04-25T18:59:18.757632+0000","last_clean":"2024-04-25T18:59:18.757632+0000","last_became_active":"2024-04-25T18:55:23.221849+0000","last_became_peered":"2024-04-25T18:55:23.221849+0000","last_unstale":"2024-04-25T18:59:18.757632+0000","last_undegraded":"2024-04-25T18:59:18.757632+0000","last_fullsized":"2024-04-25T18:59:18.757632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.957027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191769+0000","last_change":"2024-04-25T18:56:17.561912+0000","last_active":"2024-04-25T18:59:19.191769+0000","last_peered":"2024-04-25T18:59:19.191769+0000","last_clean":"2024-04-25T18:59:19.191769+0000","last_became_active":"2024-04-25T18:56:17.552120+0000","last_became_peered":"2024-04-25T18:56:17.552120+0000","last_unstale":"2024-04-25T18:59:19.191769+0000","last_undegraded":"2024-04-25T18:59:19.191769+0000","last_fullsized":"2024-04-25T18:59:19.191769+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:30.209802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.756960+0000","last_change":"2024-04-25T18:55:23.233074+0000","last_active":"2024-04-25T18:59:18.756960+0000","last_peered":"2024-04-25T18:59:18.756960+0000","last_clean":"2024-04-25T18:59:18.756960+0000","last_became_active":"2024-04-25T18:55:23.232751+0000","last_became_peered":"2024-04-25T18:55:23.232751+0000","last_unstale":"2024-04-25T18:59:18.756960+0000","last_undegraded":"2024-04-25T18:59:18.756960+0000","last_fullsized":"2024-04-25T18:59:18.756960+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.584929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923555+0000","last_change":"2024-04-25T18:55:19.166093+0000","last_active":"2024-04-25T18:59:18.923555+0000","last_peered":"2024-04-25T18:59:18.923555+0000","last_clean":"2024-04-25T18:59:18.923555+0000","last_became_active":"2024-04-25T18:55:19.165715+0000","last_became_peered":"2024-04-25T18:55:19.165715+0000","last_unstale":"2024-04-25T18:59:18.923555+0000","last_undegraded":"2024-04-25T18:59:18.923555+0000","last_fullsized":"2024-04-25T18:59:18.923555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:32.177943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758234+0000","last_change":"2024-04-25T18:55:23.224617+0000","last_active":"2024-04-25T18:59:18.758234+0000","last_peered":"2024-04-25T18:59:18.758234+0000","last_clean":"2024-04-25T18:59:18.758234+0000","last_became_active":"2024-04-25T18:55:23.224269+0000","last_became_peered":"2024-04-25T18:55:23.224269+0000","last_unstale":"2024-04-25T18:59:18.758234+0000","last_undegraded":"2024-04-25T18:59:18.758234+0000","last_fullsized":"2024-04-25T18:59:18.758234+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:19.079910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":251,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895148+0000","last_change":"2024-04-25T18:55:19.164971+0000","last_active":"2024-04-25T18:59:18.895148+0000","last_peered":"2024-04-25T18:59:18.895148+0000","last_clean":"2024-04-25T18:59:18.895148+0000","last_became_active":"2024-04-25T18:55:19.164579+0000","last_became_peered":"2024-04-25T18:55:19.164579+0000","last_unstale":"2024-04-25T18:59:18.895148+0000","last_undegraded":"2024-04-25T18:59:18.895148+0000","last_fullsized":"2024-04-25T18:59:18.895148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:37.419379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":218,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757934+0000","last_change":"2024-04-25T18:55:23.222160+0000","last_active":"2024-04-25T18:59:18.757934+0000","last_peered":"2024-04-25T18:59:18.757934+0000","last_clean":"2024-04-25T18:59:18.757934+0000","last_became_active":"2024-04-25T18:55:23.221812+0000","last_became_peered":"2024-04-25T18:55:23.221812+0000","last_unstale":"2024-04-25T18:59:18.757934+0000","last_undegraded":"2024-04-25T18:59:18.757934+0000","last_fullsized":"2024-04-25T18:59:18.757934+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:53:53.388764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757095+0000","last_change":"2024-04-25T18:55:19.179970+0000","last_active":"2024-04-25T18:59:18.757095+0000","last_peered":"2024-04-25T18:59:18.757095+0000","last_clean":"2024-04-25T18:59:18.757095+0000","last_became_active":"2024-04-25T18:55:19.179545+0000","last_became_peered":"2024-04-25T18:55:19.179545+0000","last_unstale":"2024-04-25T18:59:18.757095+0000","last_undegraded":"2024-04-25T18:59:18.757095+0000","last_fullsized":"2024-04-25T18:59:18.757095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:42.841661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923474+0000","last_change":"2024-04-25T18:55:23.216134+0000","last_active":"2024-04-25T18:59:18.923474+0000","last_peered":"2024-04-25T18:59:18.923474+0000","last_clean":"2024-04-25T18:59:18.923474+0000","last_became_active":"2024-04-25T18:55:23.215535+0000","last_became_peered":"2024-04-25T18:55:23.215535+0000","last_unstale":"2024-04-25T18:59:18.923474+0000","last_undegraded":"2024-04-25T18:59:18.923474+0000","last_fullsized":"2024-04-25T18:59:18.923474+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:08.042095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757699+0000","last_change":"2024-04-25T18:55:19.162437+0000","last_active":"2024-04-25T18:59:18.757699+0000","last_peered":"2024-04-25T18:59:18.757699+0000","last_clean":"2024-04-25T18:59:18.757699+0000","last_became_active":"2024-04-25T18:55:19.162005+0000","last_became_peered":"2024-04-25T18:55:19.162005+0000","last_unstale":"2024-04-25T18:59:18.757699+0000","last_undegraded":"2024-04-25T18:59:18.757699+0000","last_fullsized":"2024-04-25T18:59:18.757699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:50:53.967081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192027+0000","last_change":"2024-04-25T18:55:23.222345+0000","last_active":"2024-04-25T18:59:19.192027+0000","last_peered":"2024-04-25T18:59:19.192027+0000","last_clean":"2024-04-25T18:59:19.192027+0000","last_became_active":"2024-04-25T18:55:23.221142+0000","last_became_peered":"2024-04-25T18:55:23.221142+0000","last_unstale":"2024-04-25T18:59:19.192027+0000","last_undegraded":"2024-04-25T18:59:19.192027+0000","last_fullsized":"2024-04-25T18:59:19.192027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:53.823585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907451+0000","last_change":"2024-04-25T18:55:19.181153+0000","last_active":"2024-04-25T18:59:18.907451+0000","last_peered":"2024-04-25T18:59:18.907451+0000","last_clean":"2024-04-25T18:59:18.907451+0000","last_became_active":"2024-04-25T18:55:19.180553+0000","last_became_peered":"2024-04-25T18:55:19.180553+0000","last_unstale":"2024-04-25T18:59:18.907451+0000","last_undegraded":"2024-04-25T18:59:18.907451+0000","last_fullsized":"2024-04-25T18:59:18.907451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:07.829617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928837+0000","last_change":"2024-04-25T18:55:23.238824+0000","last_active":"2024-04-25T18:59:18.928837+0000","last_peered":"2024-04-25T18:59:18.928837+0000","last_clean":"2024-04-25T18:59:18.928837+0000","last_became_active":"2024-04-25T18:55:23.238228+0000","last_became_peered":"2024-04-25T18:55:23.238228+0000","last_unstale":"2024-04-25T18:59:18.928837+0000","last_undegraded":"2024-04-25T18:59:18.928837+0000","last_fullsized":"2024-04-25T18:59:18.928837+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:16.279254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758247+0000","last_change":"2024-04-25T18:55:19.171186+0000","last_active":"2024-04-25T18:59:18.758247+0000","last_peered":"2024-04-25T18:59:18.758247+0000","last_clean":"2024-04-25T18:59:18.758247+0000","last_became_active":"2024-04-25T18:55:19.170800+0000","last_became_peered":"2024-04-25T18:55:19.170800+0000","last_unstale":"2024-04-25T18:59:18.758247+0000","last_undegraded":"2024-04-25T18:59:18.758247+0000","last_fullsized":"2024-04-25T18:59:18.758247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.599119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928761+0000","last_change":"2024-04-25T18:55:23.236951+0000","last_active":"2024-04-25T18:59:18.928761+0000","last_peered":"2024-04-25T18:59:18.928761+0000","last_clean":"2024-04-25T18:59:18.928761+0000","last_became_active":"2024-04-25T18:55:23.236596+0000","last_became_peered":"2024-04-25T18:55:23.236596+0000","last_unstale":"2024-04-25T18:59:18.928761+0000","last_undegraded":"2024-04-25T18:59:18.928761+0000","last_fullsized":"2024-04-25T18:59:18.928761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:14.634653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923622+0000","last_change":"2024-04-25T18:55:23.211984+0000","last_active":"2024-04-25T18:59:18.923622+0000","last_peered":"2024-04-25T18:59:18.923622+0000","last_clean":"2024-04-25T18:59:18.923622+0000","last_became_active":"2024-04-25T18:55:23.211591+0000","last_became_peered":"2024-04-25T18:55:23.211591+0000","last_unstale":"2024-04-25T18:59:18.923622+0000","last_undegraded":"2024-04-25T18:59:18.923622+0000","last_fullsized":"2024-04-25T18:59:18.923622+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:09.024891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":247,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757073+0000","last_change":"2024-04-25T18:55:19.180378+0000","last_active":"2024-04-25T18:59:18.757073+0000","last_peered":"2024-04-25T18:59:18.757073+0000","last_clean":"2024-04-25T18:59:18.757073+0000","last_became_active":"2024-04-25T18:55:19.179905+0000","last_became_peered":"2024-04-25T18:55:19.179905+0000","last_unstale":"2024-04-25T18:59:18.757073+0000","last_undegraded":"2024-04-25T18:59:18.757073+0000","last_fullsized":"2024-04-25T18:59:18.757073+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:36.434477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":213,"num_read_kb":213,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895798+0000","last_change":"2024-04-25T18:55:23.231489+0000","last_active":"2024-04-25T18:59:18.895798+0000","last_peered":"2024-04-25T18:59:18.895798+0000","last_clean":"2024-04-25T18:59:18.895798+0000","last_became_active":"2024-04-25T18:55:23.231049+0000","last_became_peered":"2024-04-25T18:55:23.231049+0000","last_unstale":"2024-04-25T18:59:18.895798+0000","last_undegraded":"2024-04-25T18:59:18.895798+0000","last_fullsized":"2024-04-25T18:59:18.895798+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:25.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895192+0000","last_change":"2024-04-25T18:55:19.171694+0000","last_active":"2024-04-25T18:59:18.895192+0000","last_peered":"2024-04-25T18:59:18.895192+0000","last_clean":"2024-04-25T18:59:18.895192+0000","last_became_active":"2024-04-25T18:55:19.171296+0000","last_became_peered":"2024-04-25T18:55:19.171296+0000","last_unstale":"2024-04-25T18:59:18.895192+0000","last_undegraded":"2024-04-25T18:59:18.895192+0000","last_fullsized":"2024-04-25T18:59:18.895192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:33.216980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758066+0000","last_change":"2024-04-25T18:53:49.127506+0000","last_active":"2024-04-25T18:59:18.758066+0000","last_peered":"2024-04-25T18:59:18.758066+0000","last_clean":"2024-04-25T18:59:18.758066+0000","last_became_active":"2024-04-25T18:53:49.126974+0000","last_became_peered":"2024-04-25T18:53:49.126974+0000","last_unstale":"2024-04-25T18:59:18.758066+0000","last_undegraded":"2024-04-25T18:59:18.758066+0000","last_fullsized":"2024-04-25T18:59:18.758066+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_clean_scrub_stamp":"2024-04-25T18:50:00.765468+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.488525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757160+0000","last_change":"2024-04-25T18:55:23.211697+0000","last_active":"2024-04-25T18:59:18.757160+0000","last_peered":"2024-04-25T18:59:18.757160+0000","last_clean":"2024-04-25T18:59:18.757160+0000","last_became_active":"2024-04-25T18:55:23.211160+0000","last_became_peered":"2024-04-25T18:55:23.211160+0000","last_unstale":"2024-04-25T18:59:18.757160+0000","last_undegraded":"2024-04-25T18:59:18.757160+0000","last_fullsized":"2024-04-25T18:59:18.757160+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:03.452578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758224+0000","last_change":"2024-04-25T18:55:19.165955+0000","last_active":"2024-04-25T18:59:18.758224+0000","last_peered":"2024-04-25T18:59:18.758224+0000","last_clean":"2024-04-25T18:59:18.758224+0000","last_became_active":"2024-04-25T18:55:19.165594+0000","last_became_peered":"2024-04-25T18:55:19.165594+0000","last_unstale":"2024-04-25T18:59:18.758224+0000","last_undegraded":"2024-04-25T18:59:18.758224+0000","last_fullsized":"2024-04-25T18:59:18.758224+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:11.692374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757737+0000","last_change":"2024-04-25T18:55:19.182698+0000","last_active":"2024-04-25T18:59:18.757737+0000","last_peered":"2024-04-25T18:59:18.757737+0000","last_clean":"2024-04-25T18:59:18.757737+0000","last_became_active":"2024-04-25T18:55:19.182393+0000","last_became_peered":"2024-04-25T18:55:19.182393+0000","last_unstale":"2024-04-25T18:59:18.757737+0000","last_undegraded":"2024-04-25T18:59:18.757737+0000","last_fullsized":"2024-04-25T18:59:18.757737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.513613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757347+0000","last_change":"2024-04-25T18:55:23.223240+0000","last_active":"2024-04-25T18:59:18.757347+0000","last_peered":"2024-04-25T18:59:18.757347+0000","last_clean":"2024-04-25T18:59:18.757347+0000","last_became_active":"2024-04-25T18:55:23.218253+0000","last_became_peered":"2024-04-25T18:55:23.218253+0000","last_unstale":"2024-04-25T18:59:18.757347+0000","last_undegraded":"2024-04-25T18:59:18.757347+0000","last_fullsized":"2024-04-25T18:59:18.757347+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:40.435633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"95'49","reported_seq":101,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.076032+0000","last_change":"2024-04-25T18:55:19.168535+0000","last_active":"2024-04-25T18:59:19.076032+0000","last_peered":"2024-04-25T18:59:19.076032+0000","last_clean":"2024-04-25T18:59:19.076032+0000","last_became_active":"2024-04-25T18:55:19.166735+0000","last_became_peered":"2024-04-25T18:55:19.166735+0000","last_unstale":"2024-04-25T18:59:19.076032+0000","last_undegraded":"2024-04-25T18:59:19.076032+0000","last_fullsized":"2024-04-25T18:59:19.076032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:50:14.008290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":88,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.896087+0000","last_change":"2024-04-25T18:55:23.233989+0000","last_active":"2024-04-25T18:59:18.896087+0000","last_peered":"2024-04-25T18:59:18.896087+0000","last_clean":"2024-04-25T18:59:18.896087+0000","last_became_active":"2024-04-25T18:55:23.226867+0000","last_became_peered":"2024-04-25T18:55:23.226867+0000","last_unstale":"2024-04-25T18:59:18.896087+0000","last_undegraded":"2024-04-25T18:59:18.896087+0000","last_fullsized":"2024-04-25T18:59:18.896087+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:02.314368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895339+0000","last_change":"2024-04-25T18:55:19.870412+0000","last_active":"2024-04-25T18:59:18.895339+0000","last_peered":"2024-04-25T18:59:18.895339+0000","last_clean":"2024-04-25T18:59:18.895339+0000","last_became_active":"2024-04-25T18:55:19.869970+0000","last_became_peered":"2024-04-25T18:55:19.869970+0000","last_unstale":"2024-04-25T18:59:18.895339+0000","last_undegraded":"2024-04-25T18:59:18.895339+0000","last_fullsized":"2024-04-25T18:59:18.895339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:22.236417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193471+0000","last_change":"2024-04-25T18:55:23.239172+0000","last_active":"2024-04-25T18:59:19.193471+0000","last_peered":"2024-04-25T18:59:19.193471+0000","last_clean":"2024-04-25T18:59:19.193471+0000","last_became_active":"2024-04-25T18:55:23.238732+0000","last_became_peered":"2024-04-25T18:55:23.238732+0000","last_unstale":"2024-04-25T18:59:19.193471+0000","last_undegraded":"2024-04-25T18:59:19.193471+0000","last_fullsized":"2024-04-25T18:59:19.193471+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:27.459659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132316+0000","last_change":"2024-04-25T18:55:19.179687+0000","last_active":"2024-04-25T18:59:19.132316+0000","last_peered":"2024-04-25T18:59:19.132316+0000","last_clean":"2024-04-25T18:59:19.132316+0000","last_became_active":"2024-04-25T18:55:19.179225+0000","last_became_peered":"2024-04-25T18:55:19.179225+0000","last_unstale":"2024-04-25T18:59:19.132316+0000","last_undegraded":"2024-04-25T18:59:19.132316+0000","last_fullsized":"2024-04-25T18:59:19.132316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:16:16.266021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757162+0000","last_change":"2024-04-25T18:55:23.221379+0000","last_active":"2024-04-25T18:59:18.757162+0000","last_peered":"2024-04-25T18:59:18.757162+0000","last_clean":"2024-04-25T18:59:18.757162+0000","last_became_active":"2024-04-25T18:55:23.221037+0000","last_became_peered":"2024-04-25T18:55:23.221037+0000","last_unstale":"2024-04-25T18:59:18.757162+0000","last_undegraded":"2024-04-25T18:59:18.757162+0000","last_fullsized":"2024-04-25T18:59:18.757162+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:27.581084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895320+0000","last_change":"2024-04-25T18:55:19.171298+0000","last_active":"2024-04-25T18:59:18.895320+0000","last_peered":"2024-04-25T18:59:18.895320+0000","last_clean":"2024-04-25T18:59:18.895320+0000","last_became_active":"2024-04-25T18:55:19.170738+0000","last_became_peered":"2024-04-25T18:55:19.170738+0000","last_unstale":"2024-04-25T18:59:18.895320+0000","last_undegraded":"2024-04-25T18:59:18.895320+0000","last_fullsized":"2024-04-25T18:59:18.895320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.236331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191708+0000","last_change":"2024-04-25T18:55:23.226577+0000","last_active":"2024-04-25T18:59:19.191708+0000","last_peered":"2024-04-25T18:59:19.191708+0000","last_clean":"2024-04-25T18:59:19.191708+0000","last_became_active":"2024-04-25T18:55:23.226198+0000","last_became_peered":"2024-04-25T18:55:23.226198+0000","last_unstale":"2024-04-25T18:59:19.191708+0000","last_undegraded":"2024-04-25T18:59:19.191708+0000","last_fullsized":"2024-04-25T18:59:19.191708+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:34:59.124523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129472+0000","last_change":"2024-04-25T18:55:19.171917+0000","last_active":"2024-04-25T18:59:19.129472+0000","last_peered":"2024-04-25T18:59:19.129472+0000","last_clean":"2024-04-25T18:59:19.129472+0000","last_became_active":"2024-04-25T18:55:19.171506+0000","last_became_peered":"2024-04-25T18:55:19.171506+0000","last_unstale":"2024-04-25T18:59:19.129472+0000","last_undegraded":"2024-04-25T18:59:19.129472+0000","last_fullsized":"2024-04-25T18:59:19.129472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:53.446504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895608+0000","last_change":"2024-04-25T18:55:23.237828+0000","last_active":"2024-04-25T18:59:18.895608+0000","last_peered":"2024-04-25T18:59:18.895608+0000","last_clean":"2024-04-25T18:59:18.895608+0000","last_became_active":"2024-04-25T18:55:23.237310+0000","last_became_peered":"2024-04-25T18:55:23.237310+0000","last_unstale":"2024-04-25T18:59:18.895608+0000","last_undegraded":"2024-04-25T18:59:18.895608+0000","last_fullsized":"2024-04-25T18:59:18.895608+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:10.987032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757414+0000","last_change":"2024-04-25T18:55:19.163824+0000","last_active":"2024-04-25T18:59:18.757414+0000","last_peered":"2024-04-25T18:59:18.757414+0000","last_clean":"2024-04-25T18:59:18.757414+0000","last_became_active":"2024-04-25T18:55:19.163465+0000","last_became_peered":"2024-04-25T18:55:19.163465+0000","last_unstale":"2024-04-25T18:59:18.757414+0000","last_undegraded":"2024-04-25T18:59:18.757414+0000","last_fullsized":"2024-04-25T18:59:18.757414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:38.347100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758396+0000","last_change":"2024-04-25T18:55:23.224648+0000","last_active":"2024-04-25T18:59:18.758396+0000","last_peered":"2024-04-25T18:59:18.758396+0000","last_clean":"2024-04-25T18:59:18.758396+0000","last_became_active":"2024-04-25T18:55:23.224323+0000","last_became_peered":"2024-04-25T18:55:23.224323+0000","last_unstale":"2024-04-25T18:59:18.758396+0000","last_undegraded":"2024-04-25T18:59:18.758396+0000","last_fullsized":"2024-04-25T18:59:18.758396+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.117792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129721+0000","last_change":"2024-04-25T18:55:19.870213+0000","last_active":"2024-04-25T18:59:19.129721+0000","last_peered":"2024-04-25T18:59:19.129721+0000","last_clean":"2024-04-25T18:59:19.129721+0000","last_became_active":"2024-04-25T18:55:19.869756+0000","last_became_peered":"2024-04-25T18:55:19.869756+0000","last_unstale":"2024-04-25T18:59:19.129721+0000","last_undegraded":"2024-04-25T18:59:19.129721+0000","last_fullsized":"2024-04-25T18:59:19.129721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:44.695876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758609+0000","last_change":"2024-04-25T18:55:23.217094+0000","last_active":"2024-04-25T18:59:18.758609+0000","last_peered":"2024-04-25T18:59:18.758609+0000","last_clean":"2024-04-25T18:59:18.758609+0000","last_became_active":"2024-04-25T18:55:23.216696+0000","last_became_peered":"2024-04-25T18:55:23.216696+0000","last_unstale":"2024-04-25T18:59:18.758609+0000","last_undegraded":"2024-04-25T18:59:18.758609+0000","last_fullsized":"2024-04-25T18:59:18.758609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:07.049349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'39","reported_seq":76,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.069002+0000","last_change":"2024-04-25T18:55:19.166377+0000","last_active":"2024-04-25T18:59:19.069002+0000","last_peered":"2024-04-25T18:59:19.069002+0000","last_clean":"2024-04-25T18:59:19.069002+0000","last_became_active":"2024-04-25T18:55:19.166036+0000","last_became_peered":"2024-04-25T18:55:19.166036+0000","last_unstale":"2024-04-25T18:59:19.069002+0000","last_undegraded":"2024-04-25T18:59:19.069002+0000","last_fullsized":"2024-04-25T18:59:19.069002+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:04.761919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758305+0000","last_change":"2024-04-25T18:55:23.236893+0000","last_active":"2024-04-25T18:59:18.758305+0000","last_peered":"2024-04-25T18:59:18.758305+0000","last_clean":"2024-04-25T18:59:18.758305+0000","last_became_active":"2024-04-25T18:55:23.236488+0000","last_became_peered":"2024-04-25T18:55:23.236488+0000","last_unstale":"2024-04-25T18:59:18.758305+0000","last_undegraded":"2024-04-25T18:59:18.758305+0000","last_fullsized":"2024-04-25T18:59:18.758305+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:44.605041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757331+0000","last_change":"2024-04-25T18:55:19.175076+0000","last_active":"2024-04-25T18:59:18.757331+0000","last_peered":"2024-04-25T18:59:18.757331+0000","last_clean":"2024-04-25T18:59:18.757331+0000","last_became_active":"2024-04-25T18:55:19.174547+0000","last_became_peered":"2024-04-25T18:55:19.174547+0000","last_unstale":"2024-04-25T18:59:18.757331+0000","last_undegraded":"2024-04-25T18:59:18.757331+0000","last_fullsized":"2024-04-25T18:59:18.757331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:43.791068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758728+0000","last_change":"2024-04-25T18:55:23.219665+0000","last_active":"2024-04-25T18:59:18.758728+0000","last_peered":"2024-04-25T18:59:18.758728+0000","last_clean":"2024-04-25T18:59:18.758728+0000","last_became_active":"2024-04-25T18:55:23.219264+0000","last_became_peered":"2024-04-25T18:55:23.219264+0000","last_unstale":"2024-04-25T18:59:18.758728+0000","last_undegraded":"2024-04-25T18:59:18.758728+0000","last_fullsized":"2024-04-25T18:59:18.758728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:19.008120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191796+0000","last_change":"2024-04-25T18:55:23.220290+0000","last_active":"2024-04-25T18:59:19.191796+0000","last_peered":"2024-04-25T18:59:19.191796+0000","last_clean":"2024-04-25T18:59:19.191796+0000","last_became_active":"2024-04-25T18:55:23.219131+0000","last_became_peered":"2024-04-25T18:55:23.219131+0000","last_unstale":"2024-04-25T18:59:19.191796+0000","last_undegraded":"2024-04-25T18:59:19.191796+0000","last_fullsized":"2024-04-25T18:59:19.191796+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:25:25.678525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129447+0000","last_change":"2024-04-25T18:55:23.216415+0000","last_active":"2024-04-25T18:59:19.129447+0000","last_peered":"2024-04-25T18:59:19.129447+0000","last_clean":"2024-04-25T18:59:19.129447+0000","last_became_active":"2024-04-25T18:55:23.216066+0000","last_became_peered":"2024-04-25T18:55:23.216066+0000","last_unstale":"2024-04-25T18:59:19.129447+0000","last_undegraded":"2024-04-25T18:59:19.129447+0000","last_fullsized":"2024-04-25T18:59:19.129447+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:54.633216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129390+0000","last_change":"2024-04-25T18:55:23.234034+0000","last_active":"2024-04-25T18:59:19.129390+0000","last_peered":"2024-04-25T18:59:19.129390+0000","last_clean":"2024-04-25T18:59:19.129390+0000","last_became_active":"2024-04-25T18:55:23.233639+0000","last_became_peered":"2024-04-25T18:55:23.233639+0000","last_unstale":"2024-04-25T18:59:19.129390+0000","last_undegraded":"2024-04-25T18:59:19.129390+0000","last_fullsized":"2024-04-25T18:59:19.129390+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:57.117958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192719+0000","last_change":"2024-04-25T18:56:17.584511+0000","last_active":"2024-04-25T18:59:19.192719+0000","last_peered":"2024-04-25T18:59:19.192719+0000","last_clean":"2024-04-25T18:59:19.192719+0000","last_became_active":"2024-04-25T18:56:17.583959+0000","last_became_peered":"2024-04-25T18:56:17.583959+0000","last_unstale":"2024-04-25T18:59:19.192719+0000","last_undegraded":"2024-04-25T18:59:19.192719+0000","last_fullsized":"2024-04-25T18:59:19.192719+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.622823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757493+0000","last_change":"2024-04-25T18:55:23.238068+0000","last_active":"2024-04-25T18:59:18.757493+0000","last_peered":"2024-04-25T18:59:18.757493+0000","last_clean":"2024-04-25T18:59:18.757493+0000","last_became_active":"2024-04-25T18:55:23.237773+0000","last_became_peered":"2024-04-25T18:55:23.237773+0000","last_unstale":"2024-04-25T18:59:18.757493+0000","last_undegraded":"2024-04-25T18:59:18.757493+0000","last_fullsized":"2024-04-25T18:59:18.757493+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.688576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191981+0000","last_change":"2024-04-25T18:55:23.215463+0000","last_active":"2024-04-25T18:59:19.191981+0000","last_peered":"2024-04-25T18:59:19.191981+0000","last_clean":"2024-04-25T18:59:19.191981+0000","last_became_active":"2024-04-25T18:55:23.215068+0000","last_became_peered":"2024-04-25T18:55:23.215068+0000","last_unstale":"2024-04-25T18:59:19.191981+0000","last_undegraded":"2024-04-25T18:59:19.191981+0000","last_fullsized":"2024-04-25T18:59:19.191981+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.450828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129793+0000","last_change":"2024-04-25T18:55:23.214362+0000","last_active":"2024-04-25T18:59:19.129793+0000","last_peered":"2024-04-25T18:59:19.129793+0000","last_clean":"2024-04-25T18:59:19.129793+0000","last_became_active":"2024-04-25T18:55:23.214035+0000","last_became_peered":"2024-04-25T18:55:23.214035+0000","last_unstale":"2024-04-25T18:59:19.129793+0000","last_undegraded":"2024-04-25T18:59:19.129793+0000","last_fullsized":"2024-04-25T18:59:19.129793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:33.386859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.896015+0000","last_change":"2024-04-25T18:55:23.235709+0000","last_active":"2024-04-25T18:59:18.896015+0000","last_peered":"2024-04-25T18:59:18.896015+0000","last_clean":"2024-04-25T18:59:18.896015+0000","last_became_active":"2024-04-25T18:55:23.226089+0000","last_became_peered":"2024-04-25T18:55:23.226089+0000","last_unstale":"2024-04-25T18:59:18.896015+0000","last_undegraded":"2024-04-25T18:59:18.896015+0000","last_fullsized":"2024-04-25T18:59:18.896015+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:57.161365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.756774+0000","last_change":"2024-04-25T18:56:17.561546+0000","last_active":"2024-04-25T18:59:18.756774+0000","last_peered":"2024-04-25T18:59:18.756774+0000","last_clean":"2024-04-25T18:59:18.756774+0000","last_became_active":"2024-04-25T18:56:17.559767+0000","last_became_peered":"2024-04-25T18:56:17.559767+0000","last_unstale":"2024-04-25T18:59:18.756774+0000","last_undegraded":"2024-04-25T18:59:18.756774+0000","last_fullsized":"2024-04-25T18:59:18.756774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:59.512008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895271+0000","last_change":"2024-04-25T18:55:23.227620+0000","last_active":"2024-04-25T18:59:18.895271+0000","last_peered":"2024-04-25T18:59:18.895271+0000","last_clean":"2024-04-25T18:59:18.895271+0000","last_became_active":"2024-04-25T18:55:23.227236+0000","last_became_peered":"2024-04-25T18:55:23.227236+0000","last_unstale":"2024-04-25T18:59:18.895271+0000","last_undegraded":"2024-04-25T18:59:18.895271+0000","last_fullsized":"2024-04-25T18:59:18.895271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:32.707619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132494+0000","last_change":"2024-04-25T18:55:23.238393+0000","last_active":"2024-04-25T18:59:19.132494+0000","last_peered":"2024-04-25T18:59:19.132494+0000","last_clean":"2024-04-25T18:59:19.132494+0000","last_became_active":"2024-04-25T18:55:23.237726+0000","last_became_peered":"2024-04-25T18:55:23.237726+0000","last_unstale":"2024-04-25T18:59:19.132494+0000","last_undegraded":"2024-04-25T18:59:19.132494+0000","last_fullsized":"2024-04-25T18:59:19.132494+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:48.543014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757312+0000","last_change":"2024-04-25T18:55:23.231374+0000","last_active":"2024-04-25T18:59:18.757312+0000","last_peered":"2024-04-25T18:59:18.757312+0000","last_clean":"2024-04-25T18:59:18.757312+0000","last_became_active":"2024-04-25T18:55:23.230976+0000","last_became_peered":"2024-04-25T18:55:23.230976+0000","last_unstale":"2024-04-25T18:59:18.757312+0000","last_undegraded":"2024-04-25T18:59:18.757312+0000","last_fullsized":"2024-04-25T18:59:18.757312+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:06.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191923+0000","last_change":"2024-04-25T18:55:23.217332+0000","last_active":"2024-04-25T18:59:19.191923+0000","last_peered":"2024-04-25T18:59:19.191923+0000","last_clean":"2024-04-25T18:59:19.191923+0000","last_became_active":"2024-04-25T18:55:23.216968+0000","last_became_peered":"2024-04-25T18:55:23.216968+0000","last_unstale":"2024-04-25T18:59:19.191923+0000","last_undegraded":"2024-04-25T18:59:19.191923+0000","last_fullsized":"2024-04-25T18:59:19.191923+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:31.000083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":601710,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1410,"num_read_kb":819,"num_write":299,"num_write_kb":1446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2039808,"data_stored":1805130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252927,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":507776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514566,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37180,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707972,"statfs":{"total":95995035648,"available":95956963328,"internally_reserved":0,"allocated":1888256,"data_stored":1302793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710793,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36760,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708392,"statfs":{"total":95995035648,"available":95957393408,"internally_reserved":0,"allocated":1454080,"data_stored":901194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972433,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28440,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716712,"statfs":{"total":95995035648,"available":95965913088,"internally_reserved":0,"allocated":1454080,"data_stored":901114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234070,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36956,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708196,"statfs":{"total":95995035648,"available":95957192704,"internally_reserved":0,"allocated":1654784,"data_stored":1100501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430297,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":507367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1122304,"data_stored":578598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738491,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36364,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708788,"statfs":{"total":95995035648,"available":95957798912,"internally_reserved":0,"allocated":1052672,"data_stored":499653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1118208,"data_stored":565306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117099,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36892,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708260,"statfs":{"total":95995035648,"available":95957258240,"internally_reserved":0,"allocated":1593344,"data_stored":1032494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":508336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691936,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1671168,"data_stored":1100165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":71438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":74068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":249856,"data_stored":205148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:59:26.216 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:59:26.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:26 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:26 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:59:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:59:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: Updating smithi062:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:59:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T18:59:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: Updating smithi079:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:59:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T18:59:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: Updating smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:59:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: Updating smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:59:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:26.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:26 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:26.685 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:27 smithi045 ceph-mon[27776]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:59:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:27 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:27 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:27 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:27.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:27 smithi062 ceph-mon[32844]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:59:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:27 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:27 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:27 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:27.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:27 smithi079 ceph-mon[33002]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-25T18:59:27.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:27 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:27.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:27 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:27.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:27 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:28.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:28 smithi045 ceph-mon[27776]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:59:28.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:28 smithi045 ceph-mon[27776]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-04-25T18:59:28.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:28 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:59:28.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:28 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:59:28.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:28 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:59:28.332 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:28 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:28 smithi062 ceph-mon[32844]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:59:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:28 smithi062 ceph-mon[32844]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-04-25T18:59:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:28 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:59:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:28 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:59:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:28 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:59:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:28 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:28 smithi079 ceph-mon[33002]: Updating smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T18:59:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:28 smithi079 ceph-mon[33002]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 76 op/s 2024-04-25T18:59:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:28 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T18:59:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:28 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T18:59:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:28 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T18:59:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:28 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T18:59:28.847 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.846+0000 7f6482400700 1 -- 172.21.15.45:0/3073377398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f647c109fa0 msgr2=0x7f647c111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.846+0000 7f6482400700 1 --2- 172.21.15.45:0/3073377398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f647c109fa0 0x7f647c111b70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6474009fc0 tx=0x7f647400b330 comp rx=0 tx=0).stop 2024-04-25T18:59:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.846+0000 7f6482400700 1 -- 172.21.15.45:0/3073377398 shutdown_connections 2024-04-25T18:59:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.846+0000 7f6482400700 1 --2- 172.21.15.45:0/3073377398 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c104ec0 0x7f647c1052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.846+0000 7f6482400700 1 --2- 172.21.15.45:0/3073377398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f647c109fa0 0x7f647c111b70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.846+0000 7f6482400700 1 --2- 172.21.15.45:0/3073377398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f647c105870 0x7f647c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.846+0000 7f6482400700 1 -- 172.21.15.45:0/3073377398 >> 172.21.15.45:0/3073377398 conn(0x7f647c100750 msgr2=0x7f647c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.846+0000 7f6482400700 1 -- 172.21.15.45:0/3073377398 shutdown_connections 2024-04-25T18:59:28.848 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.846+0000 7f6482400700 1 -- 172.21.15.45:0/3073377398 wait complete. 2024-04-25T18:59:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.847+0000 7f6482400700 1 Processor -- start 2024-04-25T18:59:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f6482400700 1 -- start start 2024-04-25T18:59:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f6482400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f647c104ec0 0x7f647c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:28.849 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f6482400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c105870 0x7f647c19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f6482400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f647c109fa0 0x7f647c19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f6482400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f647c114e30 con 0x7f647c109fa0 2024-04-25T18:59:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f6482400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f647c114cb0 con 0x7f647c105870 2024-04-25T18:59:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f6482400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f647c114fb0 con 0x7f647c104ec0 2024-04-25T18:59:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f647b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f647c104ec0 0x7f647c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f647b400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f647c104ec0 0x7f647c19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:37662/0 (socket says 172.21.15.45:37662) 2024-04-25T18:59:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f647b400700 1 -- 172.21.15.45:0/43597319 learned_addr learned my addr 172.21.15.45:0/43597319 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.848+0000 7f647aa00700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c105870 0x7f647c19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:28.850 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.849+0000 7f647be00700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f647c109fa0 0x7f647c19fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.849+0000 7f647b400700 1 -- 172.21.15.45:0/43597319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c105870 msgr2=0x7f647c19b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.849+0000 7f647b400700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c105870 0x7f647c19b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.849+0000 7f647b400700 1 -- 172.21.15.45:0/43597319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f647c109fa0 msgr2=0x7f647c19fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.849+0000 7f647b400700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f647c109fa0 0x7f647c19fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.849+0000 7f647b400700 1 -- 172.21.15.45:0/43597319 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c1a02b0 con 0x7f647c104ec0 2024-04-25T18:59:28.851 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.849+0000 7f647b400700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f647c104ec0 0x7f647c19b2a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f646c00e9a0 tx=0x7f646c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.850+0000 7f6473e00700 1 -- 172.21.15.45:0/43597319 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c00ca10 con 0x7f647c104ec0 2024-04-25T18:59:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.850+0000 7f6473e00700 1 -- 172.21.15.45:0/43597319 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f646c00cbb0 con 0x7f647c104ec0 2024-04-25T18:59:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.850+0000 7f6482400700 1 -- 172.21.15.45:0/43597319 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647c1a05a0 con 0x7f647c104ec0 2024-04-25T18:59:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.850+0000 7f6473e00700 1 -- 172.21.15.45:0/43597319 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c018680 con 0x7f647c104ec0 2024-04-25T18:59:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.850+0000 7f6482400700 1 -- 172.21.15.45:0/43597319 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647c1a7e20 con 0x7f647c104ec0 2024-04-25T18:59:28.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.851+0000 7f6482400700 1 -- 172.21.15.45:0/43597319 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647c069730 con 0x7f647c104ec0 2024-04-25T18:59:28.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.855+0000 7f6473e00700 1 -- 172.21.15.45:0/43597319 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f646c010430 con 0x7f647c104ec0 2024-04-25T18:59:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.856+0000 7f6473e00700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f64600a3ed0 0x7f64600a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.856+0000 7f6473e00700 1 -- 172.21.15.45:0/43597319 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f646c0d0fc0 con 0x7f647c104ec0 2024-04-25T18:59:28.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.856+0000 7f647aa00700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f64600a3ed0 0x7f64600a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.857+0000 7f6473e00700 1 -- 172.21.15.45:0/43597319 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f646c09d3b0 con 0x7f647c104ec0 2024-04-25T18:59:28.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:28.858+0000 7f647aa00700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f64600a3ed0 0x7f64600a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6468009750 tx=0x7f6468009310 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:29.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.084+0000 7f6482400700 1 -- 172.21.15.45:0/43597319 --> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f647c062e80 con 0x7f64600a3ed0 2024-04-25T18:59:29.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.093+0000 7f6473e00700 1 -- 172.21.15.45:0/43597319 <== mgr.25049 v2:172.21.15.45:6800/1345298339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257771 (secure 0 0 0) 0x7f647c062e80 con 0x7f64600a3ed0 2024-04-25T18:59:29.100 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:59:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.099+0000 7f6482400700 1 -- 172.21.15.45:0/43597319 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f64600a3ed0 msgr2=0x7f64600a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.099+0000 7f6482400700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f64600a3ed0 0x7f64600a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6468009750 tx=0x7f6468009310 comp rx=0 tx=0).stop 2024-04-25T18:59:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.099+0000 7f6482400700 1 -- 172.21.15.45:0/43597319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f647c104ec0 msgr2=0x7f647c19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.099+0000 7f6482400700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f647c104ec0 0x7f647c19b2a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f646c00e9a0 tx=0x7f646c00ecb0 comp rx=0 tx=0).stop 2024-04-25T18:59:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.099+0000 7f6482400700 1 -- 172.21.15.45:0/43597319 shutdown_connections 2024-04-25T18:59:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.099+0000 7f6482400700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f64600a3ed0 0x7f64600a6390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.099+0000 7f6482400700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f647c109fa0 0x7f647c19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.099+0000 7f6482400700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f647c104ec0 0x7f647c19b2a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.099+0000 7f6482400700 1 --2- 172.21.15.45:0/43597319 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f647c105870 0x7f647c19b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:29.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.099+0000 7f6482400700 1 -- 172.21.15.45:0/43597319 >> 172.21.15.45:0/43597319 conn(0x7f647c100750 msgr2=0x7f647c101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.101+0000 7f6482400700 1 -- 172.21.15.45:0/43597319 shutdown_connections 2024-04-25T18:59:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:29.101+0000 7f6482400700 1 -- 172.21.15.45:0/43597319 wait complete. 2024-04-25T18:59:29.103 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T18:59:30.228 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-25T18:59:28.778392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391855,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1931,"num_read_kb":1397,"num_write":632,"num_write_kb":5305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":966,"ondisk_log_size":966,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":395176,"kb_used_data":16524,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124546648,"statfs":{"total":1151940427776,"available":1151535767552,"internally_reserved":0,"allocated":16920576,"data_stored":10261485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199543,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":385,"num_read_kb":459,"num_write":150,"num_write_kb":919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.958276"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192112+0000","last_change":"2024-04-25T18:55:23.241030+0000","last_active":"2024-04-25T18:59:19.192112+0000","last_peered":"2024-04-25T18:59:19.192112+0000","last_clean":"2024-04-25T18:59:19.192112+0000","last_became_active":"2024-04-25T18:55:23.240724+0000","last_became_peered":"2024-04-25T18:55:23.240724+0000","last_unstale":"2024-04-25T18:59:19.192112+0000","last_undegraded":"2024-04-25T18:59:19.192112+0000","last_fullsized":"2024-04-25T18:59:19.192112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:37.995475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193313+0000","last_change":"2024-04-25T18:55:23.229875+0000","last_active":"2024-04-25T18:59:19.193313+0000","last_peered":"2024-04-25T18:59:19.193313+0000","last_clean":"2024-04-25T18:59:19.193313+0000","last_became_active":"2024-04-25T18:55:23.229640+0000","last_became_peered":"2024-04-25T18:55:23.229640+0000","last_unstale":"2024-04-25T18:59:19.193313+0000","last_undegraded":"2024-04-25T18:59:19.193313+0000","last_fullsized":"2024-04-25T18:59:19.193313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:10.262376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193164+0000","last_change":"2024-04-25T18:55:23.229821+0000","last_active":"2024-04-25T18:59:19.193164+0000","last_peered":"2024-04-25T18:59:19.193164+0000","last_clean":"2024-04-25T18:59:19.193164+0000","last_became_active":"2024-04-25T18:55:23.229414+0000","last_became_peered":"2024-04-25T18:55:23.229414+0000","last_unstale":"2024-04-25T18:59:19.193164+0000","last_undegraded":"2024-04-25T18:59:19.193164+0000","last_fullsized":"2024-04-25T18:59:19.193164+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:56.063786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907514+0000","last_change":"2024-04-25T18:56:17.560167+0000","last_active":"2024-04-25T18:59:18.907514+0000","last_peered":"2024-04-25T18:59:18.907514+0000","last_clean":"2024-04-25T18:59:18.907514+0000","last_became_active":"2024-04-25T18:56:17.558466+0000","last_became_peered":"2024-04-25T18:56:17.558466+0000","last_unstale":"2024-04-25T18:59:18.907514+0000","last_undegraded":"2024-04-25T18:59:18.907514+0000","last_fullsized":"2024-04-25T18:59:18.907514+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:22.371076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757484+0000","last_change":"2024-04-25T18:55:23.233512+0000","last_active":"2024-04-25T18:59:18.757484+0000","last_peered":"2024-04-25T18:59:18.757484+0000","last_clean":"2024-04-25T18:59:18.757484+0000","last_became_active":"2024-04-25T18:55:23.233158+0000","last_became_peered":"2024-04-25T18:55:23.233158+0000","last_unstale":"2024-04-25T18:59:18.757484+0000","last_undegraded":"2024-04-25T18:59:18.757484+0000","last_fullsized":"2024-04-25T18:59:18.757484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:44.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928760+0000","last_change":"2024-04-25T18:55:23.225921+0000","last_active":"2024-04-25T18:59:18.928760+0000","last_peered":"2024-04-25T18:59:18.928760+0000","last_clean":"2024-04-25T18:59:18.928760+0000","last_became_active":"2024-04-25T18:55:23.225547+0000","last_became_peered":"2024-04-25T18:55:23.225547+0000","last_unstale":"2024-04-25T18:59:18.928760+0000","last_undegraded":"2024-04-25T18:59:18.928760+0000","last_fullsized":"2024-04-25T18:59:18.928760+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:29.828394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758001+0000","last_change":"2024-04-25T18:55:23.217111+0000","last_active":"2024-04-25T18:59:18.758001+0000","last_peered":"2024-04-25T18:59:18.758001+0000","last_clean":"2024-04-25T18:59:18.758001+0000","last_became_active":"2024-04-25T18:55:23.216731+0000","last_became_peered":"2024-04-25T18:55:23.216731+0000","last_unstale":"2024-04-25T18:59:18.758001+0000","last_undegraded":"2024-04-25T18:59:18.758001+0000","last_fullsized":"2024-04-25T18:59:18.758001+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:17.931928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907086+0000","last_change":"2024-04-25T18:56:17.555014+0000","last_active":"2024-04-25T18:59:18.907086+0000","last_peered":"2024-04-25T18:59:18.907086+0000","last_clean":"2024-04-25T18:59:18.907086+0000","last_became_active":"2024-04-25T18:56:17.553863+0000","last_became_peered":"2024-04-25T18:56:17.553863+0000","last_unstale":"2024-04-25T18:59:18.907086+0000","last_undegraded":"2024-04-25T18:59:18.907086+0000","last_fullsized":"2024-04-25T18:59:18.907086+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.240754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132827+0000","last_change":"2024-04-25T18:55:23.218197+0000","last_active":"2024-04-25T18:59:19.132827+0000","last_peered":"2024-04-25T18:59:19.132827+0000","last_clean":"2024-04-25T18:59:19.132827+0000","last_became_active":"2024-04-25T18:55:23.217665+0000","last_became_peered":"2024-04-25T18:55:23.217665+0000","last_unstale":"2024-04-25T18:59:19.132827+0000","last_undegraded":"2024-04-25T18:59:19.132827+0000","last_fullsized":"2024-04-25T18:59:19.132827+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:11.829991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.929205+0000","last_change":"2024-04-25T18:55:23.221533+0000","last_active":"2024-04-25T18:59:18.929205+0000","last_peered":"2024-04-25T18:59:18.929205+0000","last_clean":"2024-04-25T18:59:18.929205+0000","last_became_active":"2024-04-25T18:55:23.220711+0000","last_became_peered":"2024-04-25T18:55:23.220711+0000","last_unstale":"2024-04-25T18:59:18.929205+0000","last_undegraded":"2024-04-25T18:59:18.929205+0000","last_fullsized":"2024-04-25T18:59:18.929205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:45.185166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758645+0000","last_change":"2024-04-25T18:55:23.221329+0000","last_active":"2024-04-25T18:59:18.758645+0000","last_peered":"2024-04-25T18:59:18.758645+0000","last_clean":"2024-04-25T18:59:18.758645+0000","last_became_active":"2024-04-25T18:55:23.220156+0000","last_became_peered":"2024-04-25T18:55:23.220156+0000","last_unstale":"2024-04-25T18:59:18.758645+0000","last_undegraded":"2024-04-25T18:59:18.758645+0000","last_fullsized":"2024-04-25T18:59:18.758645+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:46.981199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907481+0000","last_change":"2024-04-25T18:56:17.558018+0000","last_active":"2024-04-25T18:59:18.907481+0000","last_peered":"2024-04-25T18:59:18.907481+0000","last_clean":"2024-04-25T18:59:18.907481+0000","last_became_active":"2024-04-25T18:56:17.556229+0000","last_became_peered":"2024-04-25T18:56:17.556229+0000","last_unstale":"2024-04-25T18:59:18.907481+0000","last_undegraded":"2024-04-25T18:59:18.907481+0000","last_fullsized":"2024-04-25T18:59:18.907481+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:50.949895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757146+0000","last_change":"2024-04-25T18:55:23.227474+0000","last_active":"2024-04-25T18:59:18.757146+0000","last_peered":"2024-04-25T18:59:18.757146+0000","last_clean":"2024-04-25T18:59:18.757146+0000","last_became_active":"2024-04-25T18:55:23.227068+0000","last_became_peered":"2024-04-25T18:55:23.227068+0000","last_unstale":"2024-04-25T18:59:18.757146+0000","last_undegraded":"2024-04-25T18:59:18.757146+0000","last_fullsized":"2024-04-25T18:59:18.757146+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:13.884833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192323+0000","last_change":"2024-04-25T18:55:23.230572+0000","last_active":"2024-04-25T18:59:19.192323+0000","last_peered":"2024-04-25T18:59:19.192323+0000","last_clean":"2024-04-25T18:59:19.192323+0000","last_became_active":"2024-04-25T18:55:23.229756+0000","last_became_peered":"2024-04-25T18:55:23.229756+0000","last_unstale":"2024-04-25T18:59:19.192323+0000","last_undegraded":"2024-04-25T18:59:19.192323+0000","last_fullsized":"2024-04-25T18:59:19.192323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:15.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192637+0000","last_change":"2024-04-25T18:55:23.227913+0000","last_active":"2024-04-25T18:59:19.192637+0000","last_peered":"2024-04-25T18:59:19.192637+0000","last_clean":"2024-04-25T18:59:19.192637+0000","last_became_active":"2024-04-25T18:55:23.227598+0000","last_became_peered":"2024-04-25T18:55:23.227598+0000","last_unstale":"2024-04-25T18:59:19.192637+0000","last_undegraded":"2024-04-25T18:59:19.192637+0000","last_fullsized":"2024-04-25T18:59:19.192637+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:33.555741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132304+0000","last_change":"2024-04-25T18:55:23.230241+0000","last_active":"2024-04-25T18:59:19.132304+0000","last_peered":"2024-04-25T18:59:19.132304+0000","last_clean":"2024-04-25T18:59:19.132304+0000","last_became_active":"2024-04-25T18:55:23.229898+0000","last_became_peered":"2024-04-25T18:55:23.229898+0000","last_unstale":"2024-04-25T18:59:19.132304+0000","last_undegraded":"2024-04-25T18:59:19.132304+0000","last_fullsized":"2024-04-25T18:59:19.132304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:27.278475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192205+0000","last_change":"2024-04-25T18:55:23.240978+0000","last_active":"2024-04-25T18:59:19.192205+0000","last_peered":"2024-04-25T18:59:19.192205+0000","last_clean":"2024-04-25T18:59:19.192205+0000","last_became_active":"2024-04-25T18:55:23.240702+0000","last_became_peered":"2024-04-25T18:55:23.240702+0000","last_unstale":"2024-04-25T18:59:19.192205+0000","last_undegraded":"2024-04-25T18:59:19.192205+0000","last_fullsized":"2024-04-25T18:59:19.192205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:06.053688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907730+0000","last_change":"2024-04-25T18:55:23.232116+0000","last_active":"2024-04-25T18:59:18.907730+0000","last_peered":"2024-04-25T18:59:18.907730+0000","last_clean":"2024-04-25T18:59:18.907730+0000","last_became_active":"2024-04-25T18:55:23.231586+0000","last_became_peered":"2024-04-25T18:55:23.231586+0000","last_unstale":"2024-04-25T18:59:18.907730+0000","last_undegraded":"2024-04-25T18:59:18.907730+0000","last_fullsized":"2024-04-25T18:59:18.907730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:32.640823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129676+0000","last_change":"2024-04-25T18:55:23.224519+0000","last_active":"2024-04-25T18:59:19.129676+0000","last_peered":"2024-04-25T18:59:19.129676+0000","last_clean":"2024-04-25T18:59:19.129676+0000","last_became_active":"2024-04-25T18:55:23.224145+0000","last_became_peered":"2024-04-25T18:55:23.224145+0000","last_unstale":"2024-04-25T18:59:19.129676+0000","last_undegraded":"2024-04-25T18:59:19.129676+0000","last_fullsized":"2024-04-25T18:59:19.129676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:03.340162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907818+0000","last_change":"2024-04-25T18:55:19.164977+0000","last_active":"2024-04-25T18:59:18.907818+0000","last_peered":"2024-04-25T18:59:18.907818+0000","last_clean":"2024-04-25T18:59:18.907818+0000","last_became_active":"2024-04-25T18:55:19.164316+0000","last_became_peered":"2024-04-25T18:55:19.164316+0000","last_unstale":"2024-04-25T18:59:18.907818+0000","last_undegraded":"2024-04-25T18:59:18.907818+0000","last_fullsized":"2024-04-25T18:59:18.907818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:08.031680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895576+0000","last_change":"2024-04-25T18:56:17.559746+0000","last_active":"2024-04-25T18:59:18.895576+0000","last_peered":"2024-04-25T18:59:18.895576+0000","last_clean":"2024-04-25T18:59:18.895576+0000","last_became_active":"2024-04-25T18:56:17.558761+0000","last_became_peered":"2024-04-25T18:56:17.558761+0000","last_unstale":"2024-04-25T18:59:18.895576+0000","last_undegraded":"2024-04-25T18:59:18.895576+0000","last_fullsized":"2024-04-25T18:59:18.895576+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:43.518760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895745+0000","last_change":"2024-04-25T18:55:19.179943+0000","last_active":"2024-04-25T18:59:18.895745+0000","last_peered":"2024-04-25T18:59:18.895745+0000","last_clean":"2024-04-25T18:59:18.895745+0000","last_became_active":"2024-04-25T18:55:19.179519+0000","last_became_peered":"2024-04-25T18:55:19.179519+0000","last_unstale":"2024-04-25T18:59:18.895745+0000","last_undegraded":"2024-04-25T18:59:18.895745+0000","last_fullsized":"2024-04-25T18:59:18.895745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:46.365187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193409+0000","last_change":"2024-04-25T18:55:23.237965+0000","last_active":"2024-04-25T18:59:19.193409+0000","last_peered":"2024-04-25T18:59:19.193409+0000","last_clean":"2024-04-25T18:59:19.193409+0000","last_became_active":"2024-04-25T18:55:23.237520+0000","last_became_peered":"2024-04-25T18:55:23.237520+0000","last_unstale":"2024-04-25T18:59:19.193409+0000","last_undegraded":"2024-04-25T18:59:19.193409+0000","last_fullsized":"2024-04-25T18:59:19.193409+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:03.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757872+0000","last_change":"2024-04-25T18:55:19.163965+0000","last_active":"2024-04-25T18:59:18.757872+0000","last_peered":"2024-04-25T18:59:18.757872+0000","last_clean":"2024-04-25T18:59:18.757872+0000","last_became_active":"2024-04-25T18:55:19.163390+0000","last_became_peered":"2024-04-25T18:55:19.163390+0000","last_unstale":"2024-04-25T18:59:18.757872+0000","last_undegraded":"2024-04-25T18:59:18.757872+0000","last_fullsized":"2024-04-25T18:59:18.757872+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:54.183730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132443+0000","last_change":"2024-04-25T18:55:23.227124+0000","last_active":"2024-04-25T18:59:19.132443+0000","last_peered":"2024-04-25T18:59:19.132443+0000","last_clean":"2024-04-25T18:59:19.132443+0000","last_became_active":"2024-04-25T18:55:23.226383+0000","last_became_peered":"2024-04-25T18:55:23.226383+0000","last_unstale":"2024-04-25T18:59:19.132443+0000","last_undegraded":"2024-04-25T18:59:19.132443+0000","last_fullsized":"2024-04-25T18:59:19.132443+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:22.011212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191814+0000","last_change":"2024-04-25T18:56:17.544303+0000","last_active":"2024-04-25T18:59:19.191814+0000","last_peered":"2024-04-25T18:59:19.191814+0000","last_clean":"2024-04-25T18:59:19.191814+0000","last_became_active":"2024-04-25T18:56:17.542002+0000","last_became_peered":"2024-04-25T18:56:17.542002+0000","last_unstale":"2024-04-25T18:59:19.191814+0000","last_undegraded":"2024-04-25T18:59:19.191814+0000","last_fullsized":"2024-04-25T18:59:19.191814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:23.848398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923703+0000","last_change":"2024-04-25T18:55:23.217703+0000","last_active":"2024-04-25T18:59:18.923703+0000","last_peered":"2024-04-25T18:59:18.923703+0000","last_clean":"2024-04-25T18:59:18.923703+0000","last_became_active":"2024-04-25T18:55:23.216767+0000","last_became_peered":"2024-04-25T18:55:23.216767+0000","last_unstale":"2024-04-25T18:59:18.923703+0000","last_undegraded":"2024-04-25T18:59:18.923703+0000","last_fullsized":"2024-04-25T18:59:18.923703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:17.779356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":262,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:23.747212+0000","last_change":"2024-04-25T18:55:19.164199+0000","last_active":"2024-04-25T18:59:23.747212+0000","last_peered":"2024-04-25T18:59:23.747212+0000","last_clean":"2024-04-25T18:59:23.747212+0000","last_became_active":"2024-04-25T18:55:19.163849+0000","last_became_peered":"2024-04-25T18:55:19.163849+0000","last_unstale":"2024-04-25T18:59:23.747212+0000","last_undegraded":"2024-04-25T18:59:23.747212+0000","last_fullsized":"2024-04-25T18:59:23.747212+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:25.234229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":224,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757254+0000","last_change":"2024-04-25T18:55:23.216916+0000","last_active":"2024-04-25T18:59:18.757254+0000","last_peered":"2024-04-25T18:59:18.757254+0000","last_clean":"2024-04-25T18:59:18.757254+0000","last_became_active":"2024-04-25T18:55:23.216249+0000","last_became_peered":"2024-04-25T18:55:23.216249+0000","last_unstale":"2024-04-25T18:59:18.757254+0000","last_undegraded":"2024-04-25T18:59:18.757254+0000","last_fullsized":"2024-04-25T18:59:18.757254+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:21.522318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928712+0000","last_change":"2024-04-25T18:55:19.171004+0000","last_active":"2024-04-25T18:59:18.928712+0000","last_peered":"2024-04-25T18:59:18.928712+0000","last_clean":"2024-04-25T18:59:18.928712+0000","last_became_active":"2024-04-25T18:55:19.170570+0000","last_became_peered":"2024-04-25T18:55:19.170570+0000","last_unstale":"2024-04-25T18:59:18.928712+0000","last_undegraded":"2024-04-25T18:59:18.928712+0000","last_fullsized":"2024-04-25T18:59:18.928712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:15:24.954033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923953+0000","last_change":"2024-04-25T18:56:17.552615+0000","last_active":"2024-04-25T18:59:18.923953+0000","last_peered":"2024-04-25T18:59:18.923953+0000","last_clean":"2024-04-25T18:59:18.923953+0000","last_became_active":"2024-04-25T18:56:17.552154+0000","last_became_peered":"2024-04-25T18:56:17.552154+0000","last_unstale":"2024-04-25T18:59:18.923953+0000","last_undegraded":"2024-04-25T18:59:18.923953+0000","last_fullsized":"2024-04-25T18:59:18.923953+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:01.199859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757300+0000","last_change":"2024-04-25T18:55:19.173398+0000","last_active":"2024-04-25T18:59:18.757300+0000","last_peered":"2024-04-25T18:59:18.757300+0000","last_clean":"2024-04-25T18:59:18.757300+0000","last_became_active":"2024-04-25T18:55:19.171477+0000","last_became_peered":"2024-04-25T18:55:19.171477+0000","last_unstale":"2024-04-25T18:59:18.757300+0000","last_undegraded":"2024-04-25T18:59:18.757300+0000","last_fullsized":"2024-04-25T18:59:18.757300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:35.230596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895264+0000","last_change":"2024-04-25T18:55:23.235209+0000","last_active":"2024-04-25T18:59:18.895264+0000","last_peered":"2024-04-25T18:59:18.895264+0000","last_clean":"2024-04-25T18:59:18.895264+0000","last_became_active":"2024-04-25T18:55:23.231898+0000","last_became_peered":"2024-04-25T18:55:23.231898+0000","last_unstale":"2024-04-25T18:59:18.895264+0000","last_undegraded":"2024-04-25T18:59:18.895264+0000","last_fullsized":"2024-04-25T18:59:18.895264+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.521282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923448+0000","last_change":"2024-04-25T18:55:19.163780+0000","last_active":"2024-04-25T18:59:18.923448+0000","last_peered":"2024-04-25T18:59:18.923448+0000","last_clean":"2024-04-25T18:59:18.923448+0000","last_became_active":"2024-04-25T18:55:19.163197+0000","last_became_peered":"2024-04-25T18:55:19.163197+0000","last_unstale":"2024-04-25T18:59:18.923448+0000","last_undegraded":"2024-04-25T18:59:18.923448+0000","last_fullsized":"2024-04-25T18:59:18.923448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:02:26.519160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907206+0000","last_change":"2024-04-25T18:56:17.556804+0000","last_active":"2024-04-25T18:59:18.907206+0000","last_peered":"2024-04-25T18:59:18.907206+0000","last_clean":"2024-04-25T18:59:18.907206+0000","last_became_active":"2024-04-25T18:56:17.554885+0000","last_became_peered":"2024-04-25T18:56:17.554885+0000","last_unstale":"2024-04-25T18:59:18.907206+0000","last_undegraded":"2024-04-25T18:59:18.907206+0000","last_fullsized":"2024-04-25T18:59:18.907206+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:36.316156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":253,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:23.746746+0000","last_change":"2024-04-25T18:55:19.170565+0000","last_active":"2024-04-25T18:59:23.746746+0000","last_peered":"2024-04-25T18:59:23.746746+0000","last_clean":"2024-04-25T18:59:23.746746+0000","last_became_active":"2024-04-25T18:55:19.170162+0000","last_became_peered":"2024-04-25T18:55:19.170162+0000","last_unstale":"2024-04-25T18:59:23.746746+0000","last_undegraded":"2024-04-25T18:59:23.746746+0000","last_fullsized":"2024-04-25T18:59:23.746746+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:36.606661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":219,"num_read_kb":219,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132749+0000","last_change":"2024-04-25T18:55:23.227436+0000","last_active":"2024-04-25T18:59:19.132749+0000","last_peered":"2024-04-25T18:59:19.132749+0000","last_clean":"2024-04-25T18:59:19.132749+0000","last_became_active":"2024-04-25T18:55:23.227044+0000","last_became_peered":"2024-04-25T18:55:23.227044+0000","last_unstale":"2024-04-25T18:59:19.132749+0000","last_undegraded":"2024-04-25T18:59:19.132749+0000","last_fullsized":"2024-04-25T18:59:19.132749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:34.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758348+0000","last_change":"2024-04-25T18:55:19.167514+0000","last_active":"2024-04-25T18:59:18.758348+0000","last_peered":"2024-04-25T18:59:18.758348+0000","last_clean":"2024-04-25T18:59:18.758348+0000","last_became_active":"2024-04-25T18:55:19.167102+0000","last_became_peered":"2024-04-25T18:55:19.167102+0000","last_unstale":"2024-04-25T18:59:18.758348+0000","last_undegraded":"2024-04-25T18:59:18.758348+0000","last_fullsized":"2024-04-25T18:59:18.758348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:41.472274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193323+0000","last_change":"2024-04-25T18:55:23.237012+0000","last_active":"2024-04-25T18:59:19.193323+0000","last_peered":"2024-04-25T18:59:19.193323+0000","last_clean":"2024-04-25T18:59:19.193323+0000","last_became_active":"2024-04-25T18:55:23.230263+0000","last_became_peered":"2024-04-25T18:55:23.230263+0000","last_unstale":"2024-04-25T18:59:19.193323+0000","last_undegraded":"2024-04-25T18:59:19.193323+0000","last_fullsized":"2024-04-25T18:59:19.193323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:30.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923910+0000","last_change":"2024-04-25T18:55:19.179555+0000","last_active":"2024-04-25T18:59:18.923910+0000","last_peered":"2024-04-25T18:59:18.923910+0000","last_clean":"2024-04-25T18:59:18.923910+0000","last_became_active":"2024-04-25T18:55:19.179190+0000","last_became_peered":"2024-04-25T18:55:19.179190+0000","last_unstale":"2024-04-25T18:59:18.923910+0000","last_undegraded":"2024-04-25T18:59:18.923910+0000","last_fullsized":"2024-04-25T18:59:18.923910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:57.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132483+0000","last_change":"2024-04-25T18:55:23.219978+0000","last_active":"2024-04-25T18:59:19.132483+0000","last_peered":"2024-04-25T18:59:19.132483+0000","last_clean":"2024-04-25T18:59:19.132483+0000","last_became_active":"2024-04-25T18:55:23.219630+0000","last_became_peered":"2024-04-25T18:55:23.219630+0000","last_unstale":"2024-04-25T18:59:19.132483+0000","last_undegraded":"2024-04-25T18:59:19.132483+0000","last_fullsized":"2024-04-25T18:59:19.132483+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:31.294500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":522,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:24.748137+0000","last_change":"2024-04-25T18:55:19.164042+0000","last_active":"2024-04-25T18:59:24.748137+0000","last_peered":"2024-04-25T18:59:24.748137+0000","last_clean":"2024-04-25T18:59:24.748137+0000","last_became_active":"2024-04-25T18:55:19.163604+0000","last_became_peered":"2024-04-25T18:55:19.163604+0000","last_unstale":"2024-04-25T18:59:24.748137+0000","last_undegraded":"2024-04-25T18:59:24.748137+0000","last_fullsized":"2024-04-25T18:59:24.748137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:55.030715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":74349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":452,"num_read_kb":298,"num_write":42,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757632+0000","last_change":"2024-04-25T18:55:23.222702+0000","last_active":"2024-04-25T18:59:18.757632+0000","last_peered":"2024-04-25T18:59:18.757632+0000","last_clean":"2024-04-25T18:59:18.757632+0000","last_became_active":"2024-04-25T18:55:23.221849+0000","last_became_peered":"2024-04-25T18:55:23.221849+0000","last_unstale":"2024-04-25T18:59:18.757632+0000","last_undegraded":"2024-04-25T18:59:18.757632+0000","last_fullsized":"2024-04-25T18:59:18.757632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.957027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191769+0000","last_change":"2024-04-25T18:56:17.561912+0000","last_active":"2024-04-25T18:59:19.191769+0000","last_peered":"2024-04-25T18:59:19.191769+0000","last_clean":"2024-04-25T18:59:19.191769+0000","last_became_active":"2024-04-25T18:56:17.552120+0000","last_became_peered":"2024-04-25T18:56:17.552120+0000","last_unstale":"2024-04-25T18:59:19.191769+0000","last_undegraded":"2024-04-25T18:59:19.191769+0000","last_fullsized":"2024-04-25T18:59:19.191769+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:30.209802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.756960+0000","last_change":"2024-04-25T18:55:23.233074+0000","last_active":"2024-04-25T18:59:18.756960+0000","last_peered":"2024-04-25T18:59:18.756960+0000","last_clean":"2024-04-25T18:59:18.756960+0000","last_became_active":"2024-04-25T18:55:23.232751+0000","last_became_peered":"2024-04-25T18:55:23.232751+0000","last_unstale":"2024-04-25T18:59:18.756960+0000","last_undegraded":"2024-04-25T18:59:18.756960+0000","last_fullsized":"2024-04-25T18:59:18.756960+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.584929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923555+0000","last_change":"2024-04-25T18:55:19.166093+0000","last_active":"2024-04-25T18:59:18.923555+0000","last_peered":"2024-04-25T18:59:18.923555+0000","last_clean":"2024-04-25T18:59:18.923555+0000","last_became_active":"2024-04-25T18:55:19.165715+0000","last_became_peered":"2024-04-25T18:55:19.165715+0000","last_unstale":"2024-04-25T18:59:18.923555+0000","last_undegraded":"2024-04-25T18:59:18.923555+0000","last_fullsized":"2024-04-25T18:59:18.923555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:32.177943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758234+0000","last_change":"2024-04-25T18:55:23.224617+0000","last_active":"2024-04-25T18:59:18.758234+0000","last_peered":"2024-04-25T18:59:18.758234+0000","last_clean":"2024-04-25T18:59:18.758234+0000","last_became_active":"2024-04-25T18:55:23.224269+0000","last_became_peered":"2024-04-25T18:55:23.224269+0000","last_unstale":"2024-04-25T18:59:18.758234+0000","last_undegraded":"2024-04-25T18:59:18.758234+0000","last_fullsized":"2024-04-25T18:59:18.758234+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:19.079910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":256,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:22.903700+0000","last_change":"2024-04-25T18:55:19.164971+0000","last_active":"2024-04-25T18:59:22.903700+0000","last_peered":"2024-04-25T18:59:22.903700+0000","last_clean":"2024-04-25T18:59:22.903700+0000","last_became_active":"2024-04-25T18:55:19.164579+0000","last_became_peered":"2024-04-25T18:55:19.164579+0000","last_unstale":"2024-04-25T18:59:22.903700+0000","last_undegraded":"2024-04-25T18:59:22.903700+0000","last_fullsized":"2024-04-25T18:59:22.903700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:37.419379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":223,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757934+0000","last_change":"2024-04-25T18:55:23.222160+0000","last_active":"2024-04-25T18:59:18.757934+0000","last_peered":"2024-04-25T18:59:18.757934+0000","last_clean":"2024-04-25T18:59:18.757934+0000","last_became_active":"2024-04-25T18:55:23.221812+0000","last_became_peered":"2024-04-25T18:55:23.221812+0000","last_unstale":"2024-04-25T18:59:18.757934+0000","last_undegraded":"2024-04-25T18:59:18.757934+0000","last_fullsized":"2024-04-25T18:59:18.757934+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:53:53.388764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757095+0000","last_change":"2024-04-25T18:55:19.179970+0000","last_active":"2024-04-25T18:59:18.757095+0000","last_peered":"2024-04-25T18:59:18.757095+0000","last_clean":"2024-04-25T18:59:18.757095+0000","last_became_active":"2024-04-25T18:55:19.179545+0000","last_became_peered":"2024-04-25T18:55:19.179545+0000","last_unstale":"2024-04-25T18:59:18.757095+0000","last_undegraded":"2024-04-25T18:59:18.757095+0000","last_fullsized":"2024-04-25T18:59:18.757095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:42.841661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923474+0000","last_change":"2024-04-25T18:55:23.216134+0000","last_active":"2024-04-25T18:59:18.923474+0000","last_peered":"2024-04-25T18:59:18.923474+0000","last_clean":"2024-04-25T18:59:18.923474+0000","last_became_active":"2024-04-25T18:55:23.215535+0000","last_became_peered":"2024-04-25T18:55:23.215535+0000","last_unstale":"2024-04-25T18:59:18.923474+0000","last_undegraded":"2024-04-25T18:59:18.923474+0000","last_fullsized":"2024-04-25T18:59:18.923474+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:08.042095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757699+0000","last_change":"2024-04-25T18:55:19.162437+0000","last_active":"2024-04-25T18:59:18.757699+0000","last_peered":"2024-04-25T18:59:18.757699+0000","last_clean":"2024-04-25T18:59:18.757699+0000","last_became_active":"2024-04-25T18:55:19.162005+0000","last_became_peered":"2024-04-25T18:55:19.162005+0000","last_unstale":"2024-04-25T18:59:18.757699+0000","last_undegraded":"2024-04-25T18:59:18.757699+0000","last_fullsized":"2024-04-25T18:59:18.757699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:50:53.967081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192027+0000","last_change":"2024-04-25T18:55:23.222345+0000","last_active":"2024-04-25T18:59:19.192027+0000","last_peered":"2024-04-25T18:59:19.192027+0000","last_clean":"2024-04-25T18:59:19.192027+0000","last_became_active":"2024-04-25T18:55:23.221142+0000","last_became_peered":"2024-04-25T18:55:23.221142+0000","last_unstale":"2024-04-25T18:59:19.192027+0000","last_undegraded":"2024-04-25T18:59:19.192027+0000","last_fullsized":"2024-04-25T18:59:19.192027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:53.823585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907451+0000","last_change":"2024-04-25T18:55:19.181153+0000","last_active":"2024-04-25T18:59:18.907451+0000","last_peered":"2024-04-25T18:59:18.907451+0000","last_clean":"2024-04-25T18:59:18.907451+0000","last_became_active":"2024-04-25T18:55:19.180553+0000","last_became_peered":"2024-04-25T18:55:19.180553+0000","last_unstale":"2024-04-25T18:59:18.907451+0000","last_undegraded":"2024-04-25T18:59:18.907451+0000","last_fullsized":"2024-04-25T18:59:18.907451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:07.829617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928837+0000","last_change":"2024-04-25T18:55:23.238824+0000","last_active":"2024-04-25T18:59:18.928837+0000","last_peered":"2024-04-25T18:59:18.928837+0000","last_clean":"2024-04-25T18:59:18.928837+0000","last_became_active":"2024-04-25T18:55:23.238228+0000","last_became_peered":"2024-04-25T18:55:23.238228+0000","last_unstale":"2024-04-25T18:59:18.928837+0000","last_undegraded":"2024-04-25T18:59:18.928837+0000","last_fullsized":"2024-04-25T18:59:18.928837+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:16.279254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758247+0000","last_change":"2024-04-25T18:55:19.171186+0000","last_active":"2024-04-25T18:59:18.758247+0000","last_peered":"2024-04-25T18:59:18.758247+0000","last_clean":"2024-04-25T18:59:18.758247+0000","last_became_active":"2024-04-25T18:55:19.170800+0000","last_became_peered":"2024-04-25T18:55:19.170800+0000","last_unstale":"2024-04-25T18:59:18.758247+0000","last_undegraded":"2024-04-25T18:59:18.758247+0000","last_fullsized":"2024-04-25T18:59:18.758247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.599119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928761+0000","last_change":"2024-04-25T18:55:23.236951+0000","last_active":"2024-04-25T18:59:18.928761+0000","last_peered":"2024-04-25T18:59:18.928761+0000","last_clean":"2024-04-25T18:59:18.928761+0000","last_became_active":"2024-04-25T18:55:23.236596+0000","last_became_peered":"2024-04-25T18:55:23.236596+0000","last_unstale":"2024-04-25T18:59:18.928761+0000","last_undegraded":"2024-04-25T18:59:18.928761+0000","last_fullsized":"2024-04-25T18:59:18.928761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:14.634653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923773+0000","last_change":"2024-04-25T18:55:23.211984+0000","last_active":"2024-04-25T18:59:18.923773+0000","last_peered":"2024-04-25T18:59:18.923773+0000","last_clean":"2024-04-25T18:59:18.923773+0000","last_became_active":"2024-04-25T18:55:23.211591+0000","last_became_peered":"2024-04-25T18:55:23.211591+0000","last_unstale":"2024-04-25T18:59:18.923773+0000","last_undegraded":"2024-04-25T18:59:18.923773+0000","last_fullsized":"2024-04-25T18:59:18.923773+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:09.024891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":252,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:22.902868+0000","last_change":"2024-04-25T18:55:19.180378+0000","last_active":"2024-04-25T18:59:22.902868+0000","last_peered":"2024-04-25T18:59:22.902868+0000","last_clean":"2024-04-25T18:59:22.902868+0000","last_became_active":"2024-04-25T18:55:19.179905+0000","last_became_peered":"2024-04-25T18:55:19.179905+0000","last_unstale":"2024-04-25T18:59:22.902868+0000","last_undegraded":"2024-04-25T18:59:22.902868+0000","last_fullsized":"2024-04-25T18:59:22.902868+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:36.434477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":218,"num_read_kb":218,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895798+0000","last_change":"2024-04-25T18:55:23.231489+0000","last_active":"2024-04-25T18:59:18.895798+0000","last_peered":"2024-04-25T18:59:18.895798+0000","last_clean":"2024-04-25T18:59:18.895798+0000","last_became_active":"2024-04-25T18:55:23.231049+0000","last_became_peered":"2024-04-25T18:55:23.231049+0000","last_unstale":"2024-04-25T18:59:18.895798+0000","last_undegraded":"2024-04-25T18:59:18.895798+0000","last_fullsized":"2024-04-25T18:59:18.895798+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:25.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895192+0000","last_change":"2024-04-25T18:55:19.171694+0000","last_active":"2024-04-25T18:59:18.895192+0000","last_peered":"2024-04-25T18:59:18.895192+0000","last_clean":"2024-04-25T18:59:18.895192+0000","last_became_active":"2024-04-25T18:55:19.171296+0000","last_became_peered":"2024-04-25T18:55:19.171296+0000","last_unstale":"2024-04-25T18:59:18.895192+0000","last_undegraded":"2024-04-25T18:59:18.895192+0000","last_fullsized":"2024-04-25T18:59:18.895192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:33.216980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":410,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:20.926517+0000","last_change":"2024-04-25T18:53:49.127506+0000","last_active":"2024-04-25T18:59:20.926517+0000","last_peered":"2024-04-25T18:59:20.926517+0000","last_clean":"2024-04-25T18:59:20.926517+0000","last_became_active":"2024-04-25T18:53:49.126974+0000","last_became_peered":"2024-04-25T18:53:49.126974+0000","last_unstale":"2024-04-25T18:59:20.926517+0000","last_undegraded":"2024-04-25T18:59:20.926517+0000","last_fullsized":"2024-04-25T18:59:20.926517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_clean_scrub_stamp":"2024-04-25T18:50:00.765468+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.488525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757160+0000","last_change":"2024-04-25T18:55:23.211697+0000","last_active":"2024-04-25T18:59:18.757160+0000","last_peered":"2024-04-25T18:59:18.757160+0000","last_clean":"2024-04-25T18:59:18.757160+0000","last_became_active":"2024-04-25T18:55:23.211160+0000","last_became_peered":"2024-04-25T18:55:23.211160+0000","last_unstale":"2024-04-25T18:59:18.757160+0000","last_undegraded":"2024-04-25T18:59:18.757160+0000","last_fullsized":"2024-04-25T18:59:18.757160+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:03.452578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758287+0000","last_change":"2024-04-25T18:55:19.165955+0000","last_active":"2024-04-25T18:59:18.758287+0000","last_peered":"2024-04-25T18:59:18.758287+0000","last_clean":"2024-04-25T18:59:18.758287+0000","last_became_active":"2024-04-25T18:55:19.165594+0000","last_became_peered":"2024-04-25T18:55:19.165594+0000","last_unstale":"2024-04-25T18:59:18.758287+0000","last_undegraded":"2024-04-25T18:59:18.758287+0000","last_fullsized":"2024-04-25T18:59:18.758287+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:11.692374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757737+0000","last_change":"2024-04-25T18:55:19.182698+0000","last_active":"2024-04-25T18:59:18.757737+0000","last_peered":"2024-04-25T18:59:18.757737+0000","last_clean":"2024-04-25T18:59:18.757737+0000","last_became_active":"2024-04-25T18:55:19.182393+0000","last_became_peered":"2024-04-25T18:55:19.182393+0000","last_unstale":"2024-04-25T18:59:18.757737+0000","last_undegraded":"2024-04-25T18:59:18.757737+0000","last_fullsized":"2024-04-25T18:59:18.757737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.513613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757347+0000","last_change":"2024-04-25T18:55:23.223240+0000","last_active":"2024-04-25T18:59:18.757347+0000","last_peered":"2024-04-25T18:59:18.757347+0000","last_clean":"2024-04-25T18:59:18.757347+0000","last_became_active":"2024-04-25T18:55:23.218253+0000","last_became_peered":"2024-04-25T18:55:23.218253+0000","last_unstale":"2024-04-25T18:59:18.757347+0000","last_undegraded":"2024-04-25T18:59:18.757347+0000","last_fullsized":"2024-04-25T18:59:18.757347+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:40.435633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"95'103","reported_seq":224,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:22.023135+0000","last_change":"2024-04-25T18:55:19.168535+0000","last_active":"2024-04-25T18:59:22.023135+0000","last_peered":"2024-04-25T18:59:22.023135+0000","last_clean":"2024-04-25T18:59:22.023135+0000","last_became_active":"2024-04-25T18:55:19.166735+0000","last_became_peered":"2024-04-25T18:55:19.166735+0000","last_unstale":"2024-04-25T18:59:22.023135+0000","last_undegraded":"2024-04-25T18:59:22.023135+0000","last_fullsized":"2024-04-25T18:59:22.023135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:50:14.008290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":189,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.896087+0000","last_change":"2024-04-25T18:55:23.233989+0000","last_active":"2024-04-25T18:59:18.896087+0000","last_peered":"2024-04-25T18:59:18.896087+0000","last_clean":"2024-04-25T18:59:18.896087+0000","last_became_active":"2024-04-25T18:55:23.226867+0000","last_became_peered":"2024-04-25T18:55:23.226867+0000","last_unstale":"2024-04-25T18:59:18.896087+0000","last_undegraded":"2024-04-25T18:59:18.896087+0000","last_fullsized":"2024-04-25T18:59:18.896087+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:02.314368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895339+0000","last_change":"2024-04-25T18:55:19.870412+0000","last_active":"2024-04-25T18:59:18.895339+0000","last_peered":"2024-04-25T18:59:18.895339+0000","last_clean":"2024-04-25T18:59:18.895339+0000","last_became_active":"2024-04-25T18:55:19.869970+0000","last_became_peered":"2024-04-25T18:55:19.869970+0000","last_unstale":"2024-04-25T18:59:18.895339+0000","last_undegraded":"2024-04-25T18:59:18.895339+0000","last_fullsized":"2024-04-25T18:59:18.895339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:22.236417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193471+0000","last_change":"2024-04-25T18:55:23.239172+0000","last_active":"2024-04-25T18:59:19.193471+0000","last_peered":"2024-04-25T18:59:19.193471+0000","last_clean":"2024-04-25T18:59:19.193471+0000","last_became_active":"2024-04-25T18:55:23.238732+0000","last_became_peered":"2024-04-25T18:55:23.238732+0000","last_unstale":"2024-04-25T18:59:19.193471+0000","last_undegraded":"2024-04-25T18:59:19.193471+0000","last_fullsized":"2024-04-25T18:59:19.193471+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:27.459659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132316+0000","last_change":"2024-04-25T18:55:19.179687+0000","last_active":"2024-04-25T18:59:19.132316+0000","last_peered":"2024-04-25T18:59:19.132316+0000","last_clean":"2024-04-25T18:59:19.132316+0000","last_became_active":"2024-04-25T18:55:19.179225+0000","last_became_peered":"2024-04-25T18:55:19.179225+0000","last_unstale":"2024-04-25T18:59:19.132316+0000","last_undegraded":"2024-04-25T18:59:19.132316+0000","last_fullsized":"2024-04-25T18:59:19.132316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:16:16.266021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757162+0000","last_change":"2024-04-25T18:55:23.221379+0000","last_active":"2024-04-25T18:59:18.757162+0000","last_peered":"2024-04-25T18:59:18.757162+0000","last_clean":"2024-04-25T18:59:18.757162+0000","last_became_active":"2024-04-25T18:55:23.221037+0000","last_became_peered":"2024-04-25T18:55:23.221037+0000","last_unstale":"2024-04-25T18:59:18.757162+0000","last_undegraded":"2024-04-25T18:59:18.757162+0000","last_fullsized":"2024-04-25T18:59:18.757162+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:27.581084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895320+0000","last_change":"2024-04-25T18:55:19.171298+0000","last_active":"2024-04-25T18:59:18.895320+0000","last_peered":"2024-04-25T18:59:18.895320+0000","last_clean":"2024-04-25T18:59:18.895320+0000","last_became_active":"2024-04-25T18:55:19.170738+0000","last_became_peered":"2024-04-25T18:55:19.170738+0000","last_unstale":"2024-04-25T18:59:18.895320+0000","last_undegraded":"2024-04-25T18:59:18.895320+0000","last_fullsized":"2024-04-25T18:59:18.895320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.236331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191708+0000","last_change":"2024-04-25T18:55:23.226577+0000","last_active":"2024-04-25T18:59:19.191708+0000","last_peered":"2024-04-25T18:59:19.191708+0000","last_clean":"2024-04-25T18:59:19.191708+0000","last_became_active":"2024-04-25T18:55:23.226198+0000","last_became_peered":"2024-04-25T18:55:23.226198+0000","last_unstale":"2024-04-25T18:59:19.191708+0000","last_undegraded":"2024-04-25T18:59:19.191708+0000","last_fullsized":"2024-04-25T18:59:19.191708+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:34:59.124523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129472+0000","last_change":"2024-04-25T18:55:19.171917+0000","last_active":"2024-04-25T18:59:19.129472+0000","last_peered":"2024-04-25T18:59:19.129472+0000","last_clean":"2024-04-25T18:59:19.129472+0000","last_became_active":"2024-04-25T18:55:19.171506+0000","last_became_peered":"2024-04-25T18:55:19.171506+0000","last_unstale":"2024-04-25T18:59:19.129472+0000","last_undegraded":"2024-04-25T18:59:19.129472+0000","last_fullsized":"2024-04-25T18:59:19.129472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:53.446504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895608+0000","last_change":"2024-04-25T18:55:23.237828+0000","last_active":"2024-04-25T18:59:18.895608+0000","last_peered":"2024-04-25T18:59:18.895608+0000","last_clean":"2024-04-25T18:59:18.895608+0000","last_became_active":"2024-04-25T18:55:23.237310+0000","last_became_peered":"2024-04-25T18:55:23.237310+0000","last_unstale":"2024-04-25T18:59:18.895608+0000","last_undegraded":"2024-04-25T18:59:18.895608+0000","last_fullsized":"2024-04-25T18:59:18.895608+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:10.987032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757414+0000","last_change":"2024-04-25T18:55:19.163824+0000","last_active":"2024-04-25T18:59:18.757414+0000","last_peered":"2024-04-25T18:59:18.757414+0000","last_clean":"2024-04-25T18:59:18.757414+0000","last_became_active":"2024-04-25T18:55:19.163465+0000","last_became_peered":"2024-04-25T18:55:19.163465+0000","last_unstale":"2024-04-25T18:59:18.757414+0000","last_undegraded":"2024-04-25T18:59:18.757414+0000","last_fullsized":"2024-04-25T18:59:18.757414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:38.347100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758396+0000","last_change":"2024-04-25T18:55:23.224648+0000","last_active":"2024-04-25T18:59:18.758396+0000","last_peered":"2024-04-25T18:59:18.758396+0000","last_clean":"2024-04-25T18:59:18.758396+0000","last_became_active":"2024-04-25T18:55:23.224323+0000","last_became_peered":"2024-04-25T18:55:23.224323+0000","last_unstale":"2024-04-25T18:59:18.758396+0000","last_undegraded":"2024-04-25T18:59:18.758396+0000","last_fullsized":"2024-04-25T18:59:18.758396+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.117792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129721+0000","last_change":"2024-04-25T18:55:19.870213+0000","last_active":"2024-04-25T18:59:19.129721+0000","last_peered":"2024-04-25T18:59:19.129721+0000","last_clean":"2024-04-25T18:59:19.129721+0000","last_became_active":"2024-04-25T18:55:19.869756+0000","last_became_peered":"2024-04-25T18:55:19.869756+0000","last_unstale":"2024-04-25T18:59:19.129721+0000","last_undegraded":"2024-04-25T18:59:19.129721+0000","last_fullsized":"2024-04-25T18:59:19.129721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:44.695876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758609+0000","last_change":"2024-04-25T18:55:23.217094+0000","last_active":"2024-04-25T18:59:18.758609+0000","last_peered":"2024-04-25T18:59:18.758609+0000","last_clean":"2024-04-25T18:59:18.758609+0000","last_became_active":"2024-04-25T18:55:23.216696+0000","last_became_peered":"2024-04-25T18:55:23.216696+0000","last_unstale":"2024-04-25T18:59:18.758609+0000","last_undegraded":"2024-04-25T18:59:18.758609+0000","last_fullsized":"2024-04-25T18:59:18.758609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:07.049349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'65","reported_seq":125,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:21.488027+0000","last_change":"2024-04-25T18:55:19.166377+0000","last_active":"2024-04-25T18:59:21.488027+0000","last_peered":"2024-04-25T18:59:21.488027+0000","last_clean":"2024-04-25T18:59:21.488027+0000","last_became_active":"2024-04-25T18:55:19.166036+0000","last_became_peered":"2024-04-25T18:55:19.166036+0000","last_unstale":"2024-04-25T18:59:21.488027+0000","last_undegraded":"2024-04-25T18:59:21.488027+0000","last_fullsized":"2024-04-25T18:59:21.488027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:04.761919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758305+0000","last_change":"2024-04-25T18:55:23.236893+0000","last_active":"2024-04-25T18:59:18.758305+0000","last_peered":"2024-04-25T18:59:18.758305+0000","last_clean":"2024-04-25T18:59:18.758305+0000","last_became_active":"2024-04-25T18:55:23.236488+0000","last_became_peered":"2024-04-25T18:55:23.236488+0000","last_unstale":"2024-04-25T18:59:18.758305+0000","last_undegraded":"2024-04-25T18:59:18.758305+0000","last_fullsized":"2024-04-25T18:59:18.758305+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:44.605041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757331+0000","last_change":"2024-04-25T18:55:19.175076+0000","last_active":"2024-04-25T18:59:18.757331+0000","last_peered":"2024-04-25T18:59:18.757331+0000","last_clean":"2024-04-25T18:59:18.757331+0000","last_became_active":"2024-04-25T18:55:19.174547+0000","last_became_peered":"2024-04-25T18:55:19.174547+0000","last_unstale":"2024-04-25T18:59:18.757331+0000","last_undegraded":"2024-04-25T18:59:18.757331+0000","last_fullsized":"2024-04-25T18:59:18.757331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:43.791068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758728+0000","last_change":"2024-04-25T18:55:23.219665+0000","last_active":"2024-04-25T18:59:18.758728+0000","last_peered":"2024-04-25T18:59:18.758728+0000","last_clean":"2024-04-25T18:59:18.758728+0000","last_became_active":"2024-04-25T18:55:23.219264+0000","last_became_peered":"2024-04-25T18:55:23.219264+0000","last_unstale":"2024-04-25T18:59:18.758728+0000","last_undegraded":"2024-04-25T18:59:18.758728+0000","last_fullsized":"2024-04-25T18:59:18.758728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:19.008120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191796+0000","last_change":"2024-04-25T18:55:23.220290+0000","last_active":"2024-04-25T18:59:19.191796+0000","last_peered":"2024-04-25T18:59:19.191796+0000","last_clean":"2024-04-25T18:59:19.191796+0000","last_became_active":"2024-04-25T18:55:23.219131+0000","last_became_peered":"2024-04-25T18:55:23.219131+0000","last_unstale":"2024-04-25T18:59:19.191796+0000","last_undegraded":"2024-04-25T18:59:19.191796+0000","last_fullsized":"2024-04-25T18:59:19.191796+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:25:25.678525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129447+0000","last_change":"2024-04-25T18:55:23.216415+0000","last_active":"2024-04-25T18:59:19.129447+0000","last_peered":"2024-04-25T18:59:19.129447+0000","last_clean":"2024-04-25T18:59:19.129447+0000","last_became_active":"2024-04-25T18:55:23.216066+0000","last_became_peered":"2024-04-25T18:55:23.216066+0000","last_unstale":"2024-04-25T18:59:19.129447+0000","last_undegraded":"2024-04-25T18:59:19.129447+0000","last_fullsized":"2024-04-25T18:59:19.129447+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:54.633216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129390+0000","last_change":"2024-04-25T18:55:23.234034+0000","last_active":"2024-04-25T18:59:19.129390+0000","last_peered":"2024-04-25T18:59:19.129390+0000","last_clean":"2024-04-25T18:59:19.129390+0000","last_became_active":"2024-04-25T18:55:23.233639+0000","last_became_peered":"2024-04-25T18:55:23.233639+0000","last_unstale":"2024-04-25T18:59:19.129390+0000","last_undegraded":"2024-04-25T18:59:19.129390+0000","last_fullsized":"2024-04-25T18:59:19.129390+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:57.117958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192719+0000","last_change":"2024-04-25T18:56:17.584511+0000","last_active":"2024-04-25T18:59:19.192719+0000","last_peered":"2024-04-25T18:59:19.192719+0000","last_clean":"2024-04-25T18:59:19.192719+0000","last_became_active":"2024-04-25T18:56:17.583959+0000","last_became_peered":"2024-04-25T18:56:17.583959+0000","last_unstale":"2024-04-25T18:59:19.192719+0000","last_undegraded":"2024-04-25T18:59:19.192719+0000","last_fullsized":"2024-04-25T18:59:19.192719+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.622823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757493+0000","last_change":"2024-04-25T18:55:23.238068+0000","last_active":"2024-04-25T18:59:18.757493+0000","last_peered":"2024-04-25T18:59:18.757493+0000","last_clean":"2024-04-25T18:59:18.757493+0000","last_became_active":"2024-04-25T18:55:23.237773+0000","last_became_peered":"2024-04-25T18:55:23.237773+0000","last_unstale":"2024-04-25T18:59:18.757493+0000","last_undegraded":"2024-04-25T18:59:18.757493+0000","last_fullsized":"2024-04-25T18:59:18.757493+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.688576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191981+0000","last_change":"2024-04-25T18:55:23.215463+0000","last_active":"2024-04-25T18:59:19.191981+0000","last_peered":"2024-04-25T18:59:19.191981+0000","last_clean":"2024-04-25T18:59:19.191981+0000","last_became_active":"2024-04-25T18:55:23.215068+0000","last_became_peered":"2024-04-25T18:55:23.215068+0000","last_unstale":"2024-04-25T18:59:19.191981+0000","last_undegraded":"2024-04-25T18:59:19.191981+0000","last_fullsized":"2024-04-25T18:59:19.191981+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.450828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129793+0000","last_change":"2024-04-25T18:55:23.214362+0000","last_active":"2024-04-25T18:59:19.129793+0000","last_peered":"2024-04-25T18:59:19.129793+0000","last_clean":"2024-04-25T18:59:19.129793+0000","last_became_active":"2024-04-25T18:55:23.214035+0000","last_became_peered":"2024-04-25T18:55:23.214035+0000","last_unstale":"2024-04-25T18:59:19.129793+0000","last_undegraded":"2024-04-25T18:59:19.129793+0000","last_fullsized":"2024-04-25T18:59:19.129793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:33.386859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.896015+0000","last_change":"2024-04-25T18:55:23.235709+0000","last_active":"2024-04-25T18:59:18.896015+0000","last_peered":"2024-04-25T18:59:18.896015+0000","last_clean":"2024-04-25T18:59:18.896015+0000","last_became_active":"2024-04-25T18:55:23.226089+0000","last_became_peered":"2024-04-25T18:55:23.226089+0000","last_unstale":"2024-04-25T18:59:18.896015+0000","last_undegraded":"2024-04-25T18:59:18.896015+0000","last_fullsized":"2024-04-25T18:59:18.896015+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:57.161365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.756774+0000","last_change":"2024-04-25T18:56:17.561546+0000","last_active":"2024-04-25T18:59:18.756774+0000","last_peered":"2024-04-25T18:59:18.756774+0000","last_clean":"2024-04-25T18:59:18.756774+0000","last_became_active":"2024-04-25T18:56:17.559767+0000","last_became_peered":"2024-04-25T18:56:17.559767+0000","last_unstale":"2024-04-25T18:59:18.756774+0000","last_undegraded":"2024-04-25T18:59:18.756774+0000","last_fullsized":"2024-04-25T18:59:18.756774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:59.512008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895271+0000","last_change":"2024-04-25T18:55:23.227620+0000","last_active":"2024-04-25T18:59:18.895271+0000","last_peered":"2024-04-25T18:59:18.895271+0000","last_clean":"2024-04-25T18:59:18.895271+0000","last_became_active":"2024-04-25T18:55:23.227236+0000","last_became_peered":"2024-04-25T18:55:23.227236+0000","last_unstale":"2024-04-25T18:59:18.895271+0000","last_undegraded":"2024-04-25T18:59:18.895271+0000","last_fullsized":"2024-04-25T18:59:18.895271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:32.707619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132494+0000","last_change":"2024-04-25T18:55:23.238393+0000","last_active":"2024-04-25T18:59:19.132494+0000","last_peered":"2024-04-25T18:59:19.132494+0000","last_clean":"2024-04-25T18:59:19.132494+0000","last_became_active":"2024-04-25T18:55:23.237726+0000","last_became_peered":"2024-04-25T18:55:23.237726+0000","last_unstale":"2024-04-25T18:59:19.132494+0000","last_undegraded":"2024-04-25T18:59:19.132494+0000","last_fullsized":"2024-04-25T18:59:19.132494+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:48.543014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757312+0000","last_change":"2024-04-25T18:55:23.231374+0000","last_active":"2024-04-25T18:59:18.757312+0000","last_peered":"2024-04-25T18:59:18.757312+0000","last_clean":"2024-04-25T18:59:18.757312+0000","last_became_active":"2024-04-25T18:55:23.230976+0000","last_became_peered":"2024-04-25T18:55:23.230976+0000","last_unstale":"2024-04-25T18:59:18.757312+0000","last_undegraded":"2024-04-25T18:59:18.757312+0000","last_fullsized":"2024-04-25T18:59:18.757312+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:06.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191923+0000","last_change":"2024-04-25T18:55:23.217332+0000","last_active":"2024-04-25T18:59:19.191923+0000","last_peered":"2024-04-25T18:59:19.191923+0000","last_clean":"2024-04-25T18:59:19.191923+0000","last_became_active":"2024-04-25T18:55:23.216968+0000","last_became_peered":"2024-04-25T18:55:23.216968+0000","last_unstale":"2024-04-25T18:59:19.191923+0000","last_undegraded":"2024-04-25T18:59:19.191923+0000","last_fullsized":"2024-04-25T18:59:19.191923+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:31.000083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801253,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1694,"num_read_kb":1067,"num_write":429,"num_write_kb":2365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2403759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":316,"ondisk_log_size":316,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252928,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":507776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514567,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37492,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707660,"statfs":{"total":95995035648,"available":95956643840,"internally_reserved":0,"allocated":2084864,"data_stored":1502336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710794,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36756,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708396,"statfs":{"total":95995035648,"available":95957397504,"internally_reserved":0,"allocated":1454080,"data_stored":901194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972434,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1454080,"data_stored":901114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234071,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1654784,"data_stored":1101283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430298,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":507367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1122304,"data_stored":581509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":508638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411442,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":584139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117100,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37204,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707948,"statfs":{"total":95995035648,"available":95956938752,"internally_reserved":0,"allocated":1789952,"data_stored":1229126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":508336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691937,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29088,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716064,"statfs":{"total":95995035648,"available":95965249536,"internally_reserved":0,"allocated":1998848,"data_stored":1428667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":74349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":76979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":404691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:59:30.232 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T18:59:30.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T18:59:30.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:30 smithi045 ceph-mon[27776]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-04-25T18:59:30.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:30 smithi045 ceph-mon[27776]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:30 smithi062 ceph-mon[32844]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-04-25T18:59:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:30 smithi062 ceph-mon[32844]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:30.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:30 smithi079 ceph-mon[33002]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 103 KiB/s wr, 59 op/s 2024-04-25T18:59:30.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:30 smithi079 ceph-mon[33002]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:30.745 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:32.580 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:32 smithi045 ceph-mon[27776]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-25T18:59:32.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:32 smithi062 ceph-mon[32844]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-25T18:59:32.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:32 smithi079 ceph-mon[33002]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-04-25T18:59:33.121 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.120+0000 7f8b00800700 1 -- 172.21.15.45:0/1444811410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af810f4e0 msgr2=0x7f8af81118d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.120+0000 7f8b00800700 1 --2- 172.21.15.45:0/1444811410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af810f4e0 0x7f8af81118d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8af400b0a0 tx=0x7f8af400b3b0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.120+0000 7f8b00800700 1 -- 172.21.15.45:0/1444811410 shutdown_connections 2024-04-25T18:59:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.120+0000 7f8b00800700 1 --2- 172.21.15.45:0/1444811410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af806afe0 0x7f8af806b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.120+0000 7f8b00800700 1 --2- 172.21.15.45:0/1444811410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af810f4e0 0x7f8af81118d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.120+0000 7f8b00800700 1 --2- 172.21.15.45:0/1444811410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8af806b900 0x7f8af810efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.120+0000 7f8b00800700 1 -- 172.21.15.45:0/1444811410 >> 172.21.15.45:0/1444811410 conn(0x7f8af80788b0 msgr2=0x7f8af8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.120+0000 7f8b00800700 1 -- 172.21.15.45:0/1444811410 shutdown_connections 2024-04-25T18:59:33.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.120+0000 7f8b00800700 1 -- 172.21.15.45:0/1444811410 wait complete. 2024-04-25T18:59:33.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.122+0000 7f8b00800700 1 Processor -- start 2024-04-25T18:59:33.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.122+0000 7f8b00800700 1 -- start start 2024-04-25T18:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.122+0000 7f8b00800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af806afe0 0x7f8af819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.122+0000 7f8b00800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8af806b900 0x7f8af819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.122+0000 7f8b00800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af810f4e0 0x7f8af819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.122+0000 7f8b00800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8114e30 con 0x7f8af806afe0 2024-04-25T18:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.122+0000 7f8b00800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8114cb0 con 0x7f8af810f4e0 2024-04-25T18:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.122+0000 7f8b00800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8af8114fb0 con 0x7f8af806b900 2024-04-25T18:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.123+0000 7f8afd400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8af806b900 0x7f8af819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.123+0000 7f8afd400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8af806b900 0x7f8af819b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:37676/0 (socket says 172.21.15.45:37676) 2024-04-25T18:59:33.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.123+0000 7f8afde00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af806afe0 0x7f8af819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.123+0000 7f8afd400700 1 -- 172.21.15.45:0/2490240491 learned_addr learned my addr 172.21.15.45:0/2490240491 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.123+0000 7f8afe800700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af810f4e0 0x7f8af819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.123+0000 7f8afd400700 1 -- 172.21.15.45:0/2490240491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af810f4e0 msgr2=0x7f8af819fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.123+0000 7f8afd400700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af810f4e0 0x7f8af819fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.123+0000 7f8afd400700 1 -- 172.21.15.45:0/2490240491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af806afe0 msgr2=0x7f8af819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:33.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.123+0000 7f8afd400700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af806afe0 0x7f8af819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.123+0000 7f8afd400700 1 -- 172.21.15.45:0/2490240491 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8af81a02f0 con 0x7f8af806b900 2024-04-25T18:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.124+0000 7f8afd400700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8af806b900 0x7f8af819b820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800e9c0 tx=0x7f8ae800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.124+0000 7f8af2a00700 1 -- 172.21.15.45:0/2490240491 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae800cbd0 con 0x7f8af806b900 2024-04-25T18:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.124+0000 7f8b00800700 1 -- 172.21.15.45:0/2490240491 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8af81a05e0 con 0x7f8af806b900 2024-04-25T18:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.124+0000 7f8af2a00700 1 -- 172.21.15.45:0/2490240491 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ae800ee60 con 0x7f8af806b900 2024-04-25T18:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.124+0000 7f8b00800700 1 -- 172.21.15.45:0/2490240491 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8af81a7e60 con 0x7f8af806b900 2024-04-25T18:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.124+0000 7f8af2a00700 1 -- 172.21.15.45:0/2490240491 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ae8005550 con 0x7f8af806b900 2024-04-25T18:59:33.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.125+0000 7f8b00800700 1 -- 172.21.15.45:0/2490240491 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8af8069730 con 0x7f8af806b900 2024-04-25T18:59:33.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.128+0000 7f8af2a00700 1 -- 172.21.15.45:0/2490240491 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f8ae8004750 con 0x7f8af806b900 2024-04-25T18:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.129+0000 7f8af2a00700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ae00a3f50 0x7f8ae00a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.129+0000 7f8af2a00700 1 -- 172.21.15.45:0/2490240491 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f8ae80d0c20 con 0x7f8af806b900 2024-04-25T18:59:33.130 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.129+0000 7f8afde00700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ae00a3f50 0x7f8ae00a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:33.131 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.130+0000 7f8afde00700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ae00a3f50 0x7f8ae00a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009c60 tx=0x7f8aec009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:33.132 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.131+0000 7f8af2a00700 1 -- 172.21.15.45:0/2490240491 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ae809d010 con 0x7f8af806b900 2024-04-25T18:59:33.360 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.359+0000 7f8b00800700 1 -- 172.21.15.45:0/2490240491 --> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8af8062e80 con 0x7f8ae00a3f50 2024-04-25T18:59:33.369 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.368+0000 7f8af2a00700 1 -- 172.21.15.45:0/2490240491 <== mgr.25049 v2:172.21.15.45:6800/1345298339 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257773 (secure 0 0 0) 0x7f8af8062e80 con 0x7f8ae00a3f50 2024-04-25T18:59:33.375 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:59:33.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 -- 172.21.15.45:0/2490240491 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ae00a3f50 msgr2=0x7f8ae00a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:33.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ae00a3f50 0x7f8ae00a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8aec009c60 tx=0x7f8aec009260 comp rx=0 tx=0).stop 2024-04-25T18:59:33.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 -- 172.21.15.45:0/2490240491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8af806b900 msgr2=0x7f8af819b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:33.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8af806b900 0x7f8af819b820 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8ae800e9c0 tx=0x7f8ae800ecd0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.377 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 -- 172.21.15.45:0/2490240491 shutdown_connections 2024-04-25T18:59:33.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f8ae00a3f50 0x7f8ae00a6410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f8af810f4e0 0x7f8af819fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8af806afe0 0x7f8af819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 --2- 172.21.15.45:0/2490240491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8af806b900 0x7f8af819b820 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:33.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 -- 172.21.15.45:0/2490240491 >> 172.21.15.45:0/2490240491 conn(0x7f8af80788b0 msgr2=0x7f8af8110930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:33.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 -- 172.21.15.45:0/2490240491 shutdown_connections 2024-04-25T18:59:33.378 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:33.375+0000 7f8b00800700 1 -- 172.21.15.45:0/2490240491 wait complete. 2024-04-25T18:59:33.378 INFO:teuthology.orchestra.run.smithi045.stderr:dumped all 2024-04-25T18:59:34.245 INFO:teuthology.orchestra.run.smithi045.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-25T18:59:32.779716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1391855,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1961,"num_read_kb":1412,"num_write":632,"num_write_kb":5305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180770,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":966,"ondisk_log_size":966,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":395176,"kb_used_data":16524,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124546648,"statfs":{"total":1151940427776,"available":1151535767552,"internally_reserved":0,"allocated":16920576,"data_stored":10261485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":199543,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":415,"num_read_kb":474,"num_write":150,"num_write_kb":919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003385"},"pg_stats":[{"pgid":"3.3f","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192112+0000","last_change":"2024-04-25T18:55:23.241030+0000","last_active":"2024-04-25T18:59:19.192112+0000","last_peered":"2024-04-25T18:59:19.192112+0000","last_clean":"2024-04-25T18:59:19.192112+0000","last_became_active":"2024-04-25T18:55:23.240724+0000","last_became_peered":"2024-04-25T18:55:23.240724+0000","last_unstale":"2024-04-25T18:59:19.192112+0000","last_undegraded":"2024-04-25T18:59:19.192112+0000","last_fullsized":"2024-04-25T18:59:19.192112+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:31:37.995475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193313+0000","last_change":"2024-04-25T18:55:23.229875+0000","last_active":"2024-04-25T18:59:19.193313+0000","last_peered":"2024-04-25T18:59:19.193313+0000","last_clean":"2024-04-25T18:59:19.193313+0000","last_became_active":"2024-04-25T18:55:23.229640+0000","last_became_peered":"2024-04-25T18:55:23.229640+0000","last_unstale":"2024-04-25T18:59:19.193313+0000","last_undegraded":"2024-04-25T18:59:19.193313+0000","last_fullsized":"2024-04-25T18:59:19.193313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:10.262376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193164+0000","last_change":"2024-04-25T18:55:23.229821+0000","last_active":"2024-04-25T18:59:19.193164+0000","last_peered":"2024-04-25T18:59:19.193164+0000","last_clean":"2024-04-25T18:59:19.193164+0000","last_became_active":"2024-04-25T18:55:23.229414+0000","last_became_peered":"2024-04-25T18:55:23.229414+0000","last_unstale":"2024-04-25T18:59:19.193164+0000","last_undegraded":"2024-04-25T18:59:19.193164+0000","last_fullsized":"2024-04-25T18:59:19.193164+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:56.063786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907514+0000","last_change":"2024-04-25T18:56:17.560167+0000","last_active":"2024-04-25T18:59:18.907514+0000","last_peered":"2024-04-25T18:59:18.907514+0000","last_clean":"2024-04-25T18:59:18.907514+0000","last_became_active":"2024-04-25T18:56:17.558466+0000","last_became_peered":"2024-04-25T18:56:17.558466+0000","last_unstale":"2024-04-25T18:59:18.907514+0000","last_undegraded":"2024-04-25T18:59:18.907514+0000","last_fullsized":"2024-04-25T18:59:18.907514+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:22.371076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757484+0000","last_change":"2024-04-25T18:55:23.233512+0000","last_active":"2024-04-25T18:59:18.757484+0000","last_peered":"2024-04-25T18:59:18.757484+0000","last_clean":"2024-04-25T18:59:18.757484+0000","last_became_active":"2024-04-25T18:55:23.233158+0000","last_became_peered":"2024-04-25T18:55:23.233158+0000","last_unstale":"2024-04-25T18:59:18.757484+0000","last_undegraded":"2024-04-25T18:59:18.757484+0000","last_fullsized":"2024-04-25T18:59:18.757484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:07:44.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928760+0000","last_change":"2024-04-25T18:55:23.225921+0000","last_active":"2024-04-25T18:59:18.928760+0000","last_peered":"2024-04-25T18:59:18.928760+0000","last_clean":"2024-04-25T18:59:18.928760+0000","last_became_active":"2024-04-25T18:55:23.225547+0000","last_became_peered":"2024-04-25T18:55:23.225547+0000","last_unstale":"2024-04-25T18:59:18.928760+0000","last_undegraded":"2024-04-25T18:59:18.928760+0000","last_fullsized":"2024-04-25T18:59:18.928760+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:29.828394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758001+0000","last_change":"2024-04-25T18:55:23.217111+0000","last_active":"2024-04-25T18:59:18.758001+0000","last_peered":"2024-04-25T18:59:18.758001+0000","last_clean":"2024-04-25T18:59:18.758001+0000","last_became_active":"2024-04-25T18:55:23.216731+0000","last_became_peered":"2024-04-25T18:55:23.216731+0000","last_unstale":"2024-04-25T18:59:18.758001+0000","last_undegraded":"2024-04-25T18:59:18.758001+0000","last_fullsized":"2024-04-25T18:59:18.758001+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:00:17.931928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"94'9","reported_seq":23,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907086+0000","last_change":"2024-04-25T18:56:17.555014+0000","last_active":"2024-04-25T18:59:18.907086+0000","last_peered":"2024-04-25T18:59:18.907086+0000","last_clean":"2024-04-25T18:59:18.907086+0000","last_became_active":"2024-04-25T18:56:17.553863+0000","last_became_peered":"2024-04-25T18:56:17.553863+0000","last_unstale":"2024-04-25T18:59:18.907086+0000","last_undegraded":"2024-04-25T18:59:18.907086+0000","last_fullsized":"2024-04-25T18:59:18.907086+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.240754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132827+0000","last_change":"2024-04-25T18:55:23.218197+0000","last_active":"2024-04-25T18:59:19.132827+0000","last_peered":"2024-04-25T18:59:19.132827+0000","last_clean":"2024-04-25T18:59:19.132827+0000","last_became_active":"2024-04-25T18:55:23.217665+0000","last_became_peered":"2024-04-25T18:55:23.217665+0000","last_unstale":"2024-04-25T18:59:19.132827+0000","last_undegraded":"2024-04-25T18:59:19.132827+0000","last_fullsized":"2024-04-25T18:59:19.132827+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:11.829991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"94'16","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.929205+0000","last_change":"2024-04-25T18:55:23.221533+0000","last_active":"2024-04-25T18:59:18.929205+0000","last_peered":"2024-04-25T18:59:18.929205+0000","last_clean":"2024-04-25T18:59:18.929205+0000","last_became_active":"2024-04-25T18:55:23.220711+0000","last_became_peered":"2024-04-25T18:55:23.220711+0000","last_unstale":"2024-04-25T18:59:18.929205+0000","last_undegraded":"2024-04-25T18:59:18.929205+0000","last_fullsized":"2024-04-25T18:59:18.929205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:45.185166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758645+0000","last_change":"2024-04-25T18:55:23.221329+0000","last_active":"2024-04-25T18:59:18.758645+0000","last_peered":"2024-04-25T18:59:18.758645+0000","last_clean":"2024-04-25T18:59:18.758645+0000","last_became_active":"2024-04-25T18:55:23.220156+0000","last_became_peered":"2024-04-25T18:55:23.220156+0000","last_unstale":"2024-04-25T18:59:18.758645+0000","last_undegraded":"2024-04-25T18:59:18.758645+0000","last_fullsized":"2024-04-25T18:59:18.758645+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:46.981199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"94'7","reported_seq":21,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907481+0000","last_change":"2024-04-25T18:56:17.558018+0000","last_active":"2024-04-25T18:59:18.907481+0000","last_peered":"2024-04-25T18:59:18.907481+0000","last_clean":"2024-04-25T18:59:18.907481+0000","last_became_active":"2024-04-25T18:56:17.556229+0000","last_became_peered":"2024-04-25T18:56:17.556229+0000","last_unstale":"2024-04-25T18:59:18.907481+0000","last_undegraded":"2024-04-25T18:59:18.907481+0000","last_fullsized":"2024-04-25T18:59:18.907481+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:50.949895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757146+0000","last_change":"2024-04-25T18:55:23.227474+0000","last_active":"2024-04-25T18:59:18.757146+0000","last_peered":"2024-04-25T18:59:18.757146+0000","last_clean":"2024-04-25T18:59:18.757146+0000","last_became_active":"2024-04-25T18:55:23.227068+0000","last_became_peered":"2024-04-25T18:55:23.227068+0000","last_unstale":"2024-04-25T18:59:18.757146+0000","last_undegraded":"2024-04-25T18:59:18.757146+0000","last_fullsized":"2024-04-25T18:59:18.757146+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:44:13.884833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192323+0000","last_change":"2024-04-25T18:55:23.230572+0000","last_active":"2024-04-25T18:59:19.192323+0000","last_peered":"2024-04-25T18:59:19.192323+0000","last_clean":"2024-04-25T18:59:19.192323+0000","last_became_active":"2024-04-25T18:55:23.229756+0000","last_became_peered":"2024-04-25T18:55:23.229756+0000","last_unstale":"2024-04-25T18:59:19.192323+0000","last_undegraded":"2024-04-25T18:59:19.192323+0000","last_fullsized":"2024-04-25T18:59:19.192323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:15.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192637+0000","last_change":"2024-04-25T18:55:23.227913+0000","last_active":"2024-04-25T18:59:19.192637+0000","last_peered":"2024-04-25T18:59:19.192637+0000","last_clean":"2024-04-25T18:59:19.192637+0000","last_became_active":"2024-04-25T18:55:23.227598+0000","last_became_peered":"2024-04-25T18:55:23.227598+0000","last_unstale":"2024-04-25T18:59:19.192637+0000","last_undegraded":"2024-04-25T18:59:19.192637+0000","last_fullsized":"2024-04-25T18:59:19.192637+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:56:33.555741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132304+0000","last_change":"2024-04-25T18:55:23.230241+0000","last_active":"2024-04-25T18:59:19.132304+0000","last_peered":"2024-04-25T18:59:19.132304+0000","last_clean":"2024-04-25T18:59:19.132304+0000","last_became_active":"2024-04-25T18:55:23.229898+0000","last_became_peered":"2024-04-25T18:55:23.229898+0000","last_unstale":"2024-04-25T18:59:19.132304+0000","last_undegraded":"2024-04-25T18:59:19.132304+0000","last_fullsized":"2024-04-25T18:59:19.132304+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:52:27.278475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192205+0000","last_change":"2024-04-25T18:55:23.240978+0000","last_active":"2024-04-25T18:59:19.192205+0000","last_peered":"2024-04-25T18:59:19.192205+0000","last_clean":"2024-04-25T18:59:19.192205+0000","last_became_active":"2024-04-25T18:55:23.240702+0000","last_became_peered":"2024-04-25T18:55:23.240702+0000","last_unstale":"2024-04-25T18:59:19.192205+0000","last_undegraded":"2024-04-25T18:59:19.192205+0000","last_fullsized":"2024-04-25T18:59:19.192205+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:06.053688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907730+0000","last_change":"2024-04-25T18:55:23.232116+0000","last_active":"2024-04-25T18:59:18.907730+0000","last_peered":"2024-04-25T18:59:18.907730+0000","last_clean":"2024-04-25T18:59:18.907730+0000","last_became_active":"2024-04-25T18:55:23.231586+0000","last_became_peered":"2024-04-25T18:55:23.231586+0000","last_unstale":"2024-04-25T18:59:18.907730+0000","last_undegraded":"2024-04-25T18:59:18.907730+0000","last_fullsized":"2024-04-25T18:59:18.907730+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:32.640823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129676+0000","last_change":"2024-04-25T18:55:23.224519+0000","last_active":"2024-04-25T18:59:19.129676+0000","last_peered":"2024-04-25T18:59:19.129676+0000","last_clean":"2024-04-25T18:59:19.129676+0000","last_became_active":"2024-04-25T18:55:23.224145+0000","last_became_peered":"2024-04-25T18:55:23.224145+0000","last_unstale":"2024-04-25T18:59:19.129676+0000","last_undegraded":"2024-04-25T18:59:19.129676+0000","last_fullsized":"2024-04-25T18:59:19.129676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:35:03.340162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"92'9","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907818+0000","last_change":"2024-04-25T18:55:19.164977+0000","last_active":"2024-04-25T18:59:18.907818+0000","last_peered":"2024-04-25T18:59:18.907818+0000","last_clean":"2024-04-25T18:59:18.907818+0000","last_became_active":"2024-04-25T18:55:19.164316+0000","last_became_peered":"2024-04-25T18:55:19.164316+0000","last_unstale":"2024-04-25T18:59:18.907818+0000","last_undegraded":"2024-04-25T18:59:18.907818+0000","last_fullsized":"2024-04-25T18:59:18.907818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:08.031680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"94'5","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895576+0000","last_change":"2024-04-25T18:56:17.559746+0000","last_active":"2024-04-25T18:59:18.895576+0000","last_peered":"2024-04-25T18:59:18.895576+0000","last_clean":"2024-04-25T18:59:18.895576+0000","last_became_active":"2024-04-25T18:56:17.558761+0000","last_became_peered":"2024-04-25T18:56:17.558761+0000","last_unstale":"2024-04-25T18:59:18.895576+0000","last_undegraded":"2024-04-25T18:59:18.895576+0000","last_fullsized":"2024-04-25T18:59:18.895576+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:43.518760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"92'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895745+0000","last_change":"2024-04-25T18:55:19.179943+0000","last_active":"2024-04-25T18:59:18.895745+0000","last_peered":"2024-04-25T18:59:18.895745+0000","last_clean":"2024-04-25T18:59:18.895745+0000","last_became_active":"2024-04-25T18:55:19.179519+0000","last_became_peered":"2024-04-25T18:55:19.179519+0000","last_unstale":"2024-04-25T18:59:18.895745+0000","last_undegraded":"2024-04-25T18:59:18.895745+0000","last_fullsized":"2024-04-25T18:59:18.895745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:46.365187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193409+0000","last_change":"2024-04-25T18:55:23.237965+0000","last_active":"2024-04-25T18:59:19.193409+0000","last_peered":"2024-04-25T18:59:19.193409+0000","last_clean":"2024-04-25T18:59:19.193409+0000","last_became_active":"2024-04-25T18:55:23.237520+0000","last_became_peered":"2024-04-25T18:55:23.237520+0000","last_unstale":"2024-04-25T18:59:19.193409+0000","last_undegraded":"2024-04-25T18:59:19.193409+0000","last_fullsized":"2024-04-25T18:59:19.193409+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:03.108688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"92'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757872+0000","last_change":"2024-04-25T18:55:19.163965+0000","last_active":"2024-04-25T18:59:18.757872+0000","last_peered":"2024-04-25T18:59:18.757872+0000","last_clean":"2024-04-25T18:59:18.757872+0000","last_became_active":"2024-04-25T18:55:19.163390+0000","last_became_peered":"2024-04-25T18:55:19.163390+0000","last_unstale":"2024-04-25T18:59:18.757872+0000","last_undegraded":"2024-04-25T18:59:18.757872+0000","last_fullsized":"2024-04-25T18:59:18.757872+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:03:54.183730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132443+0000","last_change":"2024-04-25T18:55:23.227124+0000","last_active":"2024-04-25T18:59:19.132443+0000","last_peered":"2024-04-25T18:59:19.132443+0000","last_clean":"2024-04-25T18:59:19.132443+0000","last_became_active":"2024-04-25T18:55:23.226383+0000","last_became_peered":"2024-04-25T18:55:23.226383+0000","last_unstale":"2024-04-25T18:59:19.132443+0000","last_undegraded":"2024-04-25T18:59:19.132443+0000","last_fullsized":"2024-04-25T18:59:19.132443+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:22.011212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"92'2","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191814+0000","last_change":"2024-04-25T18:56:17.544303+0000","last_active":"2024-04-25T18:59:19.191814+0000","last_peered":"2024-04-25T18:59:19.191814+0000","last_clean":"2024-04-25T18:59:19.191814+0000","last_became_active":"2024-04-25T18:56:17.542002+0000","last_became_peered":"2024-04-25T18:56:17.542002+0000","last_unstale":"2024-04-25T18:59:19.191814+0000","last_undegraded":"2024-04-25T18:59:19.191814+0000","last_fullsized":"2024-04-25T18:59:19.191814+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:23.848398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923703+0000","last_change":"2024-04-25T18:55:23.217703+0000","last_active":"2024-04-25T18:59:18.923703+0000","last_peered":"2024-04-25T18:59:18.923703+0000","last_clean":"2024-04-25T18:59:18.923703+0000","last_became_active":"2024-04-25T18:55:23.216767+0000","last_became_peered":"2024-04-25T18:55:23.216767+0000","last_unstale":"2024-04-25T18:59:18.923703+0000","last_undegraded":"2024-04-25T18:59:18.923703+0000","last_fullsized":"2024-04-25T18:59:18.923703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:53:17.779356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"95'9","reported_seq":267,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:28.754440+0000","last_change":"2024-04-25T18:55:19.164199+0000","last_active":"2024-04-25T18:59:28.754440+0000","last_peered":"2024-04-25T18:59:28.754440+0000","last_clean":"2024-04-25T18:59:28.754440+0000","last_became_active":"2024-04-25T18:55:19.163849+0000","last_became_peered":"2024-04-25T18:55:19.163849+0000","last_unstale":"2024-04-25T18:59:28.754440+0000","last_undegraded":"2024-04-25T18:59:28.754440+0000","last_fullsized":"2024-04-25T18:59:28.754440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:10:25.234229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":229,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757254+0000","last_change":"2024-04-25T18:55:23.216916+0000","last_active":"2024-04-25T18:59:18.757254+0000","last_peered":"2024-04-25T18:59:18.757254+0000","last_clean":"2024-04-25T18:59:18.757254+0000","last_became_active":"2024-04-25T18:55:23.216249+0000","last_became_peered":"2024-04-25T18:55:23.216249+0000","last_unstale":"2024-04-25T18:59:18.757254+0000","last_undegraded":"2024-04-25T18:59:18.757254+0000","last_fullsized":"2024-04-25T18:59:18.757254+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:21.522318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928712+0000","last_change":"2024-04-25T18:55:19.171004+0000","last_active":"2024-04-25T18:59:18.928712+0000","last_peered":"2024-04-25T18:59:18.928712+0000","last_clean":"2024-04-25T18:59:18.928712+0000","last_became_active":"2024-04-25T18:55:19.170570+0000","last_became_peered":"2024-04-25T18:55:19.170570+0000","last_unstale":"2024-04-25T18:59:18.928712+0000","last_undegraded":"2024-04-25T18:59:18.928712+0000","last_fullsized":"2024-04-25T18:59:18.928712+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:15:24.954033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"94'8","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923953+0000","last_change":"2024-04-25T18:56:17.552615+0000","last_active":"2024-04-25T18:59:18.923953+0000","last_peered":"2024-04-25T18:59:18.923953+0000","last_clean":"2024-04-25T18:59:18.923953+0000","last_became_active":"2024-04-25T18:56:17.552154+0000","last_became_peered":"2024-04-25T18:56:17.552154+0000","last_unstale":"2024-04-25T18:59:18.923953+0000","last_undegraded":"2024-04-25T18:59:18.923953+0000","last_fullsized":"2024-04-25T18:59:18.923953+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:01.199859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3],"acting":[5,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757300+0000","last_change":"2024-04-25T18:55:19.173398+0000","last_active":"2024-04-25T18:59:18.757300+0000","last_peered":"2024-04-25T18:59:18.757300+0000","last_clean":"2024-04-25T18:59:18.757300+0000","last_became_active":"2024-04-25T18:55:19.171477+0000","last_became_peered":"2024-04-25T18:55:19.171477+0000","last_unstale":"2024-04-25T18:59:18.757300+0000","last_undegraded":"2024-04-25T18:59:18.757300+0000","last_fullsized":"2024-04-25T18:59:18.757300+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:35.230596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895264+0000","last_change":"2024-04-25T18:55:23.235209+0000","last_active":"2024-04-25T18:59:18.895264+0000","last_peered":"2024-04-25T18:59:18.895264+0000","last_clean":"2024-04-25T18:59:18.895264+0000","last_became_active":"2024-04-25T18:55:23.231898+0000","last_became_peered":"2024-04-25T18:55:23.231898+0000","last_unstale":"2024-04-25T18:59:18.895264+0000","last_undegraded":"2024-04-25T18:59:18.895264+0000","last_fullsized":"2024-04-25T18:59:18.895264+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:18.521282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"92'1","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923448+0000","last_change":"2024-04-25T18:55:19.163780+0000","last_active":"2024-04-25T18:59:18.923448+0000","last_peered":"2024-04-25T18:59:18.923448+0000","last_clean":"2024-04-25T18:59:18.923448+0000","last_became_active":"2024-04-25T18:55:19.163197+0000","last_became_peered":"2024-04-25T18:55:19.163197+0000","last_unstale":"2024-04-25T18:59:18.923448+0000","last_undegraded":"2024-04-25T18:59:18.923448+0000","last_fullsized":"2024-04-25T18:59:18.923448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:02:26.519160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"94'11","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907206+0000","last_change":"2024-04-25T18:56:17.556804+0000","last_active":"2024-04-25T18:59:18.907206+0000","last_peered":"2024-04-25T18:59:18.907206+0000","last_clean":"2024-04-25T18:59:18.907206+0000","last_became_active":"2024-04-25T18:56:17.554885+0000","last_became_peered":"2024-04-25T18:56:17.554885+0000","last_unstale":"2024-04-25T18:59:18.907206+0000","last_undegraded":"2024-04-25T18:59:18.907206+0000","last_fullsized":"2024-04-25T18:59:18.907206+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:36.316156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,10],"acting":[9,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"95'5","reported_seq":258,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:28.753788+0000","last_change":"2024-04-25T18:55:19.170565+0000","last_active":"2024-04-25T18:59:28.753788+0000","last_peered":"2024-04-25T18:59:28.753788+0000","last_clean":"2024-04-25T18:59:28.753788+0000","last_became_active":"2024-04-25T18:55:19.170162+0000","last_became_peered":"2024-04-25T18:55:19.170162+0000","last_unstale":"2024-04-25T18:59:28.753788+0000","last_undegraded":"2024-04-25T18:59:28.753788+0000","last_fullsized":"2024-04-25T18:59:28.753788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:37:36.606661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":224,"num_read_kb":224,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132749+0000","last_change":"2024-04-25T18:55:23.227436+0000","last_active":"2024-04-25T18:59:19.132749+0000","last_peered":"2024-04-25T18:59:19.132749+0000","last_clean":"2024-04-25T18:59:19.132749+0000","last_became_active":"2024-04-25T18:55:23.227044+0000","last_became_peered":"2024-04-25T18:55:23.227044+0000","last_unstale":"2024-04-25T18:59:19.132749+0000","last_undegraded":"2024-04-25T18:59:19.132749+0000","last_fullsized":"2024-04-25T18:59:19.132749+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:34.186574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758348+0000","last_change":"2024-04-25T18:55:19.167514+0000","last_active":"2024-04-25T18:59:18.758348+0000","last_peered":"2024-04-25T18:59:18.758348+0000","last_clean":"2024-04-25T18:59:18.758348+0000","last_became_active":"2024-04-25T18:55:19.167102+0000","last_became_peered":"2024-04-25T18:55:19.167102+0000","last_unstale":"2024-04-25T18:59:18.758348+0000","last_undegraded":"2024-04-25T18:59:18.758348+0000","last_fullsized":"2024-04-25T18:59:18.758348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:41.472274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193323+0000","last_change":"2024-04-25T18:55:23.237012+0000","last_active":"2024-04-25T18:59:19.193323+0000","last_peered":"2024-04-25T18:59:19.193323+0000","last_clean":"2024-04-25T18:59:19.193323+0000","last_became_active":"2024-04-25T18:55:23.230263+0000","last_became_peered":"2024-04-25T18:55:23.230263+0000","last_unstale":"2024-04-25T18:59:19.193323+0000","last_undegraded":"2024-04-25T18:59:19.193323+0000","last_fullsized":"2024-04-25T18:59:19.193323+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:30.252449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923910+0000","last_change":"2024-04-25T18:55:19.179555+0000","last_active":"2024-04-25T18:59:18.923910+0000","last_peered":"2024-04-25T18:59:18.923910+0000","last_clean":"2024-04-25T18:59:18.923910+0000","last_became_active":"2024-04-25T18:55:19.179190+0000","last_became_peered":"2024-04-25T18:55:19.179190+0000","last_unstale":"2024-04-25T18:59:18.923910+0000","last_undegraded":"2024-04-25T18:59:18.923910+0000","last_fullsized":"2024-04-25T18:59:18.923910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:57.319320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132483+0000","last_change":"2024-04-25T18:55:23.219978+0000","last_active":"2024-04-25T18:59:19.132483+0000","last_peered":"2024-04-25T18:59:19.132483+0000","last_clean":"2024-04-25T18:59:19.132483+0000","last_became_active":"2024-04-25T18:55:23.219630+0000","last_became_peered":"2024-04-25T18:55:23.219630+0000","last_unstale":"2024-04-25T18:59:19.132483+0000","last_undegraded":"2024-04-25T18:59:19.132483+0000","last_fullsized":"2024-04-25T18:59:19.132483+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:31.294500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"95'41","reported_seq":532,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:29.755336+0000","last_change":"2024-04-25T18:55:19.164042+0000","last_active":"2024-04-25T18:59:29.755336+0000","last_peered":"2024-04-25T18:59:29.755336+0000","last_clean":"2024-04-25T18:59:29.755336+0000","last_became_active":"2024-04-25T18:55:19.163604+0000","last_became_peered":"2024-04-25T18:55:19.163604+0000","last_unstale":"2024-04-25T18:59:29.755336+0000","last_undegraded":"2024-04-25T18:59:29.755336+0000","last_fullsized":"2024-04-25T18:59:29.755336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:55.030715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":74349,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":462,"num_read_kb":303,"num_write":42,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757632+0000","last_change":"2024-04-25T18:55:23.222702+0000","last_active":"2024-04-25T18:59:18.757632+0000","last_peered":"2024-04-25T18:59:18.757632+0000","last_clean":"2024-04-25T18:59:18.757632+0000","last_became_active":"2024-04-25T18:55:23.221849+0000","last_became_peered":"2024-04-25T18:55:23.221849+0000","last_unstale":"2024-04-25T18:59:18.757632+0000","last_undegraded":"2024-04-25T18:59:18.757632+0000","last_fullsized":"2024-04-25T18:59:18.757632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:17.957027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"92'6","reported_seq":24,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191769+0000","last_change":"2024-04-25T18:56:17.561912+0000","last_active":"2024-04-25T18:59:19.191769+0000","last_peered":"2024-04-25T18:59:19.191769+0000","last_clean":"2024-04-25T18:59:19.191769+0000","last_became_active":"2024-04-25T18:56:17.552120+0000","last_became_peered":"2024-04-25T18:56:17.552120+0000","last_unstale":"2024-04-25T18:59:19.191769+0000","last_undegraded":"2024-04-25T18:59:19.191769+0000","last_fullsized":"2024-04-25T18:59:19.191769+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:30.209802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.756960+0000","last_change":"2024-04-25T18:55:23.233074+0000","last_active":"2024-04-25T18:59:18.756960+0000","last_peered":"2024-04-25T18:59:18.756960+0000","last_clean":"2024-04-25T18:59:18.756960+0000","last_became_active":"2024-04-25T18:55:23.232751+0000","last_became_peered":"2024-04-25T18:55:23.232751+0000","last_unstale":"2024-04-25T18:59:18.756960+0000","last_undegraded":"2024-04-25T18:59:18.756960+0000","last_fullsized":"2024-04-25T18:59:18.756960+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:44.584929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923555+0000","last_change":"2024-04-25T18:55:19.166093+0000","last_active":"2024-04-25T18:59:18.923555+0000","last_peered":"2024-04-25T18:59:18.923555+0000","last_clean":"2024-04-25T18:59:18.923555+0000","last_became_active":"2024-04-25T18:55:19.165715+0000","last_became_peered":"2024-04-25T18:55:19.165715+0000","last_unstale":"2024-04-25T18:59:18.923555+0000","last_undegraded":"2024-04-25T18:59:18.923555+0000","last_fullsized":"2024-04-25T18:59:18.923555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:32.177943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758234+0000","last_change":"2024-04-25T18:55:23.224617+0000","last_active":"2024-04-25T18:59:18.758234+0000","last_peered":"2024-04-25T18:59:18.758234+0000","last_clean":"2024-04-25T18:59:18.758234+0000","last_became_active":"2024-04-25T18:55:23.224269+0000","last_became_peered":"2024-04-25T18:55:23.224269+0000","last_unstale":"2024-04-25T18:59:18.758234+0000","last_undegraded":"2024-04-25T18:59:18.758234+0000","last_fullsized":"2024-04-25T18:59:18.758234+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:19.079910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'5","reported_seq":261,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:27.913315+0000","last_change":"2024-04-25T18:55:19.164971+0000","last_active":"2024-04-25T18:59:27.913315+0000","last_peered":"2024-04-25T18:59:27.913315+0000","last_clean":"2024-04-25T18:59:27.913315+0000","last_became_active":"2024-04-25T18:55:19.164579+0000","last_became_peered":"2024-04-25T18:55:19.164579+0000","last_unstale":"2024-04-25T18:59:27.913315+0000","last_undegraded":"2024-04-25T18:59:27.913315+0000","last_fullsized":"2024-04-25T18:59:27.913315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:37.419379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":228,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757934+0000","last_change":"2024-04-25T18:55:23.222160+0000","last_active":"2024-04-25T18:59:18.757934+0000","last_peered":"2024-04-25T18:59:18.757934+0000","last_clean":"2024-04-25T18:59:18.757934+0000","last_became_active":"2024-04-25T18:55:23.221812+0000","last_became_peered":"2024-04-25T18:55:23.221812+0000","last_unstale":"2024-04-25T18:59:18.757934+0000","last_undegraded":"2024-04-25T18:59:18.757934+0000","last_fullsized":"2024-04-25T18:59:18.757934+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:53:53.388764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"92'15","reported_seq":45,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757095+0000","last_change":"2024-04-25T18:55:19.179970+0000","last_active":"2024-04-25T18:59:18.757095+0000","last_peered":"2024-04-25T18:59:18.757095+0000","last_clean":"2024-04-25T18:59:18.757095+0000","last_became_active":"2024-04-25T18:55:19.179545+0000","last_became_peered":"2024-04-25T18:55:19.179545+0000","last_unstale":"2024-04-25T18:59:18.757095+0000","last_undegraded":"2024-04-25T18:59:18.757095+0000","last_fullsized":"2024-04-25T18:59:18.757095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:42.841661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923474+0000","last_change":"2024-04-25T18:55:23.216134+0000","last_active":"2024-04-25T18:59:18.923474+0000","last_peered":"2024-04-25T18:59:18.923474+0000","last_clean":"2024-04-25T18:59:18.923474+0000","last_became_active":"2024-04-25T18:55:23.215535+0000","last_became_peered":"2024-04-25T18:55:23.215535+0000","last_unstale":"2024-04-25T18:59:18.923474+0000","last_undegraded":"2024-04-25T18:59:18.923474+0000","last_fullsized":"2024-04-25T18:59:18.923474+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:08.042095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"92'2","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757699+0000","last_change":"2024-04-25T18:55:19.162437+0000","last_active":"2024-04-25T18:59:18.757699+0000","last_peered":"2024-04-25T18:59:18.757699+0000","last_clean":"2024-04-25T18:59:18.757699+0000","last_became_active":"2024-04-25T18:55:19.162005+0000","last_became_peered":"2024-04-25T18:55:19.162005+0000","last_unstale":"2024-04-25T18:59:18.757699+0000","last_undegraded":"2024-04-25T18:59:18.757699+0000","last_fullsized":"2024-04-25T18:59:18.757699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:50:53.967081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192027+0000","last_change":"2024-04-25T18:55:23.222345+0000","last_active":"2024-04-25T18:59:19.192027+0000","last_peered":"2024-04-25T18:59:19.192027+0000","last_clean":"2024-04-25T18:59:19.192027+0000","last_became_active":"2024-04-25T18:55:23.221142+0000","last_became_peered":"2024-04-25T18:55:23.221142+0000","last_unstale":"2024-04-25T18:59:19.192027+0000","last_undegraded":"2024-04-25T18:59:19.192027+0000","last_fullsized":"2024-04-25T18:59:19.192027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:39:53.823585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.907480+0000","last_change":"2024-04-25T18:55:19.181153+0000","last_active":"2024-04-25T18:59:18.907480+0000","last_peered":"2024-04-25T18:59:18.907480+0000","last_clean":"2024-04-25T18:59:18.907480+0000","last_became_active":"2024-04-25T18:55:19.180553+0000","last_became_peered":"2024-04-25T18:55:19.180553+0000","last_unstale":"2024-04-25T18:59:18.907480+0000","last_undegraded":"2024-04-25T18:59:18.907480+0000","last_fullsized":"2024-04-25T18:59:18.907480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:07.829617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928837+0000","last_change":"2024-04-25T18:55:23.238824+0000","last_active":"2024-04-25T18:59:18.928837+0000","last_peered":"2024-04-25T18:59:18.928837+0000","last_clean":"2024-04-25T18:59:18.928837+0000","last_became_active":"2024-04-25T18:55:23.238228+0000","last_became_peered":"2024-04-25T18:55:23.238228+0000","last_unstale":"2024-04-25T18:59:18.928837+0000","last_undegraded":"2024-04-25T18:59:18.928837+0000","last_fullsized":"2024-04-25T18:59:18.928837+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:16.279254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"92'1","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758314+0000","last_change":"2024-04-25T18:55:19.171186+0000","last_active":"2024-04-25T18:59:18.758314+0000","last_peered":"2024-04-25T18:59:18.758314+0000","last_clean":"2024-04-25T18:59:18.758314+0000","last_became_active":"2024-04-25T18:55:19.170800+0000","last_became_peered":"2024-04-25T18:55:19.170800+0000","last_unstale":"2024-04-25T18:59:18.758314+0000","last_undegraded":"2024-04-25T18:59:18.758314+0000","last_fullsized":"2024-04-25T18:59:18.758314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.599119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.928761+0000","last_change":"2024-04-25T18:55:23.236951+0000","last_active":"2024-04-25T18:59:18.928761+0000","last_peered":"2024-04-25T18:59:18.928761+0000","last_clean":"2024-04-25T18:59:18.928761+0000","last_became_active":"2024-04-25T18:55:23.236596+0000","last_became_peered":"2024-04-25T18:55:23.236596+0000","last_unstale":"2024-04-25T18:59:18.928761+0000","last_undegraded":"2024-04-25T18:59:18.928761+0000","last_fullsized":"2024-04-25T18:59:18.928761+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:14.634653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.923773+0000","last_change":"2024-04-25T18:55:23.211984+0000","last_active":"2024-04-25T18:59:18.923773+0000","last_peered":"2024-04-25T18:59:18.923773+0000","last_clean":"2024-04-25T18:59:18.923773+0000","last_became_active":"2024-04-25T18:55:23.211591+0000","last_became_peered":"2024-04-25T18:55:23.211591+0000","last_unstale":"2024-04-25T18:59:18.923773+0000","last_undegraded":"2024-04-25T18:59:18.923773+0000","last_fullsized":"2024-04-25T18:59:18.923773+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:09.024891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":258,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:28.914824+0000","last_change":"2024-04-25T18:55:19.180378+0000","last_active":"2024-04-25T18:59:28.914824+0000","last_peered":"2024-04-25T18:59:28.914824+0000","last_clean":"2024-04-25T18:59:28.914824+0000","last_became_active":"2024-04-25T18:55:19.179905+0000","last_became_peered":"2024-04-25T18:55:19.179905+0000","last_unstale":"2024-04-25T18:59:28.914824+0000","last_undegraded":"2024-04-25T18:59:28.914824+0000","last_fullsized":"2024-04-25T18:59:28.914824+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:36.434477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":223,"num_read_kb":223,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"94'8","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895977+0000","last_change":"2024-04-25T18:55:23.231489+0000","last_active":"2024-04-25T18:59:18.895977+0000","last_peered":"2024-04-25T18:59:18.895977+0000","last_clean":"2024-04-25T18:59:18.895977+0000","last_became_active":"2024-04-25T18:55:23.231049+0000","last_became_peered":"2024-04-25T18:55:23.231049+0000","last_unstale":"2024-04-25T18:59:18.895977+0000","last_undegraded":"2024-04-25T18:59:18.895977+0000","last_fullsized":"2024-04-25T18:59:18.895977+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:25.581217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"92'3","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895283+0000","last_change":"2024-04-25T18:55:19.171694+0000","last_active":"2024-04-25T18:59:18.895283+0000","last_peered":"2024-04-25T18:59:18.895283+0000","last_clean":"2024-04-25T18:59:18.895283+0000","last_became_active":"2024-04-25T18:55:19.171296+0000","last_became_peered":"2024-04-25T18:55:19.171296+0000","last_unstale":"2024-04-25T18:59:18.895283+0000","last_undegraded":"2024-04-25T18:59:18.895283+0000","last_fullsized":"2024-04-25T18:59:18.895283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:33.216980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"95'148","reported_seq":410,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:20.926517+0000","last_change":"2024-04-25T18:53:49.127506+0000","last_active":"2024-04-25T18:59:20.926517+0000","last_peered":"2024-04-25T18:59:20.926517+0000","last_clean":"2024-04-25T18:59:20.926517+0000","last_became_active":"2024-04-25T18:53:49.126974+0000","last_became_peered":"2024-04-25T18:53:49.126974+0000","last_unstale":"2024-04-25T18:59:20.926517+0000","last_undegraded":"2024-04-25T18:59:20.926517+0000","last_fullsized":"2024-04-25T18:59:20.926517+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:50:00.765468+0000","last_clean_scrub_stamp":"2024-04-25T18:50:00.765468+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:23.488525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757160+0000","last_change":"2024-04-25T18:55:23.211697+0000","last_active":"2024-04-25T18:59:18.757160+0000","last_peered":"2024-04-25T18:59:18.757160+0000","last_clean":"2024-04-25T18:59:18.757160+0000","last_became_active":"2024-04-25T18:55:23.211160+0000","last_became_peered":"2024-04-25T18:55:23.211160+0000","last_unstale":"2024-04-25T18:59:18.757160+0000","last_undegraded":"2024-04-25T18:59:18.757160+0000","last_fullsized":"2024-04-25T18:59:18.757160+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:27:03.452578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"92'3","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758287+0000","last_change":"2024-04-25T18:55:19.165955+0000","last_active":"2024-04-25T18:59:18.758287+0000","last_peered":"2024-04-25T18:59:18.758287+0000","last_clean":"2024-04-25T18:59:18.758287+0000","last_became_active":"2024-04-25T18:55:19.165594+0000","last_became_peered":"2024-04-25T18:55:19.165594+0000","last_unstale":"2024-04-25T18:59:18.758287+0000","last_undegraded":"2024-04-25T18:59:18.758287+0000","last_fullsized":"2024-04-25T18:59:18.758287+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:11.692374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"92'1","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757788+0000","last_change":"2024-04-25T18:55:19.182698+0000","last_active":"2024-04-25T18:59:18.757788+0000","last_peered":"2024-04-25T18:59:18.757788+0000","last_clean":"2024-04-25T18:59:18.757788+0000","last_became_active":"2024-04-25T18:55:19.182393+0000","last_became_peered":"2024-04-25T18:55:19.182393+0000","last_unstale":"2024-04-25T18:59:18.757788+0000","last_undegraded":"2024-04-25T18:59:18.757788+0000","last_fullsized":"2024-04-25T18:59:18.757788+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:31.513613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757347+0000","last_change":"2024-04-25T18:55:23.223240+0000","last_active":"2024-04-25T18:59:18.757347+0000","last_peered":"2024-04-25T18:59:18.757347+0000","last_clean":"2024-04-25T18:59:18.757347+0000","last_became_active":"2024-04-25T18:55:23.218253+0000","last_became_peered":"2024-04-25T18:55:23.218253+0000","last_unstale":"2024-04-25T18:59:18.757347+0000","last_undegraded":"2024-04-25T18:59:18.757347+0000","last_fullsized":"2024-04-25T18:59:18.757347+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:40.435633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"95'103","reported_seq":224,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:22.023135+0000","last_change":"2024-04-25T18:55:19.168535+0000","last_active":"2024-04-25T18:59:22.023135+0000","last_peered":"2024-04-25T18:59:22.023135+0000","last_clean":"2024-04-25T18:59:22.023135+0000","last_became_active":"2024-04-25T18:55:19.166735+0000","last_became_peered":"2024-04-25T18:55:19.166735+0000","last_unstale":"2024-04-25T18:59:22.023135+0000","last_undegraded":"2024-04-25T18:59:22.023135+0000","last_fullsized":"2024-04-25T18:59:22.023135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:50:14.008290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":189,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.896087+0000","last_change":"2024-04-25T18:55:23.233989+0000","last_active":"2024-04-25T18:59:18.896087+0000","last_peered":"2024-04-25T18:59:18.896087+0000","last_clean":"2024-04-25T18:59:18.896087+0000","last_became_active":"2024-04-25T18:55:23.226867+0000","last_became_peered":"2024-04-25T18:55:23.226867+0000","last_unstale":"2024-04-25T18:59:18.896087+0000","last_undegraded":"2024-04-25T18:59:18.896087+0000","last_fullsized":"2024-04-25T18:59:18.896087+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:36:02.314368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"92'4","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895339+0000","last_change":"2024-04-25T18:55:19.870412+0000","last_active":"2024-04-25T18:59:18.895339+0000","last_peered":"2024-04-25T18:59:18.895339+0000","last_clean":"2024-04-25T18:59:18.895339+0000","last_became_active":"2024-04-25T18:55:19.869970+0000","last_became_peered":"2024-04-25T18:55:19.869970+0000","last_unstale":"2024-04-25T18:59:18.895339+0000","last_undegraded":"2024-04-25T18:59:18.895339+0000","last_fullsized":"2024-04-25T18:59:18.895339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:51:22.236417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.193471+0000","last_change":"2024-04-25T18:55:23.239172+0000","last_active":"2024-04-25T18:59:19.193471+0000","last_peered":"2024-04-25T18:59:19.193471+0000","last_clean":"2024-04-25T18:59:19.193471+0000","last_became_active":"2024-04-25T18:55:23.238732+0000","last_became_peered":"2024-04-25T18:55:23.238732+0000","last_unstale":"2024-04-25T18:59:19.193471+0000","last_undegraded":"2024-04-25T18:59:19.193471+0000","last_fullsized":"2024-04-25T18:59:19.193471+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:27.459659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132316+0000","last_change":"2024-04-25T18:55:19.179687+0000","last_active":"2024-04-25T18:59:19.132316+0000","last_peered":"2024-04-25T18:59:19.132316+0000","last_clean":"2024-04-25T18:59:19.132316+0000","last_became_active":"2024-04-25T18:55:19.179225+0000","last_became_peered":"2024-04-25T18:55:19.179225+0000","last_unstale":"2024-04-25T18:59:19.132316+0000","last_undegraded":"2024-04-25T18:59:19.132316+0000","last_fullsized":"2024-04-25T18:59:19.132316+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:16:16.266021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757162+0000","last_change":"2024-04-25T18:55:23.221379+0000","last_active":"2024-04-25T18:59:18.757162+0000","last_peered":"2024-04-25T18:59:18.757162+0000","last_clean":"2024-04-25T18:59:18.757162+0000","last_became_active":"2024-04-25T18:55:23.221037+0000","last_became_peered":"2024-04-25T18:55:23.221037+0000","last_unstale":"2024-04-25T18:59:18.757162+0000","last_undegraded":"2024-04-25T18:59:18.757162+0000","last_fullsized":"2024-04-25T18:59:18.757162+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:42:27.581084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"92'3","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895320+0000","last_change":"2024-04-25T18:55:19.171298+0000","last_active":"2024-04-25T18:59:18.895320+0000","last_peered":"2024-04-25T18:59:18.895320+0000","last_clean":"2024-04-25T18:59:18.895320+0000","last_became_active":"2024-04-25T18:55:19.170738+0000","last_became_peered":"2024-04-25T18:55:19.170738+0000","last_unstale":"2024-04-25T18:59:18.895320+0000","last_undegraded":"2024-04-25T18:59:18.895320+0000","last_fullsized":"2024-04-25T18:59:18.895320+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.236331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191708+0000","last_change":"2024-04-25T18:55:23.226577+0000","last_active":"2024-04-25T18:59:19.191708+0000","last_peered":"2024-04-25T18:59:19.191708+0000","last_clean":"2024-04-25T18:59:19.191708+0000","last_became_active":"2024-04-25T18:55:23.226198+0000","last_became_peered":"2024-04-25T18:55:23.226198+0000","last_unstale":"2024-04-25T18:59:19.191708+0000","last_undegraded":"2024-04-25T18:59:19.191708+0000","last_fullsized":"2024-04-25T18:59:19.191708+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:34:59.124523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"95'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129472+0000","last_change":"2024-04-25T18:55:19.171917+0000","last_active":"2024-04-25T18:59:19.129472+0000","last_peered":"2024-04-25T18:59:19.129472+0000","last_clean":"2024-04-25T18:59:19.129472+0000","last_became_active":"2024-04-25T18:55:19.171506+0000","last_became_peered":"2024-04-25T18:55:19.171506+0000","last_unstale":"2024-04-25T18:59:19.129472+0000","last_undegraded":"2024-04-25T18:59:19.129472+0000","last_fullsized":"2024-04-25T18:59:19.129472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:24:53.446504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895608+0000","last_change":"2024-04-25T18:55:23.237828+0000","last_active":"2024-04-25T18:59:18.895608+0000","last_peered":"2024-04-25T18:59:18.895608+0000","last_clean":"2024-04-25T18:59:18.895608+0000","last_became_active":"2024-04-25T18:55:23.237310+0000","last_became_peered":"2024-04-25T18:55:23.237310+0000","last_unstale":"2024-04-25T18:59:18.895608+0000","last_undegraded":"2024-04-25T18:59:18.895608+0000","last_fullsized":"2024-04-25T18:59:18.895608+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:10.987032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"92'4","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757414+0000","last_change":"2024-04-25T18:55:19.163824+0000","last_active":"2024-04-25T18:59:18.757414+0000","last_peered":"2024-04-25T18:59:18.757414+0000","last_clean":"2024-04-25T18:59:18.757414+0000","last_became_active":"2024-04-25T18:55:19.163465+0000","last_became_peered":"2024-04-25T18:55:19.163465+0000","last_unstale":"2024-04-25T18:59:18.757414+0000","last_undegraded":"2024-04-25T18:59:18.757414+0000","last_fullsized":"2024-04-25T18:59:18.757414+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:51:38.347100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"94'5","reported_seq":28,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758396+0000","last_change":"2024-04-25T18:55:23.224648+0000","last_active":"2024-04-25T18:59:18.758396+0000","last_peered":"2024-04-25T18:59:18.758396+0000","last_clean":"2024-04-25T18:59:18.758396+0000","last_became_active":"2024-04-25T18:55:23.224323+0000","last_became_peered":"2024-04-25T18:55:23.224323+0000","last_unstale":"2024-04-25T18:59:18.758396+0000","last_undegraded":"2024-04-25T18:59:18.758396+0000","last_fullsized":"2024-04-25T18:59:18.758396+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:56.117792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"92'3","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129721+0000","last_change":"2024-04-25T18:55:19.870213+0000","last_active":"2024-04-25T18:59:19.129721+0000","last_peered":"2024-04-25T18:59:19.129721+0000","last_clean":"2024-04-25T18:59:19.129721+0000","last_became_active":"2024-04-25T18:55:19.869756+0000","last_became_peered":"2024-04-25T18:55:19.869756+0000","last_unstale":"2024-04-25T18:59:19.129721+0000","last_undegraded":"2024-04-25T18:59:19.129721+0000","last_fullsized":"2024-04-25T18:59:19.129721+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:44.695876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758609+0000","last_change":"2024-04-25T18:55:23.217094+0000","last_active":"2024-04-25T18:59:18.758609+0000","last_peered":"2024-04-25T18:59:18.758609+0000","last_clean":"2024-04-25T18:59:18.758609+0000","last_became_active":"2024-04-25T18:55:23.216696+0000","last_became_peered":"2024-04-25T18:55:23.216696+0000","last_unstale":"2024-04-25T18:59:18.758609+0000","last_undegraded":"2024-04-25T18:59:18.758609+0000","last_fullsized":"2024-04-25T18:59:18.758609+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:07.049349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"95'65","reported_seq":125,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:21.488027+0000","last_change":"2024-04-25T18:55:19.166377+0000","last_active":"2024-04-25T18:59:21.488027+0000","last_peered":"2024-04-25T18:59:21.488027+0000","last_clean":"2024-04-25T18:59:21.488027+0000","last_became_active":"2024-04-25T18:55:19.166036+0000","last_became_peered":"2024-04-25T18:55:19.166036+0000","last_unstale":"2024-04-25T18:59:21.488027+0000","last_undegraded":"2024-04-25T18:59:21.488027+0000","last_fullsized":"2024-04-25T18:59:21.488027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:04.761919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"94'7","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758305+0000","last_change":"2024-04-25T18:55:23.236893+0000","last_active":"2024-04-25T18:59:18.758305+0000","last_peered":"2024-04-25T18:59:18.758305+0000","last_clean":"2024-04-25T18:59:18.758305+0000","last_became_active":"2024-04-25T18:55:23.236488+0000","last_became_peered":"2024-04-25T18:55:23.236488+0000","last_unstale":"2024-04-25T18:59:18.758305+0000","last_undegraded":"2024-04-25T18:59:18.758305+0000","last_fullsized":"2024-04-25T18:59:18.758305+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:07:44.605041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"92'2","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757331+0000","last_change":"2024-04-25T18:55:19.175076+0000","last_active":"2024-04-25T18:59:18.757331+0000","last_peered":"2024-04-25T18:59:18.757331+0000","last_clean":"2024-04-25T18:59:18.757331+0000","last_became_active":"2024-04-25T18:55:19.174547+0000","last_became_peered":"2024-04-25T18:55:19.174547+0000","last_unstale":"2024-04-25T18:59:18.757331+0000","last_undegraded":"2024-04-25T18:59:18.757331+0000","last_fullsized":"2024-04-25T18:59:18.757331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:18.133791+0000","last_clean_scrub_stamp":"2024-04-25T18:55:18.133791+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:14:43.791068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.758728+0000","last_change":"2024-04-25T18:55:23.219665+0000","last_active":"2024-04-25T18:59:18.758728+0000","last_peered":"2024-04-25T18:59:18.758728+0000","last_clean":"2024-04-25T18:59:18.758728+0000","last_became_active":"2024-04-25T18:55:23.219264+0000","last_became_peered":"2024-04-25T18:55:23.219264+0000","last_unstale":"2024-04-25T18:59:18.758728+0000","last_undegraded":"2024-04-25T18:59:18.758728+0000","last_fullsized":"2024-04-25T18:59:18.758728+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:25:19.008120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"94'9","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191796+0000","last_change":"2024-04-25T18:55:23.220290+0000","last_active":"2024-04-25T18:59:19.191796+0000","last_peered":"2024-04-25T18:59:19.191796+0000","last_clean":"2024-04-25T18:59:19.191796+0000","last_became_active":"2024-04-25T18:55:23.219131+0000","last_became_peered":"2024-04-25T18:55:23.219131+0000","last_unstale":"2024-04-25T18:59:19.191796+0000","last_undegraded":"2024-04-25T18:59:19.191796+0000","last_fullsized":"2024-04-25T18:59:19.191796+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:25:25.678525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"94'12","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129447+0000","last_change":"2024-04-25T18:55:23.216415+0000","last_active":"2024-04-25T18:59:19.129447+0000","last_peered":"2024-04-25T18:59:19.129447+0000","last_clean":"2024-04-25T18:59:19.129447+0000","last_became_active":"2024-04-25T18:55:23.216066+0000","last_became_peered":"2024-04-25T18:55:23.216066+0000","last_unstale":"2024-04-25T18:59:19.129447+0000","last_undegraded":"2024-04-25T18:59:19.129447+0000","last_fullsized":"2024-04-25T18:59:19.129447+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:54.633216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129390+0000","last_change":"2024-04-25T18:55:23.234034+0000","last_active":"2024-04-25T18:59:19.129390+0000","last_peered":"2024-04-25T18:59:19.129390+0000","last_clean":"2024-04-25T18:59:19.129390+0000","last_became_active":"2024-04-25T18:55:23.233639+0000","last_became_peered":"2024-04-25T18:55:23.233639+0000","last_unstale":"2024-04-25T18:59:19.129390+0000","last_undegraded":"2024-04-25T18:59:19.129390+0000","last_fullsized":"2024-04-25T18:59:19.129390+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:57.117958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"94'7","reported_seq":36,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.192719+0000","last_change":"2024-04-25T18:56:17.584511+0000","last_active":"2024-04-25T18:59:19.192719+0000","last_peered":"2024-04-25T18:59:19.192719+0000","last_clean":"2024-04-25T18:59:19.192719+0000","last_became_active":"2024-04-25T18:56:17.583959+0000","last_became_peered":"2024-04-25T18:56:17.583959+0000","last_unstale":"2024-04-25T18:59:19.192719+0000","last_undegraded":"2024-04-25T18:59:19.192719+0000","last_fullsized":"2024-04-25T18:59:19.192719+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.622823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"94'4","reported_seq":27,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757493+0000","last_change":"2024-04-25T18:55:23.238068+0000","last_active":"2024-04-25T18:59:18.757493+0000","last_peered":"2024-04-25T18:59:18.757493+0000","last_clean":"2024-04-25T18:59:18.757493+0000","last_became_active":"2024-04-25T18:55:23.237773+0000","last_became_peered":"2024-04-25T18:55:23.237773+0000","last_unstale":"2024-04-25T18:59:18.757493+0000","last_undegraded":"2024-04-25T18:59:18.757493+0000","last_fullsized":"2024-04-25T18:59:18.757493+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:49.688576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"94'6","reported_seq":29,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191981+0000","last_change":"2024-04-25T18:55:23.215463+0000","last_active":"2024-04-25T18:59:19.191981+0000","last_peered":"2024-04-25T18:59:19.191981+0000","last_clean":"2024-04-25T18:59:19.191981+0000","last_became_active":"2024-04-25T18:55:23.215068+0000","last_became_peered":"2024-04-25T18:55:23.215068+0000","last_unstale":"2024-04-25T18:59:19.191981+0000","last_undegraded":"2024-04-25T18:59:19.191981+0000","last_fullsized":"2024-04-25T18:59:19.191981+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:38:53.450828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"94'15","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.129793+0000","last_change":"2024-04-25T18:55:23.214362+0000","last_active":"2024-04-25T18:59:19.129793+0000","last_peered":"2024-04-25T18:59:19.129793+0000","last_clean":"2024-04-25T18:59:19.129793+0000","last_became_active":"2024-04-25T18:55:23.214035+0000","last_became_peered":"2024-04-25T18:55:23.214035+0000","last_unstale":"2024-04-25T18:59:19.129793+0000","last_undegraded":"2024-04-25T18:59:19.129793+0000","last_fullsized":"2024-04-25T18:59:19.129793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:33.386859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"94'2","reported_seq":25,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.896015+0000","last_change":"2024-04-25T18:55:23.235709+0000","last_active":"2024-04-25T18:59:18.896015+0000","last_peered":"2024-04-25T18:59:18.896015+0000","last_clean":"2024-04-25T18:59:18.896015+0000","last_became_active":"2024-04-25T18:55:23.226089+0000","last_became_peered":"2024-04-25T18:55:23.226089+0000","last_unstale":"2024-04-25T18:59:18.896015+0000","last_undegraded":"2024-04-25T18:59:18.896015+0000","last_fullsized":"2024-04-25T18:59:18.896015+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:57.161365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.756774+0000","last_change":"2024-04-25T18:56:17.561546+0000","last_active":"2024-04-25T18:59:18.756774+0000","last_peered":"2024-04-25T18:59:18.756774+0000","last_clean":"2024-04-25T18:59:18.756774+0000","last_became_active":"2024-04-25T18:56:17.559767+0000","last_became_peered":"2024-04-25T18:56:17.559767+0000","last_unstale":"2024-04-25T18:59:18.756774+0000","last_undegraded":"2024-04-25T18:59:18.756774+0000","last_fullsized":"2024-04-25T18:59:18.756774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:18:59.512008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.895271+0000","last_change":"2024-04-25T18:55:23.227620+0000","last_active":"2024-04-25T18:59:18.895271+0000","last_peered":"2024-04-25T18:59:18.895271+0000","last_clean":"2024-04-25T18:59:18.895271+0000","last_became_active":"2024-04-25T18:55:23.227236+0000","last_became_peered":"2024-04-25T18:55:23.227236+0000","last_unstale":"2024-04-25T18:59:18.895271+0000","last_undegraded":"2024-04-25T18:59:18.895271+0000","last_fullsized":"2024-04-25T18:59:18.895271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:32.707619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"94'10","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.132494+0000","last_change":"2024-04-25T18:55:23.238393+0000","last_active":"2024-04-25T18:59:19.132494+0000","last_peered":"2024-04-25T18:59:19.132494+0000","last_clean":"2024-04-25T18:59:19.132494+0000","last_became_active":"2024-04-25T18:55:23.237726+0000","last_became_peered":"2024-04-25T18:55:23.237726+0000","last_unstale":"2024-04-25T18:59:19.132494+0000","last_undegraded":"2024-04-25T18:59:19.132494+0000","last_fullsized":"2024-04-25T18:59:19.132494+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:48.543014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"94'11","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:18.757312+0000","last_change":"2024-04-25T18:55:23.231374+0000","last_active":"2024-04-25T18:59:18.757312+0000","last_peered":"2024-04-25T18:59:18.757312+0000","last_clean":"2024-04-25T18:59:18.757312+0000","last_became_active":"2024-04-25T18:55:23.230976+0000","last_became_peered":"2024-04-25T18:55:23.230976+0000","last_unstale":"2024-04-25T18:59:18.757312+0000","last_undegraded":"2024-04-25T18:59:18.757312+0000","last_fullsized":"2024-04-25T18:59:18.757312+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:12:06.539013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"94'8","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T18:59:19.191923+0000","last_change":"2024-04-25T18:55:23.217332+0000","last_active":"2024-04-25T18:59:19.191923+0000","last_peered":"2024-04-25T18:59:19.191923+0000","last_clean":"2024-04-25T18:59:19.191923+0000","last_became_active":"2024-04-25T18:55:23.216968+0000","last_became_peered":"2024-04-25T18:55:23.216968+0000","last_unstale":"2024-04-25T18:59:19.191923+0000","last_undegraded":"2024-04-25T18:59:19.191923+0000","last_fullsized":"2024-04-25T18:59:19.191923+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T18:55:22.170037+0000","last_clean_scrub_stamp":"2024-04-25T18:55:22.170037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:31.000083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":801253,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1724,"num_read_kb":1082,"num_write":429,"num_write_kb":2365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":2403759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":316,"ondisk_log_size":316,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":84,"seq":360777252929,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":507776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":76,"seq":326417514568,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37492,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707660,"statfs":{"total":95995035648,"available":95956643840,"internally_reserved":0,"allocated":2084864,"data_stored":1502336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":70,"seq":300647710795,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36756,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708396,"statfs":{"total":95995035648,"available":95957397504,"internally_reserved":0,"allocated":1454080,"data_stored":901194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":62,"seq":266287972435,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1454080,"data_stored":901114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":54,"seq":231928234072,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36952,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708200,"statfs":{"total":95995035648,"available":95957196800,"internally_reserved":0,"allocated":1654784,"data_stored":1101283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430299,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1048576,"data_stored":507367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1122304,"data_stored":581509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1069056,"data_stored":508638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":584139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117101,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37204,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707948,"statfs":{"total":95995035648,"available":95956938752,"internally_reserved":0,"allocated":1789952,"data_stored":1229126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":35,"seq":150323855463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1052672,"data_stored":508336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":40,"seq":171798691938,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29088,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716064,"statfs":{"total":95995035648,"available":95965249536,"internally_reserved":0,"allocated":1998848,"data_stored":1428667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":74349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":76979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":393954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":404691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T18:59:34.249 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:59:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:34 smithi045 ceph-mon[27776]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:59:34.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:34 smithi062 ceph-mon[32844]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:59:34.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:34 smithi079 ceph-mon[33002]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:59:34.751 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.391+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2904556751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8c4108d50 msgr2=0x7fb8c4115210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.391+0000 7fb8c8e00640 1 --2- 172.21.15.45:0/2904556751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8c4108d50 0x7fb8c4115210 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009be0 tx=0x7fb8bc02a750 comp rx=0 tx=0).stop 2024-04-25T18:59:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.391+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2904556751 shutdown_connections 2024-04-25T18:59:35.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.391+0000 7fb8c8e00640 1 --2- 172.21.15.45:0/2904556751 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8c4108d50 0x7fb8c4115210 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.391+0000 7fb8c8e00640 1 --2- 172.21.15.45:0/2904556751 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c41083b0 0x7fb8c4108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.391+0000 7fb8c8e00640 1 --2- 172.21.15.45:0/2904556751 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c410e470 0x7fb8c410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.391+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2904556751 >> 172.21.15.45:0/2904556751 conn(0x7fb8c4077b00 msgr2=0x7fb8c4079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.391+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2904556751 shutdown_connections 2024-04-25T18:59:35.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.391+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2904556751 wait complete. 2024-04-25T18:59:35.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.392+0000 7fb8c8e00640 1 Processor -- start 2024-04-25T18:59:35.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.392+0000 7fb8c8e00640 1 -- start start 2024-04-25T18:59:35.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8c8e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c41083b0 0x7fb8c41a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8c8e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8c4108d50 0x7fb8c41a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8c8e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c410e470 0x7fb8c419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8c8e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c41185e0 con 0x7fb8c41083b0 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8c8e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c4118460 con 0x7fb8c410e470 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8c8e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c4118760 con 0x7fb8c4108d50 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c41083b0 0x7fb8c41a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c410e470 0x7fb8c419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8c3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c410e470 0x7fb8c419e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:54828/0 (socket says 172.21.15.45:54828) 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8c3e00640 1 -- 172.21.15.45:0/2875553770 learned_addr learned my addr 172.21.15.45:0/2875553770 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.393+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8c4108d50 0x7fb8c41a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.394+0000 7fb8c3e00640 1 -- 172.21.15.45:0/2875553770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8c4108d50 msgr2=0x7fb8c41a4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.394+0000 7fb8c3e00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8c4108d50 0x7fb8c41a4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.394+0000 7fb8c3e00640 1 -- 172.21.15.45:0/2875553770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c41083b0 msgr2=0x7fb8c41a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.394+0000 7fb8c3e00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c41083b0 0x7fb8c41a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.394+0000 7fb8c3e00640 1 -- 172.21.15.45:0/2875553770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c419f040 con 0x7fb8c410e470 2024-04-25T18:59:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.394+0000 7fb8c3e00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c410e470 0x7fb8c419e8c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009b30 tx=0x7fb8bc003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.394+0000 7fb8c0c00640 1 -- 172.21.15.45:0/2875553770 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc041020 con 0x7fb8c410e470 2024-04-25T18:59:35.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.394+0000 7fb8c0c00640 1 -- 172.21.15.45:0/2875553770 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8bc0318f0 con 0x7fb8c410e470 2024-04-25T18:59:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.394+0000 7fb8c0c00640 1 -- 172.21.15.45:0/2875553770 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8bc036040 con 0x7fb8c410e470 2024-04-25T18:59:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.394+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2875553770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c419f2d0 con 0x7fb8c410e470 2024-04-25T18:59:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.395+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2875553770 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c41ab580 con 0x7fb8c410e470 2024-04-25T18:59:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.399+0000 7fb8cd368740 1 Processor -- start 2024-04-25T18:59:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.400+0000 7fb8c0c00640 1 -- 172.21.15.45:0/2875553770 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fb8bc044020 con 0x7fb8c410e470 2024-04-25T18:59:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.400+0000 7fb8cd368740 1 -- start start 2024-04-25T18:59:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.400+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248acd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.400+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ad2d0 0x5607248b7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.400+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248b7b90 0x5607248b9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.400+0000 7fb8cd368740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560724812900 con 0x5607248ac9b0 2024-04-25T18:59:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.400+0000 7fb8cd368740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5607247b8ef0 con 0x5607248b7b90 2024-04-25T18:59:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.400+0000 7fb8cd368740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56072480e8c0 con 0x5607248ad2d0 2024-04-25T18:59:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.401+0000 7fb8bbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.401+0000 7fb8bbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248acd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36526/0 (socket says 172.21.15.45:36526) 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.401+0000 7fb8bbe00640 1 -- 172.21.15.45:0/1652666787 learned_addr learned my addr 172.21.15.45:0/1652666787 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.401+0000 7fb8c3400640 1 --2- 172.21.15.45:0/1652666787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ad2d0 0x5607248b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.401+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/1652666787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248b7b90 0x5607248b9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.401+0000 7fb8c3400640 1 -- 172.21.15.45:0/1652666787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248b7b90 msgr2=0x5607248b9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.401+0000 7fb8c3400640 1 --2- 172.21.15.45:0/1652666787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248b7b90 0x5607248b9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.401+0000 7fb8c3400640 1 -- 172.21.15.45:0/1652666787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 msgr2=0x5607248acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.401+0000 7fb8c3400640 1 --2- 172.21.15.45:0/1652666787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.401+0000 7fb8c3400640 1 -- 172.21.15.45:0/1652666787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560724525210 con 0x5607248ad2d0 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.402+0000 7fb8c3400640 1 --2- 172.21.15.45:0/1652666787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ad2d0 0x5607248b7650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009530 tx=0x7fb8b4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4022ffd9f30b89c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.402+0000 7fb8c0c00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb8a80a3ea0 0x7fb8a80a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.402+0000 7fb8b8c00640 1 -- 172.21.15.45:0/1652666787 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4004220 con 0x5607248ad2d0 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.402+0000 7fb8c0c00640 1 -- 172.21.15.45:0/2875553770 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fb8bc0f14a0 con 0x7fb8c410e470 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.402+0000 7fb8b8c00640 1 -- 172.21.15.45:0/1652666787 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8b40043c0 con 0x5607248ad2d0 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.402+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb8a80a3ea0 0x7fb8a80a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.402+0000 7fb8b8c00640 1 -- 172.21.15.45:0/1652666787 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4019630 con 0x5607248ad2d0 2024-04-25T18:59:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8cd368740 1 -- 172.21.15.45:0/1652666787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ad2d0 msgr2=0x5607248b7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1652666787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ad2d0 0x5607248b7650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb8b4009530 tx=0x7fb8b4011810 comp rx=0 tx=0).stop 2024-04-25T18:59:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb8a80a3ea0 0x7fb8a80a6360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x5607248aa2e0 tx=0x7fb8ac0097a0 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8cd368740 1 -- 172.21.15.45:0/1652666787 shutdown_connections 2024-04-25T18:59:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1652666787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248b7b90 0x5607248b9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1652666787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ad2d0 0x5607248b7650 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1652666787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8cd368740 1 -- 172.21.15.45:0/1652666787 >> 172.21.15.45:0/1652666787 conn(0x5607248a8ec0 msgr2=0x5607248a92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8cd368740 1 -- 172.21.15.45:0/1652666787 shutdown_connections 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8cd368740 1 -- 172.21.15.45:0/1652666787 wait complete. 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.403+0000 7fb8cd368740 1 -- start start 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.404+0000 7fb8cd368740 1 Processor -- start 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.404+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.404+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.404+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 0x5607248b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.404+0000 7fb8cd368740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5607248ba7c0 con 0x5607248ac9b0 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.404+0000 7fb8cd368740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5607247cc310 con 0x5607248ad2d0 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.404+0000 7fb8cd368740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5607248ba940 con 0x5607248b7b90 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.404+0000 7fb8cd368740 10 client.0 _ll_register_callbacks cb 0x56072479acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36542/0 (socket says 172.21.15.45:36542) 2024-04-25T18:59:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3400640 1 -- 172.21.15.45:0/1156491717 learned_addr learned my addr 172.21.15.45:0/1156491717 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3e00640 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 0x5607248b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3400640 1 -- 172.21.15.45:0/1156491717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 msgr2=0x5607248b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3400640 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 0x5607248b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3400640 1 -- 172.21.15.45:0/1156491717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 msgr2=0x5607248b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3400640 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3400640 1 -- 172.21.15.45:0/1156491717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607245f1620 con 0x5607248ac9b0 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3e00640 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 0x5607248b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8c3400640 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248b4a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40052d0 tx=0x7fb8b4005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59c5ac961aad6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.405+0000 7fb8b1600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.406+0000 7fb8b1600640 1 -- 172.21.15.45:0/1156491717 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b4028070 con 0x5607248ac9b0 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.406+0000 7fb8b1600640 1 -- 172.21.15.45:0/1156491717 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fb8b40035f0 con 0x5607248ac9b0 2024-04-25T18:59:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.406+0000 7fb8cd368740 1 -- 172.21.15.45:0/1156491717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8b40043c0 con 0x5607248ac9b0 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.406+0000 7fb8b1600640 1 -- 172.21.15.45:0/1156491717 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb8b4034050 con 0x5607248ac9b0 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.406+0000 7fb8cd368740 10 client.15057 fetch_fsmap learned FSMap version 26 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.406+0000 7fb8cd368740 1 -- 172.21.15.45:0/1156491717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5607248c69c0 con 0x5607248ac9b0 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.406+0000 7fb8b1600640 1 -- 172.21.15.45:0/1156491717 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8b40043c0 con 0x5607248ac9b0 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.407+0000 7fb8b1600640 1 -- 172.21.15.45:0/1156491717 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb8b4004a00 con 0x5607248ac9b0 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.407+0000 7fb8cd368740 10 client.15057 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.407+0000 7fb8cd368740 10 client.15057 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.407+0000 7fb8cd368740 20 client.15057 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.407+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607248c7dd0 0x5607248ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.407+0000 7fb8cd368740 4 client.15057 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.407+0000 7fb8cd368740 1 -- 172.21.15.45:0/1156491717 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607248268b0 con 0x5607248c7dd0 2024-04-25T18:59:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.407+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607248c7dd0 0x5607248ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.408+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607248c7dd0 0x5607248ca1c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3ad2ce10e08e188 server_cookie=877c650351a81262 in_seq=0 out_seq=0 2024-04-25T18:59:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.408+0000 7fb8b1600640 10 client.15057 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8b1600640 1 -- 172.21.15.45:0/1156491717 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607248268b0 con 0x5607248c7dd0 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8b1600640 10 client.15057 handle_command_reply: tid=0 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 client.15057 shutdown 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 -- 172.21.15.45:0/1156491717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 msgr2=0x5607248b4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248b4a20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb8b40052d0 tx=0x7fb8b4005300 comp rx=0 tx=0).stop 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 -- 172.21.15.45:0/1156491717 shutdown_connections 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607248c7dd0 0x5607248ca1c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 0x5607248b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1156491717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248b4a20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8b1600640 0 client.15057 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 -- 172.21.15.45:0/1156491717 >> 172.21.15.45:0/1156491717 conn(0x5607248a8ec0 msgr2=0x5607248aec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 -- 172.21.15.45:0/1156491717 shutdown_connections 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.409+0000 7fb8cd368740 1 -- 172.21.15.45:0/1156491717 wait complete. 2024-04-25T18:59:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.410+0000 7fb8cd368740 20 client.15057 trim_cache size 0 max 16384 2024-04-25T18:59:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.426+0000 7fb8cd368740 1 Processor -- start 2024-04-25T18:59:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.426+0000 7fb8cd368740 1 -- start start 2024-04-25T18:59:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.426+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ac9b0 0x5607248c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.426+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.426+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248b7b90 0x5607248b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.426+0000 7fb8cd368740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c41b6830 con 0x5607248b7b90 2024-04-25T18:59:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.426+0000 7fb8cd368740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c41b4100 con 0x5607248ad2d0 2024-04-25T18:59:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.426+0000 7fb8cd368740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c41b1f50 con 0x5607248ac9b0 2024-04-25T18:59:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:54852/0 (socket says 172.21.15.45:54852) 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8bbe00640 1 -- 172.21.15.45:0/1372723715 learned_addr learned my addr 172.21.15.45:0/1372723715 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8c3400640 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ac9b0 0x5607248c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8c3e00640 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248b7b90 0x5607248b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8bbe00640 1 -- 172.21.15.45:0/1372723715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ac9b0 msgr2=0x5607248c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ac9b0 0x5607248c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8bbe00640 1 -- 172.21.15.45:0/1372723715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248b7b90 msgr2=0x5607248b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248b7b90 0x5607248b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8bbe00640 1 -- 172.21.15.45:0/1372723715 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607248a4820 con 0x5607248ad2d0 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8c3400640 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ac9b0 0x5607248c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8c3e00640 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248b7b90 0x5607248b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.427+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248b3b80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009a60 tx=0x7fb8ac0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=583b95f17ef54d69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8b3e00640 1 -- 172.21.15.45:0/1372723715 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac021ab0 con 0x5607248ad2d0 2024-04-25T18:59:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8b3e00640 1 -- 172.21.15.45:0/1372723715 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ac021c50 con 0x5607248ad2d0 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8b3e00640 1 -- 172.21.15.45:0/1372723715 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac0035f0 con 0x5607248ad2d0 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8cd368740 1 -- 172.21.15.45:0/1372723715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 msgr2=0x5607248b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248b3b80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009a60 tx=0x7fb8ac0046c0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8cd368740 1 -- 172.21.15.45:0/1372723715 shutdown_connections 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248b7b90 0x5607248b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248b3b80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8cd368740 1 --2- 172.21.15.45:0/1372723715 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248ac9b0 0x5607248c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8cd368740 1 -- 172.21.15.45:0/1372723715 >> 172.21.15.45:0/1372723715 conn(0x5607248a8ec0 msgr2=0x5607248b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8cd368740 1 -- 172.21.15.45:0/1372723715 shutdown_connections 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.428+0000 7fb8cd368740 1 -- 172.21.15.45:0/1372723715 wait complete. 2024-04-25T18:59:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.429+0000 7fb8cd368740 1 -- start start 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.429+0000 7fb8cd368740 1 Processor -- start 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.429+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248bcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.429+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248bd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8cd368740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 0x56072492c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8cd368740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c41b25c0 con 0x5607248ac9b0 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8cd368740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c41b3a30 con 0x5607248ad2d0 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8cd368740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c41b2c90 con 0x5607248b7b90 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8cd368740 10 client.0 _ll_register_callbacks cb 0x7fb8ac009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248bcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8bbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248bd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8c3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248bcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36550/0 (socket says 172.21.15.45:36550) 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8c3400640 1 -- 172.21.15.45:0/813603077 learned_addr learned my addr 172.21.15.45:0/813603077 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8c3e00640 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 0x56072492c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8bbe00640 1 -- 172.21.15.45:0/813603077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 msgr2=0x56072492c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 0x56072492c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8bbe00640 1 -- 172.21.15.45:0/813603077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 msgr2=0x5607248bcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248bcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8bbe00640 1 -- 172.21.15.45:0/813603077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56072492c7c0 con 0x5607248ad2d0 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.430+0000 7fb8c3400640 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248bcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.431+0000 7fb8bbe00640 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248bd2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5607248c4400 tx=0x7fb8ac02e790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f1bc15211f5360a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.431+0000 7fb8b1600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.431+0000 7fb8b1600640 1 -- 172.21.15.45:0/813603077 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac035070 con 0x5607248ad2d0 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.431+0000 7fb8cd368740 1 -- 172.21.15.45:0/813603077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8ac021c50 con 0x5607248ad2d0 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.431+0000 7fb8b1600640 1 -- 172.21.15.45:0/813603077 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fb8ac004020 con 0x5607248ad2d0 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.431+0000 7fb8b1600640 1 -- 172.21.15.45:0/813603077 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb8ac004240 con 0x5607248ad2d0 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.431+0000 7fb8cd368740 10 client.25184 fetch_fsmap learned FSMap version 26 2024-04-25T18:59:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.431+0000 7fb8cd368740 1 -- 172.21.15.45:0/813603077 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56072492da40 con 0x5607248ad2d0 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.432+0000 7fb8b1600640 1 -- 172.21.15.45:0/813603077 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8ac021c50 con 0x5607248ad2d0 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.432+0000 7fb8b1600640 1 -- 172.21.15.45:0/813603077 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fb8ac010430 con 0x5607248ad2d0 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.432+0000 7fb8cd368740 10 client.25184 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.432+0000 7fb8cd368740 10 client.25184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.432+0000 7fb8cd368740 20 client.25184 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.432+0000 7fb8cd368740 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56072492e110 0x560724930500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.432+0000 7fb8cd368740 4 client.25184 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.432+0000 7fb8cd368740 1 -- 172.21.15.45:0/813603077 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560724820320 con 0x56072492e110 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.432+0000 7fb8c3400640 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56072492e110 0x560724930500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.433+0000 7fb8c3400640 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56072492e110 0x560724930500 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709849f779e0c3c8 server_cookie=fa57370dfe65e6f8 in_seq=0 out_seq=0 2024-04-25T18:59:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.433+0000 7fb8b1600640 10 client.25184 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8b1600640 1 -- 172.21.15.45:0/813603077 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560724820320 con 0x56072492e110 2024-04-25T18:59:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8b1600640 10 client.25184 handle_command_reply: tid=0 2024-04-25T18:59:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8cd368740 1 client.25184 shutdown 2024-04-25T18:59:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8cd368740 1 -- 172.21.15.45:0/813603077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 msgr2=0x5607248bd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8cd368740 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248bd2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5607248c4400 tx=0x7fb8ac02e790 comp rx=0 tx=0).stop 2024-04-25T18:59:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8cd368740 1 -- 172.21.15.45:0/813603077 shutdown_connections 2024-04-25T18:59:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8cd368740 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56072492e110 0x560724930500 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.446 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8cd368740 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607248b7b90 0x56072492c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8cd368740 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607248ad2d0 0x5607248bd2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8cd368740 1 --2- 172.21.15.45:0/813603077 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607248ac9b0 0x5607248bcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8b1600640 0 client.25184 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8cd368740 1 -- 172.21.15.45:0/813603077 >> 172.21.15.45:0/813603077 conn(0x5607248a8ec0 msgr2=0x5607248a7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:35.447 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.445+0000 7fb8cd368740 1 -- 172.21.15.45:0/813603077 shutdown_connections 2024-04-25T18:59:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.446+0000 7fb8cd368740 1 -- 172.21.15.45:0/813603077 wait complete. 2024-04-25T18:59:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.446+0000 7fb8cd368740 20 client.25184 trim_cache size 0 max 16384 2024-04-25T18:59:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T18:59:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T18:59:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T18:59:35.448 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T18:59:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.446+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2875553770 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb8a80a3ea0 msgr2=0x7fb8a80a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.446+0000 7fb8c8e00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb8a80a3ea0 0x7fb8a80a6360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x5607248aa2e0 tx=0x7fb8ac0097a0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.447+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2875553770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c410e470 msgr2=0x7fb8c419e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.447+0000 7fb8c8e00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c410e470 0x7fb8c419e8c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc009b30 tx=0x7fb8bc003ba0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.447+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2875553770 shutdown_connections 2024-04-25T18:59:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.447+0000 7fb8c8e00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb8a80a3ea0 0x7fb8a80a6360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.447+0000 7fb8c8e00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb8c410e470 0x7fb8c419e8c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.447+0000 7fb8c8e00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8c4108d50 0x7fb8c41a4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.447+0000 7fb8c8e00640 1 --2- 172.21.15.45:0/2875553770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8c41083b0 0x7fb8c41a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:35.449 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.447+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2875553770 >> 172.21.15.45:0/2875553770 conn(0x7fb8c4077b00 msgr2=0x7fb8c4079360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.447+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2875553770 shutdown_connections 2024-04-25T18:59:35.450 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:35.447+0000 7fb8c8e00640 1 -- 172.21.15.45:0/2875553770 wait complete. 2024-04-25T18:59:35.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:35 smithi045 ceph-mon[27776]: from='client.25174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:35.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:35 smithi062 ceph-mon[32844]: from='client.25174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:35.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:35 smithi079 ceph-mon[33002]: from='client.25174 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T18:59:36.545 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:36 smithi045 ceph-mon[27776]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:59:36.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:36 smithi062 ceph-mon[32844]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:59:36.678 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:36 smithi079 ceph-mon[33002]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T18:59:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.080+0000 7f49ebe00700 1 -- 172.21.15.45:0/2318783474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49e4103450 msgr2=0x7f49e410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.080+0000 7f49ebe00700 1 --2- 172.21.15.45:0/2318783474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49e4103450 0x7f49e410f8b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00b0a0 tx=0x7f49dc00b3b0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.081+0000 7f49ebe00700 1 -- 172.21.15.45:0/2318783474 shutdown_connections 2024-04-25T18:59:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.081+0000 7f49ebe00700 1 --2- 172.21.15.45:0/2318783474 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49e4108a50 0x7f49e4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.081+0000 7f49ebe00700 1 --2- 172.21.15.45:0/2318783474 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49e4103450 0x7f49e410f8b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.081+0000 7f49ebe00700 1 --2- 172.21.15.45:0/2318783474 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e4102a90 0x7f49e4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.081+0000 7f49ebe00700 1 -- 172.21.15.45:0/2318783474 >> 172.21.15.45:0/2318783474 conn(0x7f49e4079ba0 msgr2=0x7f49e4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:37.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.081+0000 7f49ebe00700 1 -- 172.21.15.45:0/2318783474 shutdown_connections 2024-04-25T18:59:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.081+0000 7f49ebe00700 1 -- 172.21.15.45:0/2318783474 wait complete. 2024-04-25T18:59:37.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.082+0000 7f49ebe00700 1 Processor -- start 2024-04-25T18:59:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.082+0000 7f49ebe00700 1 -- start start 2024-04-25T18:59:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.082+0000 7f49ebe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49e4102a90 0x7f49e4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.082+0000 7f49ebe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e4103450 0x7f49e4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.082+0000 7f49ebe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49e4108a50 0x7f49e40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49ebe00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f49e4112c50 con 0x7f49e4103450 2024-04-25T18:59:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49ebe00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f49e4112ad0 con 0x7f49e4102a90 2024-04-25T18:59:37.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49ebe00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f49e4112dd0 con 0x7f49e4108a50 2024-04-25T18:59:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49e9e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49e4108a50 0x7f49e40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49e9e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49e4108a50 0x7f49e40757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43434/0 (socket says 172.21.15.45:43434) 2024-04-25T18:59:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49e9e00700 1 -- 172.21.15.45:0/893858352 learned_addr learned my addr 172.21.15.45:0/893858352 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49e9400700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49e4102a90 0x7f49e4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49e8a00700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e4103450 0x7f49e4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49e9e00700 1 -- 172.21.15.45:0/893858352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49e4102a90 msgr2=0x7f49e4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49e9e00700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49e4102a90 0x7f49e4074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49e9e00700 1 -- 172.21.15.45:0/893858352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e4103450 msgr2=0x7f49e4075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49e9e00700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e4103450 0x7f49e4075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.087 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.083+0000 7f49e9e00700 1 -- 172.21.15.45:0/893858352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49e41a7730 con 0x7f49e4108a50 2024-04-25T18:59:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.084+0000 7f49e9e00700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49e4108a50 0x7f49e40757c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00b070 tx=0x7f49dc009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.084+0000 7f49e2000700 1 -- 172.21.15.45:0/893858352 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc021070 con 0x7f49e4108a50 2024-04-25T18:59:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.084+0000 7f49e2000700 1 -- 172.21.15.45:0/893858352 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49dc007e30 con 0x7f49e4108a50 2024-04-25T18:59:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.084+0000 7f49ebe00700 1 -- 172.21.15.45:0/893858352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49e41a79c0 con 0x7f49e4108a50 2024-04-25T18:59:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.084+0000 7f49e2000700 1 -- 172.21.15.45:0/893858352 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0046a0 con 0x7f49e4108a50 2024-04-25T18:59:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.084+0000 7f49ebe00700 1 -- 172.21.15.45:0/893858352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49e41a7f00 con 0x7f49e4108a50 2024-04-25T18:59:37.088 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.085+0000 7f49ebe00700 1 -- 172.21.15.45:0/893858352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49e4069730 con 0x7f49e4108a50 2024-04-25T18:59:37.089 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.087+0000 7f49e2000700 1 -- 172.21.15.45:0/893858352 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f49dc007980 con 0x7f49e4108a50 2024-04-25T18:59:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.089+0000 7f49e2000700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49cc0a3f50 0x7f49cc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.089+0000 7f49e2000700 1 -- 172.21.15.45:0/893858352 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f49dc0d1920 con 0x7f49e4108a50 2024-04-25T18:59:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.089+0000 7f49e9400700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49cc0a3f50 0x7f49cc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:37.091 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.090+0000 7f49e9400700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49cc0a3f50 0x7f49cc0a6410 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009be0 tx=0x7f49d8009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:37.094 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.092+0000 7f49e2000700 1 -- 172.21.15.45:0/893858352 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f49dc09dda0 con 0x7f49e4108a50 2024-04-25T18:59:37.375 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.372+0000 7f49ebe00700 1 -- 172.21.15.45:0/893858352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49e402e140 con 0x7f49e4108a50 2024-04-25T18:59:37.376 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.374+0000 7f49e2000700 1 -- 172.21.15.45:0/893858352 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7f49dc09d6a0 con 0x7f49e4108a50 2024-04-25T18:59:37.376 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:59:37.376 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:59:18.321335+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24512,24505]},"id":1}]} 2024-04-25T18:59:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.376+0000 7f49ebe00700 1 -- 172.21.15.45:0/893858352 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49cc0a3f50 msgr2=0x7f49cc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.376+0000 7f49ebe00700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49cc0a3f50 0x7f49cc0a6410 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009be0 tx=0x7f49d8009260 comp rx=0 tx=0).stop 2024-04-25T18:59:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.376+0000 7f49ebe00700 1 -- 172.21.15.45:0/893858352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49e4108a50 msgr2=0x7f49e40757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.376+0000 7f49ebe00700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49e4108a50 0x7f49e40757c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49dc00b070 tx=0x7f49dc009c90 comp rx=0 tx=0).stop 2024-04-25T18:59:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.377+0000 7f49ebe00700 1 -- 172.21.15.45:0/893858352 shutdown_connections 2024-04-25T18:59:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.377+0000 7f49ebe00700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49cc0a3f50 0x7f49cc0a6410 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.377+0000 7f49ebe00700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f49e4102a90 0x7f49e4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.377+0000 7f49ebe00700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f49e4108a50 0x7f49e40757c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.377+0000 7f49ebe00700 1 --2- 172.21.15.45:0/893858352 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f49e4103450 0x7f49e4075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:37.379 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.377+0000 7f49ebe00700 1 -- 172.21.15.45:0/893858352 >> 172.21.15.45:0/893858352 conn(0x7f49e4079ba0 msgr2=0x7f49e4101890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:37.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.377+0000 7f49ebe00700 1 -- 172.21.15.45:0/893858352 shutdown_connections 2024-04-25T18:59:37.380 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:37.377+0000 7f49ebe00700 1 -- 172.21.15.45:0/893858352 wait complete. 2024-04-25T18:59:37.380 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-04-25T18:59:38.210 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:59:38.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:38 smithi045 ceph-mon[27776]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-25T18:59:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:38 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/893858352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:38.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:38 smithi062 ceph-mon[32844]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-25T18:59:38.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:38 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/893858352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:38 smithi079 ceph-mon[33002]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-04-25T18:59:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:38 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/893858352' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:38.690 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:40.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:40 smithi045 ceph-mon[27776]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:59:40.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:40 smithi062 ceph-mon[32844]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:59:40.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:40 smithi079 ceph-mon[33002]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T18:59:41.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.022+0000 7f7354c00700 1 -- 172.21.15.45:0/3051937802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7350077890 msgr2=0x7f73501133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:41.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.022+0000 7f7354c00700 1 --2- 172.21.15.45:0/3051937802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7350077890 0x7f73501133b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7338009990 tx=0x7f7338009ca0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.022+0000 7f7354c00700 1 -- 172.21.15.45:0/3051937802 shutdown_connections 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.022+0000 7f7354c00700 1 --2- 172.21.15.45:0/3051937802 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7350076f70 0x7f7350077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.022+0000 7f7354c00700 1 --2- 172.21.15.45:0/3051937802 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73501138f0 0x7f7350115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.022+0000 7f7354c00700 1 --2- 172.21.15.45:0/3051937802 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7350077890 0x7f73501133b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.022+0000 7f7354c00700 1 -- 172.21.15.45:0/3051937802 >> 172.21.15.45:0/3051937802 conn(0x7f7350100720 msgr2=0x7f7350102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.022+0000 7f7354c00700 1 -- 172.21.15.45:0/3051937802 shutdown_connections 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.022+0000 7f7354c00700 1 -- 172.21.15.45:0/3051937802 wait complete. 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.023+0000 7f7354c00700 1 Processor -- start 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f7354c00700 1 -- start start 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f7354c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7350076f70 0x7f73501a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f7354c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7350077890 0x7f73501a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f7354c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73501138f0 0x7f735019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f7354c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f73501192c0 con 0x7f7350077890 2024-04-25T18:59:41.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f7354c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7350119140 con 0x7f73501138f0 2024-04-25T18:59:41.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f7354c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7350119440 con 0x7f7350076f70 2024-04-25T18:59:41.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f734de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7350076f70 0x7f73501a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:41.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f734de00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7350076f70 0x7f73501a5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43458/0 (socket says 172.21.15.45:43458) 2024-04-25T18:59:41.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f734de00700 1 -- 172.21.15.45:0/3575277433 learned_addr learned my addr 172.21.15.45:0/3575277433 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:41.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f7347e00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7350077890 0x7f73501a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:41.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.024+0000 7f734e800700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73501138f0 0x7f735019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:41.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.025+0000 7f7347e00700 1 -- 172.21.15.45:0/3575277433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7350076f70 msgr2=0x7f73501a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:41.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.025+0000 7f7347e00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7350076f70 0x7f73501a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.025+0000 7f7347e00700 1 -- 172.21.15.45:0/3575277433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73501138f0 msgr2=0x7f735019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:41.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.025+0000 7f7347e00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73501138f0 0x7f735019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.025+0000 7f7347e00700 1 -- 172.21.15.45:0/3575277433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f735019fd80 con 0x7f7350077890 2024-04-25T18:59:41.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.025+0000 7f734de00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7350076f70 0x7f73501a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:41.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.025+0000 7f7347e00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7350077890 0x7f73501a5950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7338000c00 tx=0x7f73380047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:41.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.026+0000 7f7346a00700 1 -- 172.21.15.45:0/3575277433 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338026070 con 0x7f7350077890 2024-04-25T18:59:41.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.026+0000 7f7346a00700 1 -- 172.21.15.45:0/3575277433 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7338021410 con 0x7f7350077890 2024-04-25T18:59:41.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.026+0000 7f7354c00700 1 -- 172.21.15.45:0/3575277433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73501a0010 con 0x7f7350077890 2024-04-25T18:59:41.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.026+0000 7f7346a00700 1 -- 172.21.15.45:0/3575277433 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7338017650 con 0x7f7350077890 2024-04-25T18:59:41.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.026+0000 7f7354c00700 1 -- 172.21.15.45:0/3575277433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73501ac2e0 con 0x7f7350077890 2024-04-25T18:59:41.030 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.027+0000 7f7354c00700 1 -- 172.21.15.45:0/3575277433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7350069730 con 0x7f7350077890 2024-04-25T18:59:41.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.029+0000 7f7346a00700 1 -- 172.21.15.45:0/3575277433 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f7338029070 con 0x7f7350077890 2024-04-25T18:59:41.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.030+0000 7f7346a00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73340a3f50 0x7f73340a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:41.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.031+0000 7f7346a00700 1 -- 172.21.15.45:0/3575277433 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f73380d1670 con 0x7f7350077890 2024-04-25T18:59:41.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.031+0000 7f734de00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73340a3f50 0x7f73340a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:41.033 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.032+0000 7f734de00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73340a3f50 0x7f73340a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7340011630 tx=0x7f7340009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:41.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.036+0000 7f7346a00700 1 -- 172.21.15.45:0/3575277433 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f733805dff0 con 0x7f7350077890 2024-04-25T18:59:41.320 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.318+0000 7f7354c00700 1 -- 172.21.15.45:0/3575277433 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73501a0b00 con 0x7f7350077890 2024-04-25T18:59:41.321 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.319+0000 7f7346a00700 1 -- 172.21.15.45:0/3575277433 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7f7338013070 con 0x7f7350077890 2024-04-25T18:59:41.322 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:59:41.322 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:59:18.321335+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24512,24505]},"id":1}]} 2024-04-25T18:59:41.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.322+0000 7f7354c00700 1 -- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73340a3f50 msgr2=0x7f73340a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:41.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.322+0000 7f7354c00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73340a3f50 0x7f73340a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7340011630 tx=0x7f7340009260 comp rx=0 tx=0).stop 2024-04-25T18:59:41.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.322+0000 7f7354c00700 1 -- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7350077890 msgr2=0x7f73501a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:41.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.322+0000 7f7354c00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7350077890 0x7f73501a5950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7338000c00 tx=0x7f73380047b0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.323+0000 7f7354c00700 1 -- 172.21.15.45:0/3575277433 shutdown_connections 2024-04-25T18:59:41.325 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.323+0000 7f7354c00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73340a3f50 0x7f73340a6410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.323+0000 7f7354c00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f73501138f0 0x7f735019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.323+0000 7f7354c00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7350076f70 0x7f73501a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.323+0000 7f7354c00700 1 --2- 172.21.15.45:0/3575277433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7350077890 0x7f73501a5950 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:41.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.323+0000 7f7354c00700 1 -- 172.21.15.45:0/3575277433 >> 172.21.15.45:0/3575277433 conn(0x7f7350100720 msgr2=0x7f7350102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:41.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.323+0000 7f7354c00700 1 -- 172.21.15.45:0/3575277433 shutdown_connections 2024-04-25T18:59:41.326 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:41.323+0000 7f7354c00700 1 -- 172.21.15.45:0/3575277433 wait complete. 2024-04-25T18:59:41.327 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-04-25T18:59:42.126 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T18:59:42.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:42 smithi045 ceph-mon[27776]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:59:42.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:42 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3575277433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:42.613 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:42.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:42 smithi062 ceph-mon[32844]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:59:42.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:42 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3575277433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:42.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:42 smithi079 ceph-mon[33002]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:59:42.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:42 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3575277433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:44.405 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:44 smithi045 ceph-mon[27776]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:59:44.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:44 smithi062 ceph-mon[32844]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:59:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:44 smithi079 ceph-mon[33002]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:59:44.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.967+0000 7fc9a1200700 1 -- 172.21.15.45:0/95760709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc99c10f340 msgr2=0x7fc99c111730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:44.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.967+0000 7fc9a1200700 1 --2- 172.21.15.45:0/95760709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc99c10f340 0x7fc99c111730 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc99400b0a0 tx=0x7fc99400b3b0 comp rx=0 tx=0).stop 2024-04-25T18:59:44.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.967+0000 7fc9a1200700 1 -- 172.21.15.45:0/95760709 shutdown_connections 2024-04-25T18:59:44.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.967+0000 7fc9a1200700 1 --2- 172.21.15.45:0/95760709 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc99c101790 0x7fc99c101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:44.970 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.967+0000 7fc9a1200700 1 --2- 172.21.15.45:0/95760709 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc99c10f340 0x7fc99c111730 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.967+0000 7fc9a1200700 1 --2- 172.21.15.45:0/95760709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc99c102140 0x7fc99c10ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.967+0000 7fc9a1200700 1 -- 172.21.15.45:0/95760709 >> 172.21.15.45:0/95760709 conn(0x7fc99c0fd3d0 msgr2=0x7fc99c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.967+0000 7fc9a1200700 1 -- 172.21.15.45:0/95760709 shutdown_connections 2024-04-25T18:59:44.971 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.968+0000 7fc9a1200700 1 -- 172.21.15.45:0/95760709 wait complete. 2024-04-25T18:59:44.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.969+0000 7fc9a1200700 1 Processor -- start 2024-04-25T18:59:44.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.969+0000 7fc9a1200700 1 -- start start 2024-04-25T18:59:44.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.969+0000 7fc9a1200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc99c101790 0x7fc99c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:44.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.969+0000 7fc9a1200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc99c102140 0x7fc99c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:44.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.969+0000 7fc9a1200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc99c10f340 0x7fc99c19fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:44.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.969+0000 7fc9a1200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c114de0 con 0x7fc99c102140 2024-04-25T18:59:44.972 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.969+0000 7fc9a1200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c114c60 con 0x7fc99c101790 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.969+0000 7fc9a1200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c114f60 con 0x7fc99c10f340 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc99c102140 0x7fc99c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc99c101790 0x7fc99c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc99c10f340 0x7fc99c19fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc99c102140 0x7fc99c19b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46166/0 (socket says 172.21.15.45:46166) 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99aa00700 1 -- 172.21.15.45:0/3386843536 learned_addr learned my addr 172.21.15.45:0/3386843536 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99b400700 1 -- 172.21.15.45:0/3386843536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc99c10f340 msgr2=0x7fc99c19fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99b400700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc99c10f340 0x7fc99c19fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99b400700 1 -- 172.21.15.45:0/3386843536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc99c102140 msgr2=0x7fc99c19b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99b400700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc99c102140 0x7fc99c19b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99b400700 1 -- 172.21.15.45:0/3386843536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c1a0330 con 0x7fc99c101790 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99aa00700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc99c102140 0x7fc99c19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:44.973 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.970+0000 7fc99be00700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc99c10f340 0x7fc99c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:44.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.971+0000 7fc99b400700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc99c101790 0x7fc99c19b370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00e9f0 tx=0x7fc98c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:44.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.971+0000 7fc993e00700 1 -- 172.21.15.45:0/3386843536 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00c9c0 con 0x7fc99c101790 2024-04-25T18:59:44.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.971+0000 7fc993e00700 1 -- 172.21.15.45:0/3386843536 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc98c00cb60 con 0x7fc99c101790 2024-04-25T18:59:44.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.971+0000 7fc9a1200700 1 -- 172.21.15.45:0/3386843536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c1a0620 con 0x7fc99c101790 2024-04-25T18:59:44.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.971+0000 7fc993e00700 1 -- 172.21.15.45:0/3386843536 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c018680 con 0x7fc99c101790 2024-04-25T18:59:44.974 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.971+0000 7fc9a1200700 1 -- 172.21.15.45:0/3386843536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c1a7ea0 con 0x7fc99c101790 2024-04-25T18:59:44.975 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.972+0000 7fc9a1200700 1 -- 172.21.15.45:0/3386843536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99c069730 con 0x7fc99c101790 2024-04-25T18:59:44.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.974+0000 7fc993e00700 1 -- 172.21.15.45:0/3386843536 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fc98c010430 con 0x7fc99c101790 2024-04-25T18:59:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.976+0000 7fc993e00700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fc9800a3ed0 0x7fc9800a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.976+0000 7fc993e00700 1 -- 172.21.15.45:0/3386843536 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fc98c0d0fc0 con 0x7fc99c101790 2024-04-25T18:59:44.978 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.976+0000 7fc99aa00700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fc9800a3ed0 0x7fc9800a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:44.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.977+0000 7fc99aa00700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fc9800a3ed0 0x7fc9800a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc99c103190 tx=0x7fc988009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:44.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:44.978+0000 7fc993e00700 1 -- 172.21.15.45:0/3386843536 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc98c09d3b0 con 0x7fc99c101790 2024-04-25T18:59:45.213 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.211+0000 7fc9a1200700 1 -- 172.21.15.45:0/3386843536 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc99c02df80 con 0x7fc99c101790 2024-04-25T18:59:45.215 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.213+0000 7fc993e00700 1 -- 172.21.15.45:0/3386843536 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23030 (secure 0 0 0) 0x7fc98c09ccb0 con 0x7fc99c101790 2024-04-25T18:59:45.216 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:59:45.216 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":95,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T18:59:18.747004+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:55:17.591864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:55:22.070218+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/943541413":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/2599503580":"2024-04-26T18:59:18.746957+0000","172.21.15.45:6800/943541413":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/3590060181":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/4010951391":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3367087840":"2024-04-26T18:59:18.746957+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T18:59:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.216+0000 7fc9a1200700 1 -- 172.21.15.45:0/3386843536 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fc9800a3ed0 msgr2=0x7fc9800a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.216+0000 7fc9a1200700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fc9800a3ed0 0x7fc9800a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc99c103190 tx=0x7fc988009260 comp rx=0 tx=0).stop 2024-04-25T18:59:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.216+0000 7fc9a1200700 1 -- 172.21.15.45:0/3386843536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc99c101790 msgr2=0x7fc99c19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.216+0000 7fc9a1200700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc99c101790 0x7fc99c19b370 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00e9f0 tx=0x7fc98c00ed00 comp rx=0 tx=0).stop 2024-04-25T18:59:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.217+0000 7fc9a1200700 1 -- 172.21.15.45:0/3386843536 shutdown_connections 2024-04-25T18:59:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.217+0000 7fc9a1200700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fc9800a3ed0 0x7fc9800a6390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.217+0000 7fc9a1200700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fc99c101790 0x7fc99c19b370 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.217+0000 7fc9a1200700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc99c10f340 0x7fc99c19fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.217+0000 7fc9a1200700 1 --2- 172.21.15.45:0/3386843536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc99c102140 0x7fc99c19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:45.219 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.217+0000 7fc9a1200700 1 -- 172.21.15.45:0/3386843536 >> 172.21.15.45:0/3386843536 conn(0x7fc99c0fd3d0 msgr2=0x7fc99c0ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.217+0000 7fc9a1200700 1 -- 172.21.15.45:0/3386843536 shutdown_connections 2024-04-25T18:59:45.220 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:45.217+0000 7fc9a1200700 1 -- 172.21.15.45:0/3386843536 wait complete. 2024-04-25T18:59:45.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:45 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3386843536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:59:45.615 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:45 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3386843536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:59:45.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:45 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3386843536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T18:59:46.005 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-25T18:59:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:46 smithi079 ceph-mon[33002]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:59:46.481 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:46.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:46 smithi062 ceph-mon[32844]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:59:46.638 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:46 smithi045 ceph-mon[27776]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:59:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.593+0000 7f56a3200640 1 -- 172.21.15.45:0/3013741287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a4104bb0 msgr2=0x7f56a4108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.593+0000 7f56a3200640 1 --2- 172.21.15.45:0/3013741287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a4104bb0 0x7f56a4108bf0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f568c009900 tx=0x7f568c02a410 comp rx=0 tx=0).stop 2024-04-25T18:59:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.593+0000 7f56a3200640 1 -- 172.21.15.45:0/3013741287 shutdown_connections 2024-04-25T18:59:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.593+0000 7f56a3200640 1 --2- 172.21.15.45:0/3013741287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a41092d0 0x7f56a4110ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.593+0000 7f56a3200640 1 --2- 172.21.15.45:0/3013741287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a4104bb0 0x7f56a4108bf0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.593+0000 7f56a3200640 1 --2- 172.21.15.45:0/3013741287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a4104200 0x7f56a41045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.593+0000 7f56a3200640 1 -- 172.21.15.45:0/3013741287 >> 172.21.15.45:0/3013741287 conn(0x7f56a40ffa60 msgr2=0x7f56a4101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.593+0000 7f56a3200640 1 -- 172.21.15.45:0/3013741287 shutdown_connections 2024-04-25T18:59:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.593+0000 7f56a3200640 1 -- 172.21.15.45:0/3013741287 wait complete. 2024-04-25T18:59:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.594+0000 7f56a3200640 1 Processor -- start 2024-04-25T18:59:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.594+0000 7f56a3200640 1 -- start start 2024-04-25T18:59:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a3200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a4104200 0x7f56a41a0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a41092d0 0x7f56a41a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a3200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a419a920 0x7f56a419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a3200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56a4114240 con 0x7f56a41092d0 2024-04-25T18:59:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a3200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41140c0 con 0x7f56a419a920 2024-04-25T18:59:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a3200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41143c0 con 0x7f56a4104200 2024-04-25T18:59:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a41092d0 0x7f56a41a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a41092d0 0x7f56a41a0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46192/0 (socket says 172.21.15.45:46192) 2024-04-25T18:59:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a1400640 1 -- 172.21.15.45:0/1808244102 learned_addr learned my addr 172.21.15.45:0/1808244102 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a1e00640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a4104200 0x7f56a41a0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.595+0000 7f56a2800640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a419a920 0x7f56a419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f56a1400640 1 -- 172.21.15.45:0/1808244102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a4104200 msgr2=0x7f56a41a0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f56a1400640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a4104200 0x7f56a41a0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f56a1400640 1 -- 172.21.15.45:0/1808244102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a419a920 msgr2=0x7f56a419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f56a1400640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a419a920 0x7f56a419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f56a1400640 1 -- 172.21.15.45:0/1808244102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a419b440 con 0x7f56a41092d0 2024-04-25T18:59:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f56a1e00640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a4104200 0x7f56a41a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f56a1400640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a41092d0 0x7f56a41a0cb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f568c005ab0 tx=0x7f568c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f569aa00640 1 -- 172.21.15.45:0/1808244102 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c03f070 con 0x7f56a41092d0 2024-04-25T18:59:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f569aa00640 1 -- 172.21.15.45:0/1808244102 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f568c031b50 con 0x7f56a41092d0 2024-04-25T18:59:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f56a3200640 1 -- 172.21.15.45:0/1808244102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a419b6d0 con 0x7f56a41092d0 2024-04-25T18:59:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f569aa00640 1 -- 172.21.15.45:0/1808244102 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c035030 con 0x7f56a41092d0 2024-04-25T18:59:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.596+0000 7f56a3200640 1 -- 172.21.15.45:0/1808244102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a41a7920 con 0x7f56a41092d0 2024-04-25T18:59:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.600+0000 7f56abbe7740 1 Processor -- start 2024-04-25T18:59:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.600+0000 7f569aa00640 1 -- 172.21.15.45:0/1808244102 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f568c042070 con 0x7f56a41092d0 2024-04-25T18:59:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.600+0000 7f56abbe7740 1 -- start start 2024-04-25T18:59:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.600+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b84d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.600+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.600+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b8fb90 0x555600b91f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.600+0000 7f56abbe7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555600aea900 con 0x555600b8fb90 2024-04-25T18:59:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.600+0000 7f56abbe7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555600a90ef0 con 0x555600b849b0 2024-04-25T18:59:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.600+0000 7f56abbe7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555600ae68c0 con 0x555600b852d0 2024-04-25T18:59:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.600+0000 7f56a1e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b84d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b8fb90 0x555600b91f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a1e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48870/0 (socket says 172.21.15.45:48870) 2024-04-25T18:59:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a1e00640 1 -- 172.21.15.45:0/1881373407 learned_addr learned my addr 172.21.15.45:0/1881373407 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a2800640 1 -- 172.21.15.45:0/1881373407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 msgr2=0x555600b8f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a2800640 1 --2- 172.21.15.45:0/1881373407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a2800640 1 -- 172.21.15.45:0/1881373407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b8fb90 msgr2=0x555600b91f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a2800640 1 --2- 172.21.15.45:0/1881373407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b8fb90 0x555600b91f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a2800640 1 -- 172.21.15.45:0/1881373407 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556007fd210 con 0x555600b849b0 2024-04-25T18:59:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a1e00640 1 --2- 172.21.15.45:0/1881373407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f56a2800640 1 --2- 172.21.15.45:0/1881373407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b84d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5694009b80 tx=0x7f5694011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6023cfa59d9d6133 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.601+0000 7f569aa00640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56880a3ea0 0x7f56880a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.602+0000 7f5693400640 1 -- 172.21.15.45:0/1881373407 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694004050 con 0x555600b849b0 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.602+0000 7f5693400640 1 -- 172.21.15.45:0/1881373407 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56940054d0 con 0x555600b849b0 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.602+0000 7f5693400640 1 -- 172.21.15.45:0/1881373407 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5694004c40 con 0x555600b849b0 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.602+0000 7f569aa00640 1 -- 172.21.15.45:0/1808244102 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f568c0f0f20 con 0x7f56a41092d0 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.602+0000 7f56a1e00640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56880a3ea0 0x7f56880a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56abbe7740 1 -- 172.21.15.45:0/1881373407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 msgr2=0x555600b84d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56abbe7740 1 --2- 172.21.15.45:0/1881373407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b84d90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5694009b80 tx=0x7f5694011810 comp rx=0 tx=0).stop 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56a1e00640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56880a3ea0 0x7f56880a6360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x555600b85f50 tx=0x7f569c009500 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56abbe7740 1 -- 172.21.15.45:0/1881373407 shutdown_connections 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56abbe7740 1 --2- 172.21.15.45:0/1881373407 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b8fb90 0x555600b91f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56abbe7740 1 --2- 172.21.15.45:0/1881373407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56abbe7740 1 --2- 172.21.15.45:0/1881373407 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b84d90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56abbe7740 1 -- 172.21.15.45:0/1881373407 >> 172.21.15.45:0/1881373407 conn(0x555600b80ec0 msgr2=0x555600b812a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56abbe7740 1 -- 172.21.15.45:0/1881373407 shutdown_connections 2024-04-25T18:59:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56abbe7740 1 -- 172.21.15.45:0/1881373407 wait complete. 2024-04-25T18:59:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.603+0000 7f56abbe7740 1 -- start start 2024-04-25T18:59:48.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56abbe7740 1 Processor -- start 2024-04-25T18:59:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b849b0 0x555600b8cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b8fb90 0x555600b8d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56abbe7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555600b927c0 con 0x555600b849b0 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56abbe7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x555600aa4310 con 0x555600b8fb90 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56abbe7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x555600b92940 con 0x555600b852d0 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56abbe7740 10 client.0 _ll_register_callbacks cb 0x555600a72cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56a1400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56a1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b849b0 0x555600b8cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56a1400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48878/0 (socket says 172.21.15.45:48878) 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56a1400640 1 -- 172.21.15.45:0/1834362789 learned_addr learned my addr 172.21.15.45:0/1834362789 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.604+0000 7f56a2800640 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b8fb90 0x555600b8d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.605+0000 7f56a1400640 1 -- 172.21.15.45:0/1834362789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b8fb90 msgr2=0x555600b8d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.605+0000 7f56a1400640 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b8fb90 0x555600b8d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.605+0000 7f56a1400640 1 -- 172.21.15.45:0/1834362789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b849b0 msgr2=0x555600b8cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:48.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.605+0000 7f56a1400640 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b849b0 0x555600b8cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.605+0000 7f56a1400640 1 -- 172.21.15.45:0/1834362789 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556008c9620 con 0x555600b852d0 2024-04-25T18:59:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.605+0000 7f56a1e00640 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b849b0 0x555600b8cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.605+0000 7f56a1400640 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8d080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f568c031440 tx=0x7f568c031f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56aa503f344d105a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.605+0000 7f5687e00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T18:59:48.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.606+0000 7f5687e00640 1 -- 172.21.15.45:0/1834362789 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c049030 con 0x555600b852d0 2024-04-25T18:59:48.610 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:48 smithi045 ceph-mon[27776]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:59:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.606+0000 7f5687e00640 1 -- 172.21.15.45:0/1834362789 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f568c0b9400 con 0x555600b852d0 2024-04-25T18:59:48.612 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.606+0000 7f56abbe7740 1 -- 172.21.15.45:0/1834362789 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56940054d0 con 0x555600b852d0 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.606+0000 7f5687e00640 1 -- 172.21.15.45:0/1834362789 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f568c0bc8d0 con 0x555600b852d0 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.606+0000 7f56abbe7740 10 client.25222 fetch_fsmap learned FSMap version 26 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.606+0000 7f56abbe7740 1 -- 172.21.15.45:0/1834362789 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555600b927c0 con 0x555600b852d0 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.606+0000 7f5687e00640 1 -- 172.21.15.45:0/1834362789 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56940054d0 con 0x555600b852d0 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.607+0000 7f5687e00640 1 -- 172.21.15.45:0/1834362789 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f568c0f4c20 con 0x555600b852d0 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.607+0000 7f56abbe7740 10 client.25222 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.607+0000 7f56abbe7740 10 client.25222 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.607+0000 7f56abbe7740 20 client.25222 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.607+0000 7f56abbe7740 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x555600b9fdc0 0x555600ba2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.607+0000 7f56abbe7740 4 client.25222 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.607+0000 7f56abbe7740 1 -- 172.21.15.45:0/1834362789 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555600afe8b0 con 0x555600b9fdc0 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.607+0000 7f56a1e00640 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x555600b9fdc0 0x555600ba2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.608+0000 7f56a1e00640 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x555600b9fdc0 0x555600ba2240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5144e51fbf604c2 server_cookie=7907ed928ade683d in_seq=0 out_seq=0 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.608+0000 7f5687e00640 10 client.25222 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:48.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f5687e00640 1 -- 172.21.15.45:0/1834362789 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555600afe8b0 con 0x555600b9fdc0 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f5687e00640 10 client.25222 handle_command_reply: tid=0 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 client.25222 shutdown 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 -- 172.21.15.45:0/1834362789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 msgr2=0x555600b8d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8d080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f568c031440 tx=0x7f568c031f50 comp rx=0 tx=0).stop 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 -- 172.21.15.45:0/1834362789 shutdown_connections 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x555600b9fdc0 0x555600ba2240 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b8fb90 0x555600b8d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b852d0 0x555600b8d080 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 --2- 172.21.15.45:0/1834362789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b849b0 0x555600b8cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f5687e00640 0 client.25222 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 -- 172.21.15.45:0/1834362789 >> 172.21.15.45:0/1834362789 conn(0x555600b80ec0 msgr2=0x555600b86bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 -- 172.21.15.45:0/1834362789 shutdown_connections 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 1 -- 172.21.15.45:0/1834362789 wait complete. 2024-04-25T18:59:48.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.609+0000 7f56abbe7740 20 client.25222 trim_cache size 0 max 16384 2024-04-25T18:59:48.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:48 smithi062 ceph-mon[32844]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.630+0000 7f56abbe7740 1 Processor -- start 2024-04-25T18:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.630+0000 7f56abbe7740 1 -- start start 2024-04-25T18:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b9b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b87370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 0x555600ba7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.633 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56abbe7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41b2490 con 0x555600b852d0 2024-04-25T18:59:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56abbe7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41afd60 con 0x555600b849b0 2024-04-25T18:59:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56abbe7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41adbb0 con 0x555600b8fb90 2024-04-25T18:59:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56a1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b9b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56a1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b9b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:40522/0 (socket says 172.21.15.45:40522) 2024-04-25T18:59:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56a1400640 1 --2- 172.21.15.45:0/4099353371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b87370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56a1e00640 1 -- 172.21.15.45:0/4099353371 learned_addr learned my addr 172.21.15.45:0/4099353371 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:48.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.631+0000 7f56a2800640 1 --2- 172.21.15.45:0/4099353371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 0x555600ba7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.632+0000 7f56a1400640 1 -- 172.21.15.45:0/4099353371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 msgr2=0x555600ba7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:48.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.632+0000 7f56a1400640 1 --2- 172.21.15.45:0/4099353371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 0x555600ba7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.632+0000 7f56a1400640 1 -- 172.21.15.45:0/4099353371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 msgr2=0x555600b9b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.632+0000 7f56a1400640 1 --2- 172.21.15.45:0/4099353371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b9b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.632+0000 7f56a1400640 1 -- 172.21.15.45:0/4099353371 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555600ba28c0 con 0x555600b852d0 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.632+0000 7f56a2800640 1 --2- 172.21.15.45:0/4099353371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 0x555600ba7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.632+0000 7f56a1400640 1 --2- 172.21.15.45:0/4099353371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b87370 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f568c0bca90 tx=0x7f568c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe95fd0fd81177a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.632+0000 7f5692a00640 1 -- 172.21.15.45:0/4099353371 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c049070 con 0x555600b852d0 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.632+0000 7f5692a00640 1 -- 172.21.15.45:0/4099353371 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f568c0bcd90 con 0x555600b852d0 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.632+0000 7f5692a00640 1 -- 172.21.15.45:0/4099353371 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c0bc6f0 con 0x555600b852d0 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.633+0000 7f56abbe7740 1 -- 172.21.15.45:0/4099353371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 msgr2=0x555600b87370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.633+0000 7f56abbe7740 1 --2- 172.21.15.45:0/4099353371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b87370 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f568c0bca90 tx=0x7f568c02a410 comp rx=0 tx=0).stop 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.633+0000 7f56abbe7740 1 -- 172.21.15.45:0/4099353371 shutdown_connections 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.633+0000 7f56abbe7740 1 --2- 172.21.15.45:0/4099353371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 0x555600ba7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.633+0000 7f56abbe7740 1 --2- 172.21.15.45:0/4099353371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b87370 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.633+0000 7f56abbe7740 1 --2- 172.21.15.45:0/4099353371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b9b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.633+0000 7f56abbe7740 1 -- 172.21.15.45:0/4099353371 >> 172.21.15.45:0/4099353371 conn(0x555600b80ec0 msgr2=0x555600b8ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.633+0000 7f56abbe7740 1 -- 172.21.15.45:0/4099353371 shutdown_connections 2024-04-25T18:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.633+0000 7f56abbe7740 1 -- 172.21.15.45:0/4099353371 wait complete. 2024-04-25T18:59:48.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.634+0000 7f56abbe7740 1 -- start start 2024-04-25T18:59:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.634+0000 7f56abbe7740 1 Processor -- start 2024-04-25T18:59:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.634+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b95620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b95b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56abbe7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 0x555600b960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.639 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56abbe7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41ae220 con 0x555600b852d0 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56abbe7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41af690 con 0x555600b849b0 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56abbe7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56a41ae8f0 con 0x555600b8fb90 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56abbe7740 10 client.0 _ll_register_callbacks cb 0x7f568c0f46a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b95b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b95620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b95b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46220/0 (socket says 172.21.15.45:46220) 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a1400640 1 -- 172.21.15.45:0/755207184 learned_addr learned my addr 172.21.15.45:0/755207184 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b95620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:40534/0 (socket says 172.21.15.45:40534) 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a2800640 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 0x555600b960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a1400640 1 -- 172.21.15.45:0/755207184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 msgr2=0x555600b960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a1400640 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 0x555600b960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a1400640 1 -- 172.21.15.45:0/755207184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 msgr2=0x555600b95620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a1400640 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b95620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.635+0000 7f56a1400640 1 -- 172.21.15.45:0/755207184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555600c053d0 con 0x555600b852d0 2024-04-25T18:59:48.640 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.636+0000 7f56a1400640 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b95b60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f568c0bcd20 tx=0x7f568c0bb2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a119eef3d6c85f53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.636+0000 7f5687e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.636+0000 7f5687e00640 1 -- 172.21.15.45:0/755207184 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f568c049070 con 0x555600b852d0 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.636+0000 7f56abbe7740 1 -- 172.21.15.45:0/755207184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f568c0bcd90 con 0x555600b852d0 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.636+0000 7f5687e00640 1 -- 172.21.15.45:0/755207184 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f568c0dd2d0 con 0x555600b852d0 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.636+0000 7f5687e00640 1 -- 172.21.15.45:0/755207184 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f568c0db050 con 0x555600b852d0 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.636+0000 7f56abbe7740 10 client.15099 fetch_fsmap learned FSMap version 26 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.636+0000 7f56abbe7740 1 -- 172.21.15.45:0/755207184 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555600c053d0 con 0x555600b852d0 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.637+0000 7f5687e00640 1 -- 172.21.15.45:0/755207184 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f568c0bcd90 con 0x555600b852d0 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.637+0000 7f5687e00640 1 -- 172.21.15.45:0/755207184 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f568c0c54c0 con 0x555600b852d0 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.637+0000 7f56abbe7740 10 client.15099 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.637+0000 7f56abbe7740 10 client.15099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.637+0000 7f56abbe7740 20 client.15099 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.637+0000 7f56abbe7740 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x555600c06910 0x555600c08d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.637+0000 7f56abbe7740 4 client.15099 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T18:59:48.641 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.637+0000 7f56abbe7740 1 -- 172.21.15.45:0/755207184 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555600af8320 con 0x555600c06910 2024-04-25T18:59:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.637+0000 7f56a1e00640 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x555600c06910 0x555600c08d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.638+0000 7f56a1e00640 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x555600c06910 0x555600c08d00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46c38f2606c9222b server_cookie=eedc6167518fe591 in_seq=0 out_seq=0 2024-04-25T18:59:48.642 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.638+0000 7f5687e00640 10 client.15099 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f5687e00640 1 -- 172.21.15.45:0/755207184 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555600af8320 con 0x555600c06910 2024-04-25T18:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f5687e00640 10 client.15099 handle_command_reply: tid=0 2024-04-25T18:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f56abbe7740 1 client.15099 shutdown 2024-04-25T18:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f56abbe7740 1 -- 172.21.15.45:0/755207184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 msgr2=0x555600b95b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f56abbe7740 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b95b60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f568c0bcd20 tx=0x7f568c0bb2a0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f56abbe7740 1 -- 172.21.15.45:0/755207184 shutdown_connections 2024-04-25T18:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f56abbe7740 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x555600c06910 0x555600c08d00 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f56abbe7740 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x555600b8fb90 0x555600b960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f56abbe7740 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555600b852d0 0x555600b95b60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.647 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f56abbe7740 1 --2- 172.21.15.45:0/755207184 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x555600b849b0 0x555600b95620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f5687e00640 0 client.15099 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T18:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f56abbe7740 1 -- 172.21.15.45:0/755207184 >> 172.21.15.45:0/755207184 conn(0x555600b80ec0 msgr2=0x555600b8e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.645+0000 7f56abbe7740 1 -- 172.21.15.45:0/755207184 shutdown_connections 2024-04-25T18:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.646+0000 7f56abbe7740 1 -- 172.21.15.45:0/755207184 wait complete. 2024-04-25T18:59:48.648 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.646+0000 7f56abbe7740 20 client.15099 trim_cache size 0 max 16384 2024-04-25T18:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T18:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T18:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T18:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T18:59:48.649 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.646+0000 7f56a3200640 1 -- 172.21.15.45:0/1808244102 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56880a3ea0 msgr2=0x7f56880a6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.646+0000 7f56a3200640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56880a3ea0 0x7f56880a6360 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x555600b85f50 tx=0x7f569c009500 comp rx=0 tx=0).stop 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.646+0000 7f56a3200640 1 -- 172.21.15.45:0/1808244102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a41092d0 msgr2=0x7f56a41a0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.646+0000 7f56a3200640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a41092d0 0x7f56a41a0cb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f568c005ab0 tx=0x7f568c003b90 comp rx=0 tx=0).stop 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.647+0000 7f56a3200640 1 -- 172.21.15.45:0/1808244102 shutdown_connections 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.647+0000 7f56a3200640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56880a3ea0 0x7f56880a6360 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.647+0000 7f56a3200640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56a419a920 0x7f56a419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.647+0000 7f56a3200640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56a41092d0 0x7f56a41a0cb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.647+0000 7f56a3200640 1 --2- 172.21.15.45:0/1808244102 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56a4104200 0x7f56a41a0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.647+0000 7f56a3200640 1 -- 172.21.15.45:0/1808244102 >> 172.21.15.45:0/1808244102 conn(0x7f56a40ffa60 msgr2=0x7f56a4101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.647+0000 7f56a3200640 1 -- 172.21.15.45:0/1808244102 shutdown_connections 2024-04-25T18:59:48.650 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T18:59:48.647+0000 7f56a3200640 1 -- 172.21.15.45:0/1808244102 wait complete. 2024-04-25T18:59:48.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:48 smithi079 ceph-mon[33002]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-25T18:59:49.743 INFO:teuthology.orchestra.run.smithi045.stdout:{"run_dir":"/var/run/ceph"} 2024-04-25T18:59:49.743 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-25T18:59:49.744 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 50566 2024-04-25T18:59:49.744 DEBUG:tasks.quiescer:Yielding 2024-04-25T18:59:49.744 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T18:59:49.758 INFO:tasks.workunit:Pulling workunits from ref 7797745e6d8b3ac036504dc61491d368615024da 2024-04-25T18:59:49.758 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T18:59:49.758 INFO:tasks.workunit:timeout=3h 2024-04-25T18:59:49.758 INFO:tasks.workunit:cleanup=True 2024-04-25T18:59:49.759 DEBUG:teuthology.orchestra.run.smithi045:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T18:59:49.759 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T18:59:49.815 INFO:teuthology.orchestra.run.smithi045.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T18:59:49.815 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T18:59:49.815 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-25T18:59:49.815 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T18:59:49.815 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T18:59:49.815 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-25 18:56:05.451661863 +0000 2024-04-25T18:59:49.815 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-25 18:56:05.451661863 +0000 2024-04-25T18:59:49.815 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-25 18:59:17.232298697 +0000 2024-04-25T18:59:49.815 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-25 18:56:05.451661863 +0000 2024-04-25T18:59:49.815 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T18:59:49.816 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T18:59:49.948 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7797745e6d8b3ac036504dc61491d368615024da 2024-04-25T18:59:49.992 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T18:59:50.215 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:50.566 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:50 smithi045 ceph-mon[27776]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:59:50.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:50 smithi062 ceph-mon[32844]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:59:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:50 smithi079 ceph-mon[33002]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2024-04-25T18:59:51.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:51 smithi062 ceph-mon[32844]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T18:59:51.631 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:51 smithi045 ceph-mon[27776]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T18:59:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:51 smithi079 ceph-mon[33002]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-25T18:59:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.487+0000 7f6fe2200700 1 -- 172.21.15.45:0/3574409773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fdc103bd0 msgr2=0x7f6fdc103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.487+0000 7f6fe2200700 1 --2- 172.21.15.45:0/3574409773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fdc103bd0 0x7f6fdc103fb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009990 tx=0x7f6fd0009ca0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.487+0000 7f6fe2200700 1 -- 172.21.15.45:0/3574409773 shutdown_connections 2024-04-25T18:59:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.487+0000 7f6fe2200700 1 --2- 172.21.15.45:0/3574409773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fdc10cfb0 0x7f6fdc10f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.487+0000 7f6fe2200700 1 --2- 172.21.15.45:0/3574409773 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fdc103bd0 0x7f6fdc103fb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.487+0000 7f6fe2200700 1 --2- 172.21.15.45:0/3574409773 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc1044f0 0x7f6fdc10c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.487+0000 7f6fe2200700 1 -- 172.21.15.45:0/3574409773 >> 172.21.15.45:0/3574409773 conn(0x7f6fdc0fd360 msgr2=0x7f6fdc0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.487+0000 7f6fe2200700 1 -- 172.21.15.45:0/3574409773 shutdown_connections 2024-04-25T18:59:52.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.487+0000 7f6fe2200700 1 -- 172.21.15.45:0/3574409773 wait complete. 2024-04-25T18:59:52.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.489+0000 7f6fe2200700 1 Processor -- start 2024-04-25T18:59:52.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.489+0000 7f6fe2200700 1 -- start start 2024-04-25T18:59:52.491 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.489+0000 7f6fe2200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fdc103bd0 0x7f6fdc19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:52.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.489+0000 7f6fe2200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc1044f0 0x7f6fdc19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:52.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe2200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fdc10cfb0 0x7f6fdc19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:52.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe2200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc112bc0 con 0x7f6fdc103bd0 2024-04-25T18:59:52.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe2200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc112a40 con 0x7f6fdc1044f0 2024-04-25T18:59:52.492 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe2200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc112d40 con 0x7f6fdc10cfb0 2024-04-25T18:59:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe0e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fdc103bd0 0x7f6fdc19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe0e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fdc103bd0 0x7f6fdc19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46230/0 (socket says 172.21.15.45:46230) 2024-04-25T18:59:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe0e00700 1 -- 172.21.15.45:0/2399762981 learned_addr learned my addr 172.21.15.45:0/2399762981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fdbe00700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc1044f0 0x7f6fdc19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe1800700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fdc10cfb0 0x7f6fdc19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe0e00700 1 -- 172.21.15.45:0/2399762981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fdc10cfb0 msgr2=0x7f6fdc19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe0e00700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fdc10cfb0 0x7f6fdc19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe0e00700 1 -- 172.21.15.45:0/2399762981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc1044f0 msgr2=0x7f6fdc19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe0e00700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc1044f0 0x7f6fdc19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.490+0000 7f6fe0e00700 1 -- 172.21.15.45:0/2399762981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc1a0290 con 0x7f6fdc103bd0 2024-04-25T18:59:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.491+0000 7f6fe0e00700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fdc103bd0 0x7f6fdc19b280 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0000c00 tx=0x7f6fd00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.491+0000 7f6fd9600700 1 -- 172.21.15.45:0/2399762981 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0027070 con 0x7f6fdc103bd0 2024-04-25T18:59:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.491+0000 7f6fd9600700 1 -- 172.21.15.45:0/2399762981 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fd0022470 con 0x7f6fdc103bd0 2024-04-25T18:59:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.491+0000 7f6fd9600700 1 -- 172.21.15.45:0/2399762981 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0017650 con 0x7f6fdc103bd0 2024-04-25T18:59:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.491+0000 7f6fe2200700 1 -- 172.21.15.45:0/2399762981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc1a0520 con 0x7f6fdc103bd0 2024-04-25T18:59:52.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.491+0000 7f6fe2200700 1 -- 172.21.15.45:0/2399762981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc1a7e00 con 0x7f6fdc103bd0 2024-04-25T18:59:52.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.493+0000 7f6fe2200700 1 -- 172.21.15.45:0/2399762981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc069730 con 0x7f6fdc103bd0 2024-04-25T18:59:52.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.495+0000 7f6fd9600700 1 -- 172.21.15.45:0/2399762981 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f6fd002a070 con 0x7f6fdc103bd0 2024-04-25T18:59:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.496+0000 7f6fd9600700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6fc40a3f50 0x7f6fc40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:52.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.497+0000 7f6fd9600700 1 -- 172.21.15.45:0/2399762981 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f6fd00d1b80 con 0x7f6fdc103bd0 2024-04-25T18:59:52.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.497+0000 7f6fdbe00700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6fc40a3f50 0x7f6fc40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:52.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.498+0000 7f6fdbe00700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6fc40a3f50 0x7f6fc40a6410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc10e7a0 tx=0x7f6fc8009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:52.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.500+0000 7f6fd9600700 1 -- 172.21.15.45:0/2399762981 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fd0013070 con 0x7f6fdc103bd0 2024-04-25T18:59:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.806+0000 7f6fe2200700 1 -- 172.21.15.45:0/2399762981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fdc1a7fd0 con 0x7f6fdc103bd0 2024-04-25T18:59:52.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.807+0000 7f6fd9600700 1 -- 172.21.15.45:0/2399762981 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7f6fd009e000 con 0x7f6fdc103bd0 2024-04-25T18:59:52.810 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T18:59:52.810 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:59:18.321335+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24512,24505]},"id":1}]} 2024-04-25T18:59:52.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 -- 172.21.15.45:0/2399762981 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6fc40a3f50 msgr2=0x7f6fc40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:52.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6fc40a3f50 0x7f6fc40a6410 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc10e7a0 tx=0x7f6fc8009260 comp rx=0 tx=0).stop 2024-04-25T18:59:52.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 -- 172.21.15.45:0/2399762981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fdc103bd0 msgr2=0x7f6fdc19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:52.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fdc103bd0 0x7f6fdc19b280 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0000c00 tx=0x7f6fd00057c0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 -- 172.21.15.45:0/2399762981 shutdown_connections 2024-04-25T18:59:52.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f6fc40a3f50 0x7f6fc40a6410 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6fdc10cfb0 0x7f6fdc19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6fdc103bd0 0x7f6fdc19b280 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 --2- 172.21.15.45:0/2399762981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6fdc1044f0 0x7f6fdc19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:52.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 -- 172.21.15.45:0/2399762981 >> 172.21.15.45:0/2399762981 conn(0x7f6fdc0fd360 msgr2=0x7f6fdc0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:52.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 -- 172.21.15.45:0/2399762981 shutdown_connections 2024-04-25T18:59:52.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:52.811+0000 7f6fe2200700 1 -- 172.21.15.45:0/2399762981 wait complete. 2024-04-25T18:59:52.815 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-04-25T18:59:53.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:52 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2399762981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:53.183 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:52 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2399762981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:53.200 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:52 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2399762981' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T18:59:53.747 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T18:55:26.025536+0000', 'modified': '2024-04-25T18:59:18.321335+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24524, 'mds_1': 24512, 'mds_2': 24505}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6835/1201787807', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6834', 'nonce': 1201787807}, {'type': 'v1', 'addr': '172.21.15.79:6835', 'nonce': 1201787807}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24485': {'gid': 24485, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.62:6837/2985890701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6836', 'nonce': 2985890701}, {'type': 'v1', 'addr': '172.21.15.62:6837', 'nonce': 2985890701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24493': {'gid': 24493, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.79:6837/3290580293', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6836', 'nonce': 3290580293}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 3290580293}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.79:6839/235958219', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6838', 'nonce': 235958219}, {'type': 'v1', 'addr': '172.21.15.79:6839', 'nonce': 235958219}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24512': {'gid': 24512, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.45:6841/2490735053', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6840', 'nonce': 2490735053}, {'type': 'v1', 'addr': '172.21.15.45:6841', 'nonce': 2490735053}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24524': {'gid': 24524, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.62:6839/3288398937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.62:6838', 'nonce': 3288398937}, {'type': 'v1', 'addr': '172.21.15.62:6839', 'nonce': 3288398937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24524, 'qdb_cluster': [24524, 24512, 24505]} 2024-04-25T18:59:53.748 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T18:59:53.748 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T18:59:54.113 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:53 smithi062 ceph-mon[32844]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:59:54.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:53 smithi079 ceph-mon[33002]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:59:54.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:53 smithi045 ceph-mon[27776]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:59:54.250 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:56.127 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:55 smithi045 ceph-mon[27776]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:59:56.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:55 smithi079 ceph-mon[33002]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:59:56.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:55 smithi062 ceph-mon[32844]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T18:59:56.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.671+0000 7fd998c00700 1 -- 172.21.15.45:0/1114518488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd99410f1b0 msgr2=0x7fd994111620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:56.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.671+0000 7fd998c00700 1 --2- 172.21.15.45:0/1114518488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd99410f1b0 0x7fd994111620 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009f90 tx=0x7fd98c00b330 comp rx=0 tx=0).stop 2024-04-25T18:59:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.672+0000 7fd998c00700 1 -- 172.21.15.45:0/1114518488 shutdown_connections 2024-04-25T18:59:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.672+0000 7fd998c00700 1 --2- 172.21.15.45:0/1114518488 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd994074e70 0x7fd994075250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.672+0000 7fd998c00700 1 --2- 172.21.15.45:0/1114518488 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd99410f1b0 0x7fd994111620 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.672+0000 7fd998c00700 1 --2- 172.21.15.45:0/1114518488 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd994075790 0x7fd99410ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.672+0000 7fd998c00700 1 -- 172.21.15.45:0/1114518488 >> 172.21.15.45:0/1114518488 conn(0x7fd9940fe4c0 msgr2=0x7fd9941008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.672+0000 7fd998c00700 1 -- 172.21.15.45:0/1114518488 shutdown_connections 2024-04-25T18:59:56.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.672+0000 7fd998c00700 1 -- 172.21.15.45:0/1114518488 wait complete. 2024-04-25T18:59:56.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.673+0000 7fd998c00700 1 Processor -- start 2024-04-25T18:59:56.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.673+0000 7fd998c00700 1 -- start start 2024-04-25T18:59:56.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.673+0000 7fd998c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd994074e70 0x7fd99419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.676 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.673+0000 7fd998c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd994075790 0x7fd99419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.673+0000 7fd998c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd99410f1b0 0x7fd99419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.673+0000 7fd998c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd994114da0 con 0x7fd994074e70 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.673+0000 7fd998c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd994114c20 con 0x7fd994075790 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.673+0000 7fd998c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd994114f20 con 0x7fd99410f1b0 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd993400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd994074e70 0x7fd99419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd993400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd994074e70 0x7fd99419b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46356/0 (socket says 172.21.15.45:46356) 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd993400700 1 -- 172.21.15.45:0/992042861 learned_addr learned my addr 172.21.15.45:0/992042861 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd992a00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd994075790 0x7fd99419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd993e00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd99410f1b0 0x7fd99419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd993400700 1 -- 172.21.15.45:0/992042861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd99410f1b0 msgr2=0x7fd99419fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd993400700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd99410f1b0 0x7fd99419fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd993400700 1 -- 172.21.15.45:0/992042861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd994075790 msgr2=0x7fd99419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd993400700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd994075790 0x7fd99419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.677 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd993400700 1 -- 172.21.15.45:0/992042861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9941a01d0 con 0x7fd994074e70 2024-04-25T18:59:56.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.674+0000 7fd992a00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd994075790 0x7fd99419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:56.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.675+0000 7fd993400700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd994074e70 0x7fd99419b210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd98400e9c0 tx=0x7fd98400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:56.678 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.675+0000 7fd98be00700 1 -- 172.21.15.45:0/992042861 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984004a30 con 0x7fd994074e70 2024-04-25T18:59:56.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.675+0000 7fd98be00700 1 -- 172.21.15.45:0/992042861 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd98400cc70 con 0x7fd994074e70 2024-04-25T18:59:56.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.675+0000 7fd998c00700 1 -- 172.21.15.45:0/992042861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9941a04c0 con 0x7fd994074e70 2024-04-25T18:59:56.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.675+0000 7fd98be00700 1 -- 172.21.15.45:0/992042861 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984005490 con 0x7fd994074e70 2024-04-25T18:59:56.679 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.675+0000 7fd998c00700 1 -- 172.21.15.45:0/992042861 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9941a7d40 con 0x7fd994074e70 2024-04-25T18:59:56.681 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.679+0000 7fd98be00700 1 -- 172.21.15.45:0/992042861 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7fd984004bd0 con 0x7fd994074e70 2024-04-25T18:59:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.680+0000 7fd9a4aa6b80 1 Processor -- start 2024-04-25T18:59:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.680+0000 7fd9a4aa6b80 1 -- start start 2024-04-25T18:59:56.682 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd98be00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9780a3ed0 0x7fd9780a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd98be00700 1 -- 172.21.15.45:0/992042861 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fd9840d0bf0 con 0x7fd994074e70 2024-04-25T18:59:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96add27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96addd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96ade0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c96ad25e40 con 0x55c96adddc40 2024-04-25T18:59:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c96accc6e0 con 0x55c96add2d00 2024-04-25T18:59:56.683 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c96ad21e00 con 0x55c96add23e0 2024-04-25T18:59:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd993e00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9780a3ed0 0x7fd9780a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd993400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96add27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.682+0000 7fd993e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96ade0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.682+0000 7fd992a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96addd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.681+0000 7fd993400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96add27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56014/0 (socket says 172.21.15.45:56014) 2024-04-25T18:59:56.684 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.682+0000 7fd993400700 1 -- 172.21.15.45:0/2639269070 learned_addr learned my addr 172.21.15.45:0/2639269070 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:56.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.682+0000 7fd992a00700 1 -- 172.21.15.45:0/2639269070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 msgr2=0x55c96add27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:56.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.682+0000 7fd992a00700 1 --2- 172.21.15.45:0/2639269070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96add27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.682+0000 7fd992a00700 1 -- 172.21.15.45:0/2639269070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 msgr2=0x55c96ade0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:56.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.682+0000 7fd992a00700 1 --2- 172.21.15.45:0/2639269070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96ade0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.682+0000 7fd992a00700 1 -- 172.21.15.45:0/2639269070 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96acc2630 con 0x55c96add2d00 2024-04-25T18:59:56.685 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.683+0000 7fd992a00700 1 --2- 172.21.15.45:0/2639269070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96addd700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd9800114c0 tx=0x7fd980009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c967ea1e64e31dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:56.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.683+0000 7fd993e00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9780a3ed0 0x7fd9780a6390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009550 tx=0x7fd98c0093a0 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:56.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.683+0000 7fd988c00700 1 -- 172.21.15.45:0/2639269070 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9800112f0 con 0x55c96add2d00 2024-04-25T18:59:56.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.683+0000 7fd988c00700 1 -- 172.21.15.45:0/2639269070 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd980004500 con 0x55c96add2d00 2024-04-25T18:59:56.686 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.683+0000 7fd988c00700 1 -- 172.21.15.45:0/2639269070 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98001a6a0 con 0x55c96add2d00 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.684+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/2639269070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 msgr2=0x55c96addd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.684+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/2639269070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96addd700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd9800114c0 tx=0x7fd980009f90 comp rx=0 tx=0).stop 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.684+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/2639269070 shutdown_connections 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.684+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/2639269070 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96ade0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.684+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/2639269070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96add27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.684+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/2639269070 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96addd700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.684+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/2639269070 >> 172.21.15.45:0/2639269070 conn(0x55c96ad01550 msgr2=0x55c96adcea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.684+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/2639269070 shutdown_connections 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.684+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/2639269070 wait complete. 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.685+0000 7fd9a4aa6b80 1 -- start start 2024-04-25T18:59:56.687 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.685+0000 7fd9a4aa6b80 1 Processor -- start 2024-04-25T18:59:56.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96add23e0 0x55c96addd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96adda570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96adddc40 0x55c96addaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c96ade07c0 con 0x55c96add23e0 2024-04-25T18:59:56.689 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c96acdf810 con 0x55c96add2d00 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c96ade0940 con 0x55c96adddc40 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd9a4aa6b80 10 client.0 _ll_register_callbacks cb 0x55c96acbe6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd992a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96adda570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd993400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96add23e0 0x55c96addd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd992a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96adda570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:34166/0 (socket says 172.21.15.45:34166) 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd992a00700 1 -- 172.21.15.45:0/3190140789 learned_addr learned my addr 172.21.15.45:0/3190140789 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.686+0000 7fd993e00700 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96adddc40 0x55c96addaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd992a00700 1 -- 172.21.15.45:0/3190140789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96adddc40 msgr2=0x55c96addaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd992a00700 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96adddc40 0x55c96addaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd992a00700 1 -- 172.21.15.45:0/3190140789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96add23e0 msgr2=0x55c96addd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd992a00700 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96add23e0 0x55c96addd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd992a00700 1 -- 172.21.15.45:0/3190140789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96ac73880 con 0x55c96add2d00 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd993e00700 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96adddc40 0x55c96addaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:56.690 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd993400700 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96add23e0 0x55c96addd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T18:59:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd992a00700 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96adda570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd980003de0 tx=0x7fd980004280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92ff0c4b01065101 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd97d600700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T18:59:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd97d600700 1 -- 172.21.15.45:0/3190140789 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd980003a90 con 0x55c96add2d00 2024-04-25T18:59:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd97d600700 1 -- 172.21.15.45:0/3190140789 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fd980005030 con 0x55c96add2d00 2024-04-25T18:59:56.691 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.687+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3190140789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd980004500 con 0x55c96add2d00 2024-04-25T18:59:56.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.688+0000 7fd97d600700 1 -- 172.21.15.45:0/3190140789 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd980005230 con 0x55c96add2d00 2024-04-25T18:59:56.692 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.688+0000 7fd9a4aa6b80 10 client.25238 fetch_fsmap learned FSMap version 26 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.688+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3190140789 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c96acbcaa0 con 0x55c96add2d00 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.688+0000 7fd97d600700 1 -- 172.21.15.45:0/3190140789 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd980004500 con 0x55c96add2d00 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.688+0000 7fd97d600700 1 -- 172.21.15.45:0/3190140789 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd980018020 con 0x55c96add2d00 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.688+0000 7fd9a4aa6b80 10 client.25238 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.688+0000 7fd9a4aa6b80 10 client.25238 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.689+0000 7fd9a4aa6b80 20 client.25238 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.689+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x55c96aded1c0 0x55c96adee630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.689+0000 7fd9a4aa6b80 4 client.25238 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.689+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3190140789 --> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96ad33630 con 0x55c96aded1c0 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.689+0000 7fd993400700 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x55c96aded1c0 0x55c96adee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.689+0000 7fd993400700 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x55c96aded1c0 0x55c96adee630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11d1312c32a20ea5 server_cookie=1104aa7c4e975099 in_seq=0 out_seq=0 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.689+0000 7fd97d600700 10 client.25238 ms_handle_connect on v2:172.21.15.79:6838/235958219 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.690+0000 7fd97d600700 1 -- 172.21.15.45:0/3190140789 <== mds.2 v2:172.21.15.79:6838/235958219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c96ad33630 con 0x55c96aded1c0 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.690+0000 7fd97d600700 10 client.25238 handle_command_reply: tid=0 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.690+0000 7fd9a4aa6b80 1 client.25238 shutdown 2024-04-25T18:59:56.693 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3190140789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 msgr2=0x55c96adda570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96adda570 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd980003de0 tx=0x7fd980004280 comp rx=0 tx=0).stop 2024-04-25T18:59:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3190140789 shutdown_connections 2024-04-25T18:59:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x55c96aded1c0 0x55c96adee630 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96adddc40 0x55c96addaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.694 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96add23e0 0x55c96addd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd97d600700 0 client.25238 ms_handle_reset on v2:172.21.15.79:6838/235958219 2024-04-25T18:59:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/3190140789 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96adda570 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3190140789 >> 172.21.15.45:0/3190140789 conn(0x55c96ad01550 msgr2=0x55c96adce680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3190140789 shutdown_connections 2024-04-25T18:59:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3190140789 wait complete. 2024-04-25T18:59:56.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.691+0000 7fd9a4aa6b80 20 client.25238 trim_cache size 0 max 16384 2024-04-25T18:59:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.724+0000 7fd9a4aa6b80 1 Processor -- start 2024-04-25T18:59:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.724+0000 7fd9a4aa6b80 1 -- start start 2024-04-25T18:59:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.724+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add23e0 0x55c96adceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.724+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add2d00 0x55c96add7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.724+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96add7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.724+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c96adec970 con 0x55c96adddc40 2024-04-25T18:59:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.724+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c96aded040 con 0x55c96add23e0 2024-04-25T18:59:56.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.724+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c96aded710 con 0x55c96add2d00 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.725+0000 7fd993400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add23e0 0x55c96adceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.725+0000 7fd993e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96add7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.725+0000 7fd993400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add23e0 0x55c96adceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:34182/0 (socket says 172.21.15.45:34182) 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.725+0000 7fd993400700 1 -- 172.21.15.45:0/3500779222 learned_addr learned my addr 172.21.15.45:0/3500779222 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.725+0000 7fd992a00700 1 --2- 172.21.15.45:0/3500779222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add2d00 0x55c96add7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.725+0000 7fd993400700 1 -- 172.21.15.45:0/3500779222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add2d00 msgr2=0x55c96add7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.725+0000 7fd993400700 1 --2- 172.21.15.45:0/3500779222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add2d00 0x55c96add7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.725+0000 7fd993400700 1 -- 172.21.15.45:0/3500779222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 msgr2=0x55c96add7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.725+0000 7fd993400700 1 --2- 172.21.15.45:0/3500779222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96add7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.725+0000 7fd993400700 1 -- 172.21.15.45:0/3500779222 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96addda70 con 0x55c96add23e0 2024-04-25T18:59:56.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.726+0000 7fd993400700 1 --2- 172.21.15.45:0/3500779222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add23e0 0x55c96adceb70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd98409c870 tx=0x7fd984004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=984006c345fa66ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:56.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.726+0000 7fd97fe00700 1 -- 172.21.15.45:0/3500779222 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9840a0c90 con 0x55c96add23e0 2024-04-25T18:59:56.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.726+0000 7fd97fe00700 1 -- 172.21.15.45:0/3500779222 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9840a5550 con 0x55c96add23e0 2024-04-25T18:59:56.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.726+0000 7fd97fe00700 1 -- 172.21.15.45:0/3500779222 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98409cda0 con 0x55c96add23e0 2024-04-25T18:59:56.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.727+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3500779222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add23e0 msgr2=0x55c96adceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:56.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.727+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/3500779222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add23e0 0x55c96adceb70 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd98409c870 tx=0x7fd984004f70 comp rx=0 tx=0).stop 2024-04-25T18:59:56.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.727+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3500779222 shutdown_connections 2024-04-25T18:59:56.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.727+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/3500779222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add23e0 0x55c96adceb70 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.727+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/3500779222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96add7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.727+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/3500779222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add2d00 0x55c96add7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.727+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3500779222 >> 172.21.15.45:0/3500779222 conn(0x55c96ad01550 msgr2=0x55c96adec210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:56.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.727+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3500779222 shutdown_connections 2024-04-25T18:59:56.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.727+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/3500779222 wait complete. 2024-04-25T18:59:56.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.727+0000 7fd9a4aa6b80 1 -- start start 2024-04-25T18:59:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.728+0000 7fd9a4aa6b80 1 Processor -- start 2024-04-25T18:59:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.728+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96adf3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.728+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96adf4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.728+0000 7fd9a4aa6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96ae5bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.728+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c96adee4b0 con 0x55c96adddc40 2024-04-25T18:59:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.728+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c96adedde0 con 0x55c96add2d00 2024-04-25T18:59:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.728+0000 7fd9a4aa6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c96adeeb80 con 0x55c96add23e0 2024-04-25T18:59:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.729+0000 7fd9a4aa6b80 10 client.0 _ll_register_callbacks cb 0x55c96acbe6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T18:59:56.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.729+0000 7fd993400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96adf3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.729+0000 7fd993e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96ae5bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.729+0000 7fd993400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96adf3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56030/0 (socket says 172.21.15.45:56030) 2024-04-25T18:59:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.729+0000 7fd993400700 1 -- 172.21.15.45:0/1358766932 learned_addr learned my addr 172.21.15.45:0/1358766932 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T18:59:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.729+0000 7fd992a00700 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96adf4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.729+0000 7fd993400700 1 -- 172.21.15.45:0/1358766932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 msgr2=0x55c96adf4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.729+0000 7fd993400700 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96adf4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.729+0000 7fd993400700 1 -- 172.21.15.45:0/1358766932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 msgr2=0x55c96ae5bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.729+0000 7fd993400700 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96ae5bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.730+0000 7fd993400700 1 -- 172.21.15.45:0/1358766932 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96ae5c040 con 0x55c96add23e0 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.730+0000 7fd993400700 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96adf3b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd9840bd0d0 tx=0x7fd98400cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45caea2c827dc48d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.730+0000 7fd97d600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.730+0000 7fd97d600700 1 -- 172.21.15.45:0/1358766932 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984065070 con 0x55c96add23e0 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.730+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/1358766932 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9840a5550 con 0x55c96add23e0 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.730+0000 7fd97d600700 1 -- 172.21.15.45:0/1358766932 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fd984005650 con 0x55c96add23e0 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd97d600700 1 -- 172.21.15.45:0/1358766932 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd9840672c0 con 0x55c96add23e0 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd9a4aa6b80 10 client.25252 fetch_fsmap learned FSMap version 26 2024-04-25T18:59:56.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/1358766932 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55c96ae5d2a0 con 0x55c96add23e0 2024-04-25T18:59:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd97d600700 1 -- 172.21.15.45:0/1358766932 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9840a5550 con 0x55c96add23e0 2024-04-25T18:59:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd97d600700 1 -- 172.21.15.45:0/1358766932 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd9840ce440 con 0x55c96add23e0 2024-04-25T18:59:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd9a4aa6b80 10 client.25252 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T18:59:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd9a4aa6b80 10 client.25252 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T18:59:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd9a4aa6b80 20 client.25252 populate_metadata read hostname 'smithi045' 2024-04-25T18:59:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x55c96ae61910 0x55c96ae60620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T18:59:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd9a4aa6b80 4 client.25252 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T18:59:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.731+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/1358766932 --> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] -- command(tid 0: {"prefix": "status"}) -- 0x55c96ad2d6d0 con 0x55c96ae61910 2024-04-25T18:59:56.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.732+0000 7fd992a00700 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x55c96ae61910 0x55c96ae60620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T18:59:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.732+0000 7fd992a00700 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x55c96ae61910 0x55c96ae60620 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=845c8061987153ff server_cookie=dbb76b72df045a1b in_seq=0 out_seq=0 2024-04-25T18:59:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.732+0000 7fd97d600700 10 client.25252 ms_handle_connect on v2:172.21.15.79:6838/235958219 2024-04-25T18:59:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd97d600700 1 -- 172.21.15.45:0/1358766932 <== mds.2 v2:172.21.15.79:6838/235958219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c96ad2d6d0 con 0x55c96ae61910 2024-04-25T18:59:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd97d600700 10 client.25252 handle_command_reply: tid=0 2024-04-25T18:59:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd9a4aa6b80 1 client.25252 shutdown 2024-04-25T18:59:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/1358766932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 msgr2=0x55c96adf3b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96adf3b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd9840bd0d0 tx=0x7fd98400cea0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/1358766932 shutdown_connections 2024-04-25T18:59:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x55c96ae61910 0x55c96ae60620 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.736 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c96adddc40 0x55c96ae5bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c96add23e0 0x55c96adf3b20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd9a4aa6b80 1 --2- 172.21.15.45:0/1358766932 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c96add2d00 0x55c96adf4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd97d600700 0 client.25252 ms_handle_reset on v2:172.21.15.79:6838/235958219 2024-04-25T18:59:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.733+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/1358766932 >> 172.21.15.45:0/1358766932 conn(0x55c96ad01550 msgr2=0x55c96addbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.734+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/1358766932 shutdown_connections 2024-04-25T18:59:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.734+0000 7fd9a4aa6b80 1 -- 172.21.15.45:0/1358766932 wait complete. 2024-04-25T18:59:56.737 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.734+0000 7fd9a4aa6b80 20 client.25252 trim_cache size 0 max 16384 2024-04-25T18:59:56.737 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T18:59:56.737 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 2, 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24505, 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 253.359353803, 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 95, 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 95, 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 304.61072249199998 2024-04-25T18:59:56.738 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T18:59:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.735+0000 7fd998c00700 1 -- 172.21.15.45:0/992042861 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9780a3ed0 msgr2=0x7fd9780a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.735+0000 7fd998c00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9780a3ed0 0x7fd9780a6390 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009550 tx=0x7fd98c0093a0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.735+0000 7fd998c00700 1 -- 172.21.15.45:0/992042861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd994074e70 msgr2=0x7fd99419b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T18:59:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.735+0000 7fd998c00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd994074e70 0x7fd99419b210 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fd98400e9c0 tx=0x7fd98400ecd0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.735+0000 7fd998c00700 1 -- 172.21.15.45:0/992042861 shutdown_connections 2024-04-25T18:59:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.735+0000 7fd998c00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9780a3ed0 0x7fd9780a6390 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.735+0000 7fd998c00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd99410f1b0 0x7fd99419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.735+0000 7fd998c00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd994074e70 0x7fd99419b210 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.735+0000 7fd998c00700 1 --2- 172.21.15.45:0/992042861 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd994075790 0x7fd99419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T18:59:56.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.735+0000 7fd998c00700 1 -- 172.21.15.45:0/992042861 >> 172.21.15.45:0/992042861 conn(0x7fd9940fe4c0 msgr2=0x7fd9940ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T18:59:56.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.736+0000 7fd998c00700 1 -- 172.21.15.45:0/992042861 shutdown_connections 2024-04-25T18:59:56.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T18:59:56.736+0000 7fd998c00700 1 -- 172.21.15.45:0/992042861 wait complete. 2024-04-25T18:59:57.652 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.a status 2024-04-25T18:59:58.150 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T18:59:58.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:57 smithi079 ceph-mon[33002]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T18:59:58.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:57 smithi045 ceph-mon[27776]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T18:59:58.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:57 smithi062 ceph-mon[32844]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T18:59:59.963 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 18:59:59 smithi045 ceph-mon[27776]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T19:00:00.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 18:59:59 smithi079 ceph-mon[33002]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T19:00:00.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 18:59:59 smithi062 ceph-mon[32844]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T19:00:00.465 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:00:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T19:00:00.023+0000 7f9258c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:00:00.465 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:00:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T19:00:00.023+0000 7f9258c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:00:00.465 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:00:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T19:00:00.023+0000 7f9258c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:00:00.465 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:00:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T19:00:00.023+0000 7f9258c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:00:00.465 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:00:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T19:00:00.023+0000 7f9258c00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T19:00:00.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.712+0000 7f7debe00700 1 -- 172.21.15.45:0/4081813068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dec10cfb0 msgr2=0x7f7dec10f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.712+0000 7f7debe00700 1 --2- 172.21.15.45:0/4081813068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dec10cfb0 0x7f7dec10f470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7de400b0a0 tx=0x7f7de400b3b0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.712+0000 7f7debe00700 1 -- 172.21.15.45:0/4081813068 shutdown_connections 2024-04-25T19:00:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.712+0000 7f7debe00700 1 --2- 172.21.15.45:0/4081813068 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dec103b80 0x7f7dec103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.712+0000 7f7debe00700 1 --2- 172.21.15.45:0/4081813068 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dec10cfb0 0x7f7dec10f470 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.712+0000 7f7debe00700 1 --2- 172.21.15.45:0/4081813068 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dec1044a0 0x7f7dec10c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.712+0000 7f7debe00700 1 -- 172.21.15.45:0/4081813068 >> 172.21.15.45:0/4081813068 conn(0x7f7dec0fd3d0 msgr2=0x7f7dec0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.712+0000 7f7debe00700 1 -- 172.21.15.45:0/4081813068 shutdown_connections 2024-04-25T19:00:00.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.712+0000 7f7debe00700 1 -- 172.21.15.45:0/4081813068 wait complete. 2024-04-25T19:00:00.716 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.713+0000 7f7debe00700 1 Processor -- start 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.713+0000 7f7debe00700 1 -- start start 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7debe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dec103b80 0x7f7dec19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7debe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dec1044a0 0x7f7dec19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7debe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dec10cfb0 0x7f7dec19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7debe00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec112be0 con 0x7f7dec1044a0 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7debe00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec112a60 con 0x7f7dec103b80 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7debe00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7dec112d60 con 0x7f7dec10cfb0 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7deaa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dec103b80 0x7f7dec19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dec1044a0 0x7f7dec19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7deaa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dec103b80 0x7f7dec19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:34228/0 (socket says 172.21.15.45:34228) 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7deaa00700 1 -- 172.21.15.45:0/2928342609 learned_addr learned my addr 172.21.15.45:0/2928342609 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.714+0000 7f7deb400700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dec10cfb0 0x7f7dec19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.715+0000 7f7de3e00700 1 -- 172.21.15.45:0/2928342609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dec10cfb0 msgr2=0x7f7dec19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.715+0000 7f7de3e00700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dec10cfb0 0x7f7dec19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.715+0000 7f7de3e00700 1 -- 172.21.15.45:0/2928342609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dec103b80 msgr2=0x7f7dec19b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.715+0000 7f7de3e00700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dec103b80 0x7f7dec19b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.715+0000 7f7de3e00700 1 -- 172.21.15.45:0/2928342609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dec1a0290 con 0x7f7dec1044a0 2024-04-25T19:00:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.715+0000 7f7deaa00700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dec103b80 0x7f7dec19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.715+0000 7f7de3e00700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dec1044a0 0x7f7dec19b7c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400e9c0 tx=0x7f7dd400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.715+0000 7f7de3400700 1 -- 172.21.15.45:0/2928342609 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40095c0 con 0x7f7dec1044a0 2024-04-25T19:00:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.715+0000 7f7de3400700 1 -- 172.21.15.45:0/2928342609 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd4004830 con 0x7f7dec1044a0 2024-04-25T19:00:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.715+0000 7f7debe00700 1 -- 172.21.15.45:0/2928342609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dec1a0580 con 0x7f7dec1044a0 2024-04-25T19:00:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.716+0000 7f7debe00700 1 -- 172.21.15.45:0/2928342609 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dec1a7e00 con 0x7f7dec1044a0 2024-04-25T19:00:00.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.716+0000 7f7de3400700 1 -- 172.21.15.45:0/2928342609 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd400cd70 con 0x7f7dec1044a0 2024-04-25T19:00:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.720+0000 7f7de3400700 1 -- 172.21.15.45:0/2928342609 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138873+0+0 (secure 0 0 0) 0x7f7dd4005040 con 0x7f7dec1044a0 2024-04-25T19:00:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.721+0000 7f7dfc09ab80 1 Processor -- start 2024-04-25T19:00:00.723 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.721+0000 7f7dfc09ab80 1 -- start start 2024-04-25T19:00:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3ae1d0 0x56171e3b8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3b9080 0x56171e3bb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56171e300e40 con 0x56171e3ad8b0 2024-04-25T19:00:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56171e2a76e0 con 0x56171e3b9080 2024-04-25T19:00:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56171e2fce00 con 0x56171e3ae1d0 2024-04-25T19:00:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7deb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7deb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46444/0 (socket says 172.21.15.45:46444) 2024-04-25T19:00:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7deb400700 1 -- 172.21.15.45:0/2625355444 learned_addr learned my addr 172.21.15.45:0/2625355444 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7deaa00700 1 --2- 172.21.15.45:0/2625355444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3ae1d0 0x56171e3b8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7de3e00700 1 --2- 172.21.15.45:0/2625355444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3b9080 0x56171e3bb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.722+0000 7f7de3400700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f7dd00a3f50 0x7f7dd00a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.723+0000 7f7deb400700 1 -- 172.21.15.45:0/2625355444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3ae1d0 msgr2=0x56171e3b8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.723+0000 7f7deb400700 1 --2- 172.21.15.45:0/2625355444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3ae1d0 0x56171e3b8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.723+0000 7f7deb400700 1 -- 172.21.15.45:0/2625355444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3b9080 msgr2=0x56171e3bb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.723+0000 7f7deb400700 1 --2- 172.21.15.45:0/2625355444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3b9080 0x56171e3bb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.723+0000 7f7deb400700 1 -- 172.21.15.45:0/2625355444 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171e29d630 con 0x56171e3ad8b0 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.723+0000 7f7de3400700 1 -- 172.21.15.45:0/2928342609 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f7dd40d0750 con 0x7f7dec1044a0 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.723+0000 7f7deb400700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f7dd00a3f50 0x7f7dd00a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.723+0000 7f7deb400700 1 --2- 172.21.15.45:0/2625355444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3adc90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009360 tx=0x7f7de4009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a303cb001016074 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.724+0000 7f7ddbe00700 1 -- 172.21.15.45:0/2625355444 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de402e070 con 0x56171e3ad8b0 2024-04-25T19:00:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.724+0000 7f7ddbe00700 1 -- 172.21.15.45:0/2625355444 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7de4024070 con 0x56171e3ad8b0 2024-04-25T19:00:00.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.724+0000 7f7ddbe00700 1 -- 172.21.15.45:0/2625355444 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7de4005240 con 0x56171e3ad8b0 2024-04-25T19:00:00.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/2625355444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 msgr2=0x56171e3adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/2625355444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3adc90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009360 tx=0x7f7de4009390 comp rx=0 tx=0).stop 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7deb400700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f7dd00a3f50 0x7f7dd00a6410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7de402b000 tx=0x7f7de402c000 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/2625355444 shutdown_connections 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/2625355444 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3b9080 0x56171e3bb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/2625355444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3adc90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/2625355444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3ae1d0 0x56171e3b8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/2625355444 >> 172.21.15.45:0/2625355444 conn(0x56171e2dcf00 msgr2=0x56171e3a9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/2625355444 shutdown_connections 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/2625355444 wait complete. 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.725+0000 7f7dfc09ab80 1 -- start start 2024-04-25T19:00:00.728 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.726+0000 7f7dfc09ab80 1 Processor -- start 2024-04-25T19:00:00.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.726+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3b8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.726+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ae1d0 0x56171e3b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.726+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3b9080 0x56171e3b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.726+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56171e3bbc00 con 0x56171e3ae1d0 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.726+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56171e2ba810 con 0x56171e3ad8b0 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.726+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56171e3bbd80 con 0x56171e3b9080 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.726+0000 7f7dfc09ab80 10 client.0 _ll_register_callbacks cb 0x56171e2996d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ae1d0 0x56171e3b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ae1d0 0x56171e3b5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46446/0 (socket says 172.21.15.45:46446) 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7de3e00700 1 -- 172.21.15.45:0/1206091260 learned_addr learned my addr 172.21.15.45:0/1206091260 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7deaa00700 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7de3e00700 1 -- 172.21.15.45:0/1206091260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3b9080 msgr2=0x56171e3b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7de3e00700 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3b9080 0x56171e3b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7de3e00700 1 -- 172.21.15.45:0/1206091260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 msgr2=0x56171e3b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7de3e00700 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7de3e00700 1 -- 172.21.15.45:0/1206091260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171e24e880 con 0x56171e3ae1d0 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7deaa00700 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.727+0000 7f7de3e00700 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ae1d0 0x56171e3b5a20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40988a0 tx=0x7f7dd40af140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce395ae2761990ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:00.730 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.728+0000 7f7dd8c00700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:00:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.728+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1206091260 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd401c070 con 0x56171e3ae1d0 2024-04-25T19:00:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.728+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1206091260 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f7dd4004220 con 0x56171e3ae1d0 2024-04-25T19:00:00.731 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.728+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1206091260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7de4024070 con 0x56171e3ae1d0 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.728+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1206091260 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7dd409cb40 con 0x56171e3ae1d0 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.728+0000 7f7dfc09ab80 10 client.15141 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.728+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1206091260 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56171e297aa0 con 0x56171e3ae1d0 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.729+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1206091260 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7de4024070 con 0x56171e3ae1d0 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.729+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1206091260 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7dd40b5bc0 con 0x56171e3ae1d0 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.729+0000 7f7dfc09ab80 10 client.15141 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.729+0000 7f7dfc09ab80 10 client.15141 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.729+0000 7f7dfc09ab80 20 client.15141 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.729+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x56171e3c88c0 0x56171e3c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.729+0000 7f7dfc09ab80 4 client.15141 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.729+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1206091260 --> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56171e30e630 con 0x56171e3c88c0 2024-04-25T19:00:00.732 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.729+0000 7f7deaa00700 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x56171e3c88c0 0x56171e3c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.730+0000 7f7deaa00700 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x56171e3c88c0 0x56171e3c9d30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=272fc13f43b1c4b4 server_cookie=c2191a32f3acc1ae in_seq=0 out_seq=0 2024-04-25T19:00:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.730+0000 7f7dd8c00700 10 client.15141 ms_handle_connect on v2:172.21.15.45:6840/2490735053 2024-04-25T19:00:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.731+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1206091260 <== mds.1 v2:172.21.15.45:6840/2490735053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56171e30e630 con 0x56171e3c88c0 2024-04-25T19:00:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.731+0000 7f7dd8c00700 10 client.15141 handle_command_reply: tid=0 2024-04-25T19:00:00.733 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.731+0000 7f7dfc09ab80 1 client.15141 shutdown 2024-04-25T19:00:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.731+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1206091260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ae1d0 msgr2=0x56171e3b5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.731+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ae1d0 0x56171e3b5a20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40988a0 tx=0x7f7dd40af140 comp rx=0 tx=0).stop 2024-04-25T19:00:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.732+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1206091260 shutdown_connections 2024-04-25T19:00:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.732+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x56171e3c88c0 0x56171e3c9d30 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.732+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.732+0000 7f7dd8c00700 0 client.15141 ms_handle_reset on v2:172.21.15.45:6840/2490735053 2024-04-25T19:00:00.734 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.732+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3b9080 0x56171e3b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.732+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1206091260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ae1d0 0x56171e3b5a20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.732+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1206091260 >> 172.21.15.45:0/1206091260 conn(0x56171e2dcf00 msgr2=0x56171e3a9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.732+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1206091260 shutdown_connections 2024-04-25T19:00:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.732+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1206091260 wait complete. 2024-04-25T19:00:00.735 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.732+0000 7f7dfc09ab80 20 client.15141 trim_cache size 0 max 16384 2024-04-25T19:00:00.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.755+0000 7f7dfc09ab80 1 Processor -- start 2024-04-25T19:00:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.755+0000 7f7dfc09ab80 1 -- start start 2024-04-25T19:00:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.755+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3b5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.755+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ae1d0 0x56171e3d31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.755+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3b9080 0x56171e3d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.755+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56171e3c8070 con 0x56171e3ad8b0 2024-04-25T19:00:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.755+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56171e3c8740 con 0x56171e3ae1d0 2024-04-25T19:00:00.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.755+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56171e3c8e10 con 0x56171e3b9080 2024-04-25T19:00:00.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7deaa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7de3e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ae1d0 0x56171e3d31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7deaa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3b5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46452/0 (socket says 172.21.15.45:46452) 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7deaa00700 1 -- 172.21.15.45:0/1663942830 learned_addr learned my addr 172.21.15.45:0/1663942830 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7deb400700 1 --2- 172.21.15.45:0/1663942830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3b9080 0x56171e3d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7de3e00700 1 -- 172.21.15.45:0/1663942830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3b9080 msgr2=0x56171e3d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7de3e00700 1 --2- 172.21.15.45:0/1663942830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3b9080 0x56171e3d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7de3e00700 1 -- 172.21.15.45:0/1663942830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 msgr2=0x56171e3b5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7de3e00700 1 --2- 172.21.15.45:0/1663942830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3b5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7de3e00700 1 -- 172.21.15.45:0/1663942830 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56171e3ab2c0 con 0x56171e3ae1d0 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.756+0000 7f7de3e00700 1 --2- 172.21.15.45:0/1663942830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ae1d0 0x56171e3d31a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400c5b0 tx=0x7f7dd40cde00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432f60f92a52be7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.757+0000 7f7ddb400700 1 -- 172.21.15.45:0/1663942830 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd401c070 con 0x56171e3ae1d0 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.757+0000 7f7ddb400700 1 -- 172.21.15.45:0/1663942830 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd40b4070 con 0x56171e3ae1d0 2024-04-25T19:00:00.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.757+0000 7f7ddb400700 1 -- 172.21.15.45:0/1663942830 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40b5bc0 con 0x56171e3ae1d0 2024-04-25T19:00:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.758+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1663942830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ae1d0 msgr2=0x56171e3d31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.758+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1663942830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ae1d0 0x56171e3d31a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400c5b0 tx=0x7f7dd40cde00 comp rx=0 tx=0).stop 2024-04-25T19:00:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.758+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1663942830 shutdown_connections 2024-04-25T19:00:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.758+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1663942830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3b9080 0x56171e3d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.758+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1663942830 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3ad8b0 0x56171e3b5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.758+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1663942830 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ae1d0 0x56171e3d31a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.758+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1663942830 >> 172.21.15.45:0/1663942830 conn(0x56171e2dcf00 msgr2=0x56171e3c4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.758+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1663942830 shutdown_connections 2024-04-25T19:00:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.758+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1663942830 wait complete. 2024-04-25T19:00:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.759+0000 7f7dfc09ab80 1 -- start start 2024-04-25T19:00:00.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.759+0000 7f7dfc09ab80 1 Processor -- start 2024-04-25T19:00:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.759+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3cf290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3ae1d0 0x56171e3cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7dfc09ab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3b9080 0x56171e42e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56171e3c9bb0 con 0x56171e3b9080 2024-04-25T19:00:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56171e3c94e0 con 0x56171e3ad8b0 2024-04-25T19:00:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7dfc09ab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x56171e3ca280 con 0x56171e3ae1d0 2024-04-25T19:00:00.762 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7dfc09ab80 10 client.0 _ll_register_callbacks cb 0x56171e2996d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7deaa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7deb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3b9080 0x56171e42e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7deaa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:34270/0 (socket says 172.21.15.45:34270) 2024-04-25T19:00:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7deb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3b9080 0x56171e42e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46456/0 (socket says 172.21.15.45:46456) 2024-04-25T19:00:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7deaa00700 1 -- 172.21.15.45:0/1380044872 learned_addr learned my addr 172.21.15.45:0/1380044872 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:00.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7de3e00700 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3ae1d0 0x56171e3cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7deaa00700 1 -- 172.21.15.45:0/1380044872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3ae1d0 msgr2=0x56171e3cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7deaa00700 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3ae1d0 0x56171e3cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7deaa00700 1 -- 172.21.15.45:0/1380044872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3b9080 msgr2=0x56171e42e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7deaa00700 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3b9080 0x56171e42e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.760+0000 7f7deaa00700 1 -- 172.21.15.45:0/1380044872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56171e42ef60 con 0x56171e3ad8b0 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.761+0000 7f7deaa00700 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3cf290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc011a00 tx=0x7f7ddc011a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1058dcb7b2a205f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.761+0000 7f7dd8c00700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.761+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1380044872 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc003680 con 0x56171e3ad8b0 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.761+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1380044872 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f7ddc003820 con 0x56171e3ad8b0 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.761+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1380044872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7dd40b4070 con 0x56171e3ad8b0 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.761+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1380044872 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7ddc003a20 con 0x56171e3ad8b0 2024-04-25T19:00:00.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.761+0000 7f7dfc09ab80 10 client.25265 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.761+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1380044872 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56171e4302c0 con 0x56171e3ad8b0 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.762+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1380044872 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd40b4070 con 0x56171e3ad8b0 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.762+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1380044872 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f7ddc033480 con 0x56171e3ad8b0 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.762+0000 7f7dfc09ab80 10 client.25265 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.762+0000 7f7dfc09ab80 10 client.25265 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.762+0000 7f7dfc09ab80 20 client.25265 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.762+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x56171e434e60 0x56171e433b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.762+0000 7f7dfc09ab80 4 client.25265 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.762+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1380044872 --> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] -- command(tid 0: {"prefix": "status"}) -- 0x56171e3086d0 con 0x56171e434e60 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.762+0000 7f7de3e00700 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x56171e434e60 0x56171e433b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.763+0000 7f7de3e00700 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x56171e434e60 0x56171e433b70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e3178580ae480e5 server_cookie=853a5cb0604f2f13 in_seq=0 out_seq=0 2024-04-25T19:00:00.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.763+0000 7f7dd8c00700 10 client.25265 ms_handle_connect on v2:172.21.15.45:6840/2490735053 2024-04-25T19:00:00.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.763+0000 7f7dd8c00700 1 -- 172.21.15.45:0/1380044872 <== mds.1 v2:172.21.15.45:6840/2490735053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56171e3086d0 con 0x56171e434e60 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.763+0000 7f7dd8c00700 10 client.25265 handle_command_reply: tid=0 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.763+0000 7f7dfc09ab80 1 client.25265 shutdown 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1380044872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 msgr2=0x56171e3cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3cf290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc011a00 tx=0x7f7ddc011a30 comp rx=0 tx=0).stop 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1380044872 shutdown_connections 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x56171e434e60 0x56171e433b70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56171e3ad8b0 0x56171e3cf290 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56171e3b9080 0x56171e42e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dd8c00700 0 client.25265 ms_handle_reset on v2:172.21.15.45:6840/2490735053 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 1 --2- 172.21.15.45:0/1380044872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56171e3ae1d0 0x56171e3cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1380044872 >> 172.21.15.45:0/1380044872 conn(0x56171e2dcf00 msgr2=0x56171e3b7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1380044872 shutdown_connections 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 1 -- 172.21.15.45:0/1380044872 wait complete. 2024-04-25T19:00:00.768 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.764+0000 7f7dfc09ab80 20 client.25265 trim_cache size 0 max 16384 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 1, 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24512, 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 258.39954752400001, 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 95, 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 95, 2024-04-25T19:00:00.769 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 298.27370789999998 2024-04-25T19:00:00.770 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:00:00.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 -- 172.21.15.45:0/2928342609 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f7dd00a3f50 msgr2=0x7f7dd00a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:00.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f7dd00a3f50 0x7f7dd00a6410 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7de402b000 tx=0x7f7de402c000 comp rx=0 tx=0).stop 2024-04-25T19:00:00.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 -- 172.21.15.45:0/2928342609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dec1044a0 msgr2=0x7f7dec19b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:00.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dec1044a0 0x7f7dec19b7c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400e9c0 tx=0x7f7dd400ecd0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 -- 172.21.15.45:0/2928342609 shutdown_connections 2024-04-25T19:00:00.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f7dd00a3f50 0x7f7dd00a6410 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7dec103b80 0x7f7dec19b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7dec10cfb0 0x7f7dec19fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 --2- 172.21.15.45:0/2928342609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7dec1044a0 0x7f7dec19b7c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:00.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 -- 172.21.15.45:0/2928342609 >> 172.21.15.45:0/2928342609 conn(0x7f7dec0fd3d0 msgr2=0x7f7dec0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:00.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 -- 172.21.15.45:0/2928342609 shutdown_connections 2024-04-25T19:00:00.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:00.765+0000 7f7debe00700 1 -- 172.21.15.45:0/2928342609 wait complete. 2024-04-25T19:00:01.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:00 smithi079 ceph-mon[33002]: overall HEALTH_OK 2024-04-25T19:00:01.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:00 smithi045 ceph-mon[27776]: overall HEALTH_OK 2024-04-25T19:00:01.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:00 smithi062 ceph-mon[32844]: overall HEALTH_OK 2024-04-25T19:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.805+0000 7fd59ac00640 1 -- 172.21.15.45:0/3282774477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd594100b30 msgr2=0x7fd594100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.805+0000 7fd59ac00640 1 --2- 172.21.15.45:0/3282774477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd594100b30 0x7fd594100f10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009900 tx=0x7fd58c02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.805+0000 7fd59ac00640 1 -- 172.21.15.45:0/3282774477 shutdown_connections 2024-04-25T19:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.805+0000 7fd59ac00640 1 --2- 172.21.15.45:0/3282774477 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd59410e6e0 0x7fd594110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.805+0000 7fd59ac00640 1 --2- 172.21.15.45:0/3282774477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5941014e0 0x7fd59410e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.807 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.805+0000 7fd59ac00640 1 --2- 172.21.15.45:0/3282774477 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd594100b30 0x7fd594100f10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.805+0000 7fd59ac00640 1 -- 172.21.15.45:0/3282774477 >> 172.21.15.45:0/3282774477 conn(0x7fd5940fc730 msgr2=0x7fd5940feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.805+0000 7fd59ac00640 1 -- 172.21.15.45:0/3282774477 shutdown_connections 2024-04-25T19:00:01.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.805+0000 7fd59ac00640 1 -- 172.21.15.45:0/3282774477 wait complete. 2024-04-25T19:00:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.806+0000 7fd59ac00640 1 Processor -- start 2024-04-25T19:00:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.806+0000 7fd59ac00640 1 -- start start 2024-04-25T19:00:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd59ac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd594100b30 0x7fd5941a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd59ac00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5941014e0 0x7fd5941a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd59ac00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd59410e6e0 0x7fd59419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd59ac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd594114270 con 0x7fd594100b30 2024-04-25T19:00:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd59ac00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941140f0 con 0x7fd5941014e0 2024-04-25T19:00:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd59ac00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941143f0 con 0x7fd59410e6e0 2024-04-25T19:00:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd599800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd594100b30 0x7fd5941a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd598e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5941014e0 0x7fd5941a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd599800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd594100b30 0x7fd5941a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46476/0 (socket says 172.21.15.45:46476) 2024-04-25T19:00:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd599800640 1 -- 172.21.15.45:0/3932393615 learned_addr learned my addr 172.21.15.45:0/3932393615 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:01.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.807+0000 7fd59a200640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd59410e6e0 0x7fd59419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd598e00640 1 -- 172.21.15.45:0/3932393615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd59410e6e0 msgr2=0x7fd59419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd598e00640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd59410e6e0 0x7fd59419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd598e00640 1 -- 172.21.15.45:0/3932393615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd594100b30 msgr2=0x7fd5941a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd598e00640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd594100b30 0x7fd5941a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd598e00640 1 -- 172.21.15.45:0/3932393615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59419ad30 con 0x7fd5941014e0 2024-04-25T19:00:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd599800640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd594100b30 0x7fd5941a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd598e00640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5941014e0 0x7fd5941a0940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e9f0 tx=0x7fd58000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:01.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd592000640 1 -- 172.21.15.45:0/3932393615 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58000cd80 con 0x7fd5941014e0 2024-04-25T19:00:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd592000640 1 -- 172.21.15.45:0/3932393615 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5800035f0 con 0x7fd5941014e0 2024-04-25T19:00:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd59ac00640 1 -- 172.21.15.45:0/3932393615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59419b020 con 0x7fd5941014e0 2024-04-25T19:00:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.808+0000 7fd592000640 1 -- 172.21.15.45:0/3932393615 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580010670 con 0x7fd5941014e0 2024-04-25T19:00:01.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.809+0000 7fd59ac00640 1 -- 172.21.15.45:0/3932393615 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5941a72c0 con 0x7fd5941014e0 2024-04-25T19:00:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.812+0000 7fd59eff0740 1 Processor -- start 2024-04-25T19:00:01.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.812+0000 7fd59eff0740 1 -- start start 2024-04-25T19:00:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.812+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 0x55ccc4359d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.813+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4364650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.813+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc4364b90 0x55ccc4366f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.813+0000 7fd59eff0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ccc42bf900 con 0x55ccc4364b90 2024-04-25T19:00:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.813+0000 7fd59eff0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ccc4265ef0 con 0x55ccc43599b0 2024-04-25T19:00:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.813+0000 7fd59eff0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ccc42bb8c0 con 0x55ccc435a2d0 2024-04-25T19:00:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.813+0000 7fd59a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 0x55ccc4359d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.813+0000 7fd599800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4364650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.813+0000 7fd59a200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 0x55ccc4359d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:34300/0 (socket says 172.21.15.45:34300) 2024-04-25T19:00:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.813+0000 7fd59a200640 1 -- 172.21.15.45:0/455871936 learned_addr learned my addr 172.21.15.45:0/455871936 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.813+0000 7fd599800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4364650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56138/0 (socket says 172.21.15.45:56138) 2024-04-25T19:00:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.814+0000 7fd599800640 1 -- 172.21.15.45:0/455871936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 msgr2=0x55ccc4359d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:01.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.814+0000 7fd599800640 1 --2- 172.21.15.45:0/455871936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 0x55ccc4359d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.814+0000 7fd599800640 1 -- 172.21.15.45:0/455871936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc4364b90 msgr2=0x55ccc4366f80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.814+0000 7fd599800640 1 --2- 172.21.15.45:0/455871936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc4364b90 0x55ccc4366f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.814+0000 7fd599800640 1 -- 172.21.15.45:0/455871936 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc3fd2210 con 0x55ccc435a2d0 2024-04-25T19:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.814+0000 7fd592000640 1 -- 172.21.15.45:0/3932393615 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7fd5800037a0 con 0x7fd5941014e0 2024-04-25T19:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.814+0000 7fd599800640 1 --2- 172.21.15.45:0/455871936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4364650 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd58c000c00 tx=0x7fd58c0044b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b7d60d91ad0bd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.814+0000 7fd586a00640 1 -- 172.21.15.45:0/455871936 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c040070 con 0x55ccc435a2d0 2024-04-25T19:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.815+0000 7fd586a00640 1 -- 172.21.15.45:0/455871936 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd58c0047a0 con 0x55ccc435a2d0 2024-04-25T19:00:01.817 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.815+0000 7fd586a00640 1 -- 172.21.15.45:0/455871936 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c035040 con 0x55ccc435a2d0 2024-04-25T19:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.815+0000 7fd59eff0740 1 -- 172.21.15.45:0/455871936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 msgr2=0x55ccc4364650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.815+0000 7fd59eff0740 1 --2- 172.21.15.45:0/455871936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4364650 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd58c000c00 tx=0x7fd58c0044b0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.816+0000 7fd59eff0740 1 -- 172.21.15.45:0/455871936 shutdown_connections 2024-04-25T19:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.816+0000 7fd59eff0740 1 --2- 172.21.15.45:0/455871936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc4364b90 0x55ccc4366f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.816+0000 7fd59eff0740 1 --2- 172.21.15.45:0/455871936 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4364650 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.816+0000 7fd59eff0740 1 --2- 172.21.15.45:0/455871936 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 0x55ccc4359d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.816+0000 7fd59eff0740 1 -- 172.21.15.45:0/455871936 >> 172.21.15.45:0/455871936 conn(0x55ccc4355ec0 msgr2=0x55ccc43562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:01.818 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.816+0000 7fd592000640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd57c0a3f30 0x7fd57c0a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.817+0000 7fd592000640 1 -- 172.21.15.45:0/3932393615 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fd5800d0c50 con 0x7fd5941014e0 2024-04-25T19:00:01.819 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.817+0000 7fd599800640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd57c0a3f30 0x7fd57c0a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.817+0000 7fd599800640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd57c0a3f30 0x7fd57c0a63f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0096c0 tx=0x7fd58c02a3f0 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.818+0000 7fd59eff0740 1 -- 172.21.15.45:0/455871936 shutdown_connections 2024-04-25T19:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.818+0000 7fd59eff0740 1 -- 172.21.15.45:0/455871936 wait complete. 2024-04-25T19:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.818+0000 7fd59eff0740 1 -- start start 2024-04-25T19:00:01.820 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.818+0000 7fd59eff0740 1 Processor -- start 2024-04-25T19:00:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc43599b0 0x55ccc4361ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4362010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 0x55ccc4362550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd59eff0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ccc4367760 con 0x55ccc43599b0 2024-04-25T19:00:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd59eff0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ccc4279310 con 0x55ccc4364b90 2024-04-25T19:00:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd59eff0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ccc43678e0 con 0x55ccc435a2d0 2024-04-25T19:00:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd59eff0740 10 client.0 _ll_register_callbacks cb 0x55ccc4247cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd599800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc43599b0 0x55ccc4361ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd598e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4362010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd599800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc43599b0 0x55ccc4361ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46492/0 (socket says 172.21.15.45:46492) 2024-04-25T19:00:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd598e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4362010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56140/0 (socket says 172.21.15.45:56140) 2024-04-25T19:00:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd599800640 1 -- 172.21.15.45:0/2798014346 learned_addr learned my addr 172.21.15.45:0/2798014346 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.819+0000 7fd59a200640 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 0x55ccc4362550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd598e00640 1 -- 172.21.15.45:0/2798014346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 msgr2=0x55ccc4362550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd598e00640 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 0x55ccc4362550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd598e00640 1 -- 172.21.15.45:0/2798014346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc43599b0 msgr2=0x55ccc4361ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd598e00640 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc43599b0 0x55ccc4361ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd598e00640 1 -- 172.21.15.45:0/2798014346 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc409e620 con 0x55ccc435a2d0 2024-04-25T19:00:01.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd59a200640 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 0x55ccc4362550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd598e00640 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4362010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd5800ce160 tx=0x7fd58009aec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee054339feefe30d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd57b400640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:00:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd57b400640 1 -- 172.21.15.45:0/2798014346 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5800a0d60 con 0x55ccc435a2d0 2024-04-25T19:00:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd57b400640 1 -- 172.21.15.45:0/2798014346 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fd580099020 con 0x55ccc435a2d0 2024-04-25T19:00:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.820+0000 7fd59eff0740 1 -- 172.21.15.45:0/2798014346 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd58c0047a0 con 0x55ccc435a2d0 2024-04-25T19:00:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd57b400640 1 -- 172.21.15.45:0/2798014346 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd5800a5620 con 0x55ccc435a2d0 2024-04-25T19:00:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd59eff0740 10 client.25285 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:01.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd59eff0740 1 -- 172.21.15.45:0/2798014346 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ccc4373970 con 0x55ccc435a2d0 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd57b400640 1 -- 172.21.15.45:0/2798014346 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd58c0047a0 con 0x55ccc435a2d0 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd57b400640 1 -- 172.21.15.45:0/2798014346 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd58009dce0 con 0x55ccc435a2d0 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd59eff0740 10 client.25285 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd59eff0740 10 client.25285 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd59eff0740 20 client.25285 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd59eff0740 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ccc4374e30 0x55ccc4377220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd59eff0740 4 client.25285 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.821+0000 7fd59eff0740 1 -- 172.21.15.45:0/2798014346 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccc42d38b0 con 0x55ccc4374e30 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.822+0000 7fd599800640 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ccc4374e30 0x55ccc4377220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.822+0000 7fd599800640 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ccc4374e30 0x55ccc4377220 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9d76b5c546e358 server_cookie=48901963b6a6924a in_seq=0 out_seq=0 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.822+0000 7fd57b400640 10 client.25285 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.823+0000 7fd57b400640 1 -- 172.21.15.45:0/2798014346 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ccc42d38b0 con 0x55ccc4374e30 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.823+0000 7fd57b400640 10 client.25285 handle_command_reply: tid=0 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.823+0000 7fd59eff0740 1 client.25285 shutdown 2024-04-25T19:00:01.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 1 -- 172.21.15.45:0/2798014346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 msgr2=0x55ccc4362010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4362010 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd5800ce160 tx=0x7fd58009aec0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 1 -- 172.21.15.45:0/2798014346 shutdown_connections 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ccc4374e30 0x55ccc4377220 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 0x55ccc4362550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4362010 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 1 --2- 172.21.15.45:0/2798014346 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc43599b0 0x55ccc4361ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd57b400640 0 client.25285 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 1 -- 172.21.15.45:0/2798014346 >> 172.21.15.45:0/2798014346 conn(0x55ccc4355ec0 msgr2=0x55ccc435bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 1 -- 172.21.15.45:0/2798014346 shutdown_connections 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 1 -- 172.21.15.45:0/2798014346 wait complete. 2024-04-25T19:00:01.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.824+0000 7fd59eff0740 20 client.25285 trim_cache size 0 max 16384 2024-04-25T19:00:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.842+0000 7fd59eff0740 1 Processor -- start 2024-04-25T19:00:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd59eff0740 1 -- start start 2024-04-25T19:00:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 0x55ccc4370620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4363a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc4364b90 0x55ccc437cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.845 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd59eff0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941b24e0 con 0x55ccc4364b90 2024-04-25T19:00:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd59eff0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941afdb0 con 0x55ccc43599b0 2024-04-25T19:00:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd59eff0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941adc00 con 0x55ccc435a2d0 2024-04-25T19:00:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd598e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4363a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd598e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4363a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:56144/0 (socket says 172.21.15.45:56144) 2024-04-25T19:00:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd598e00640 1 -- 172.21.15.45:0/2322665377 learned_addr learned my addr 172.21.15.45:0/2322665377 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd59a200640 1 --2- 172.21.15.45:0/2322665377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc4364b90 0x55ccc437cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.843+0000 7fd599800640 1 --2- 172.21.15.45:0/2322665377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 0x55ccc4370620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.844+0000 7fd598e00640 1 -- 172.21.15.45:0/2322665377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 msgr2=0x55ccc4370620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.844+0000 7fd598e00640 1 --2- 172.21.15.45:0/2322665377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 0x55ccc4370620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.844+0000 7fd598e00640 1 -- 172.21.15.45:0/2322665377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc4364b90 msgr2=0x55ccc437cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.844+0000 7fd598e00640 1 --2- 172.21.15.45:0/2322665377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc4364b90 0x55ccc437cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.844+0000 7fd598e00640 1 -- 172.21.15.45:0/2322665377 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc4377850 con 0x55ccc435a2d0 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.844+0000 7fd598e00640 1 --2- 172.21.15.45:0/2322665377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4363a10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd580099380 tx=0x7fd580005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8fb82610c6e0d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.844+0000 7fd586000640 1 -- 172.21.15.45:0/2322665377 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5800a0d60 con 0x55ccc435a2d0 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.844+0000 7fd586000640 1 -- 172.21.15.45:0/2322665377 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5800b8030 con 0x55ccc435a2d0 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.844+0000 7fd586000640 1 -- 172.21.15.45:0/2322665377 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580009430 con 0x55ccc435a2d0 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.845+0000 7fd59eff0740 1 -- 172.21.15.45:0/2322665377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 msgr2=0x55ccc4363a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.845+0000 7fd59eff0740 1 --2- 172.21.15.45:0/2322665377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4363a10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd580099380 tx=0x7fd580005bb0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.845+0000 7fd59eff0740 1 -- 172.21.15.45:0/2322665377 shutdown_connections 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.845+0000 7fd59eff0740 1 --2- 172.21.15.45:0/2322665377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc4364b90 0x55ccc437cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.845+0000 7fd59eff0740 1 --2- 172.21.15.45:0/2322665377 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc435a2d0 0x55ccc4363a10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.845+0000 7fd59eff0740 1 --2- 172.21.15.45:0/2322665377 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc43599b0 0x55ccc4370620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.845+0000 7fd59eff0740 1 -- 172.21.15.45:0/2322665377 >> 172.21.15.45:0/2322665377 conn(0x55ccc4355ec0 msgr2=0x55ccc4363300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.846+0000 7fd59eff0740 1 -- 172.21.15.45:0/2322665377 shutdown_connections 2024-04-25T19:00:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.846+0000 7fd59eff0740 1 -- 172.21.15.45:0/2322665377 wait complete. 2024-04-25T19:00:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.846+0000 7fd59eff0740 1 -- start start 2024-04-25T19:00:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.846+0000 7fd59eff0740 1 Processor -- start 2024-04-25T19:00:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc43599b0 0x55ccc436cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc435a2d0 0x55ccc436a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd59eff0740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 0x55ccc436a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd59eff0740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941ae270 con 0x55ccc435a2d0 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd59eff0740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941af6e0 con 0x55ccc4364b90 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd59eff0740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd5941ae940 con 0x55ccc43599b0 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd59eff0740 10 client.0 _ll_register_callbacks cb 0x7fd58009c5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd598e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc435a2d0 0x55ccc436a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd598e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc435a2d0 0x55ccc436a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46518/0 (socket says 172.21.15.45:46518) 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd598e00640 1 -- 172.21.15.45:0/537077219 learned_addr learned my addr 172.21.15.45:0/537077219 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd59a200640 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 0x55ccc436a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd599800640 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc43599b0 0x55ccc436cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd598e00640 1 -- 172.21.15.45:0/537077219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc43599b0 msgr2=0x55ccc436cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd598e00640 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc43599b0 0x55ccc436cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd598e00640 1 -- 172.21.15.45:0/537077219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 msgr2=0x55ccc436a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd598e00640 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 0x55ccc436a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.847+0000 7fd598e00640 1 -- 172.21.15.45:0/537077219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc4366f80 con 0x55ccc435a2d0 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.848+0000 7fd599800640 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc43599b0 0x55ccc436cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.848+0000 7fd598e00640 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc435a2d0 0x55ccc436a2a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd580000c00 tx=0x7fd580005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63147fa821445ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.848+0000 7fd57b400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.848+0000 7fd57b400640 1 -- 172.21.15.45:0/537077219 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5800a0d60 con 0x55ccc435a2d0 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.848+0000 7fd57b400640 1 -- 172.21.15.45:0/537077219 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fd5800870b0 con 0x55ccc435a2d0 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.848+0000 7fd59eff0740 1 -- 172.21.15.45:0/537077219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5800b8030 con 0x55ccc435a2d0 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.848+0000 7fd57b400640 1 -- 172.21.15.45:0/537077219 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd5800872f0 con 0x55ccc435a2d0 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.848+0000 7fd59eff0740 10 client.15162 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.848+0000 7fd59eff0740 1 -- 172.21.15.45:0/537077219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ccc43db1d0 con 0x55ccc435a2d0 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.849+0000 7fd57b400640 1 -- 172.21.15.45:0/537077219 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5800b8030 con 0x55ccc435a2d0 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.849+0000 7fd57b400640 1 -- 172.21.15.45:0/537077219 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fd58009dce0 con 0x55ccc435a2d0 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.849+0000 7fd59eff0740 10 client.15162 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.849+0000 7fd59eff0740 10 client.15162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:00:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.849+0000 7fd59eff0740 20 client.15162 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.849+0000 7fd59eff0740 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ccc43db5e0 0x55ccc43dda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.849+0000 7fd59eff0740 4 client.15162 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:00:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.849+0000 7fd59eff0740 1 -- 172.21.15.45:0/537077219 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccc42cd320 con 0x55ccc43db5e0 2024-04-25T19:00:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.849+0000 7fd599800640 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ccc43db5e0 0x55ccc43dda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.850+0000 7fd599800640 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ccc43db5e0 0x55ccc43dda00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a048b5b98a22200 server_cookie=41986c14f55f054e in_seq=0 out_seq=0 2024-04-25T19:00:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.850+0000 7fd57b400640 10 client.15162 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:01.858 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T19:00:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:00:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:00:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:00:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:00:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd57b400640 1 -- 172.21.15.45:0/537077219 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ccc42cd320 con 0x55ccc43db5e0 2024-04-25T19:00:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd57b400640 10 client.15162 handle_command_reply: tid=0 2024-04-25T19:00:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 client.15162 shutdown 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 -- 172.21.15.45:0/537077219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc435a2d0 msgr2=0x55ccc436a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc435a2d0 0x55ccc436a2a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd580000c00 tx=0x7fd580005ab0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 -- 172.21.15.45:0/537077219 shutdown_connections 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ccc43db5e0 0x55ccc43dda00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ccc4364b90 0x55ccc436a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd57b400640 0 client.15162 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ccc435a2d0 0x55ccc436a2a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 --2- 172.21.15.45:0/537077219 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ccc43599b0 0x55ccc436cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 -- 172.21.15.45:0/537077219 >> 172.21.15.45:0/537077219 conn(0x55ccc4355ec0 msgr2=0x55ccc4355020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 -- 172.21.15.45:0/537077219 shutdown_connections 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.859+0000 7fd59eff0740 1 -- 172.21.15.45:0/537077219 wait complete. 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59eff0740 20 client.15162 trim_cache size 0 max 16384 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59ac00640 1 -- 172.21.15.45:0/3932393615 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd57c0a3f30 msgr2=0x7fd57c0a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59ac00640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd57c0a3f30 0x7fd57c0a63f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0096c0 tx=0x7fd58c02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59ac00640 1 -- 172.21.15.45:0/3932393615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5941014e0 msgr2=0x7fd5941a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59ac00640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5941014e0 0x7fd5941a0940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd58000e9f0 tx=0x7fd58000eeb0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59ac00640 1 -- 172.21.15.45:0/3932393615 shutdown_connections 2024-04-25T19:00:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59ac00640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd57c0a3f30 0x7fd57c0a63f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59ac00640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd59410e6e0 0x7fd59419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59ac00640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd5941014e0 0x7fd5941a0940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59ac00640 1 --2- 172.21.15.45:0/3932393615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd594100b30 0x7fd5941a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.860+0000 7fd59ac00640 1 -- 172.21.15.45:0/3932393615 >> 172.21.15.45:0/3932393615 conn(0x7fd5940fc730 msgr2=0x7fd5940feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.861+0000 7fd59ac00640 1 -- 172.21.15.45:0/3932393615 shutdown_connections 2024-04-25T19:00:01.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:01.861+0000 7fd59ac00640 1 -- 172.21.15.45:0/3932393615 wait complete. 2024-04-25T19:00:02.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:01 smithi079 ceph-mon[33002]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T19:00:02.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:01 smithi079 ceph-mon[33002]: mgrmap e26: x(active, since 42s), standbys: z, y 2024-04-25T19:00:02.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:01 smithi045 ceph-mon[27776]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T19:00:02.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:01 smithi045 ceph-mon[27776]: mgrmap e26: x(active, since 42s), standbys: z, y 2024-04-25T19:00:02.321 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:02.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:01 smithi062 ceph-mon[32844]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T19:00:02.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:01 smithi062 ceph-mon[32844]: mgrmap e26: x(active, since 42s), standbys: z, y 2024-04-25T19:00:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:03 smithi079 ceph-mon[33002]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T19:00:04.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:03 smithi045 ceph-mon[27776]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T19:00:04.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:03 smithi062 ceph-mon[32844]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T19:00:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.797+0000 7f3536800700 1 -- 172.21.15.45:0/1839632615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353006a750 msgr2=0x7f353006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.797+0000 7f3536800700 1 --2- 172.21.15.45:0/1839632615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353006a750 0x7f353006abb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f351c009990 tx=0x7f351c009ca0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.797+0000 7f3536800700 1 -- 172.21.15.45:0/1839632615 shutdown_connections 2024-04-25T19:00:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.797+0000 7f3536800700 1 --2- 172.21.15.45:0/1839632615 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3530108a30 0x7f3530108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.797+0000 7f3536800700 1 --2- 172.21.15.45:0/1839632615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f353006b0f0 0x7f353010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.797+0000 7f3536800700 1 --2- 172.21.15.45:0/1839632615 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f353006a750 0x7f353006abb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.797+0000 7f3536800700 1 -- 172.21.15.45:0/1839632615 >> 172.21.15.45:0/1839632615 conn(0x7f35300788f0 msgr2=0x7f3530078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:04.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.797+0000 7f3536800700 1 -- 172.21.15.45:0/1839632615 shutdown_connections 2024-04-25T19:00:04.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.797+0000 7f3536800700 1 -- 172.21.15.45:0/1839632615 wait complete. 2024-04-25T19:00:04.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.800+0000 7f3536800700 1 Processor -- start 2024-04-25T19:00:04.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.800+0000 7f3536800700 1 -- start start 2024-04-25T19:00:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f3536800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f353006a750 0x7f3530199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f3536800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f353006b0f0 0x7f35301996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f3536800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3530108a30 0x7f353019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f3536800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3530112c60 con 0x7f3530108a30 2024-04-25T19:00:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f3536800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3530112ae0 con 0x7f353006a750 2024-04-25T19:00:04.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f3536800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3530112de0 con 0x7f353006b0f0 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f352f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f353006a750 0x7f3530199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f352ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f353006b0f0 0x7f35301996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f352ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f353006b0f0 0x7f35301996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:34680/0 (socket says 172.21.15.45:34680) 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f352ea00700 1 -- 172.21.15.45:0/475696177 learned_addr learned my addr 172.21.15.45:0/475696177 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.801+0000 7f352fe00700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3530108a30 0x7f353019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.802+0000 7f352ea00700 1 -- 172.21.15.45:0/475696177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f353006a750 msgr2=0x7f3530199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.802+0000 7f352ea00700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f353006a750 0x7f3530199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.802+0000 7f352ea00700 1 -- 172.21.15.45:0/475696177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3530108a30 msgr2=0x7f353019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.802+0000 7f352ea00700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3530108a30 0x7f353019da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.802+0000 7f352ea00700 1 -- 172.21.15.45:0/475696177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353019e1a0 con 0x7f353006b0f0 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.802+0000 7f352f400700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f353006a750 0x7f3530199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:04.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.802+0000 7f352ea00700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f353006b0f0 0x7f35301996d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f351c000c00 tx=0x7f351c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.803+0000 7f3527e00700 1 -- 172.21.15.45:0/475696177 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c027070 con 0x7f353006b0f0 2024-04-25T19:00:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.803+0000 7f3527e00700 1 -- 172.21.15.45:0/475696177 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f351c022470 con 0x7f353006b0f0 2024-04-25T19:00:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.803+0000 7f3536800700 1 -- 172.21.15.45:0/475696177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353019e430 con 0x7f353006b0f0 2024-04-25T19:00:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.803+0000 7f3527e00700 1 -- 172.21.15.45:0/475696177 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c017650 con 0x7f353006b0f0 2024-04-25T19:00:04.806 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.803+0000 7f3536800700 1 -- 172.21.15.45:0/475696177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35301a5d10 con 0x7f353006b0f0 2024-04-25T19:00:04.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.807+0000 7f3527e00700 1 -- 172.21.15.45:0/475696177 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f351c02a070 con 0x7f353006b0f0 2024-04-25T19:00:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.809+0000 7f3527e00700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f35140a3fe0 0x7f35140a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.809+0000 7f3527e00700 1 -- 172.21.15.45:0/475696177 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f351c0d16b0 con 0x7f353006b0f0 2024-04-25T19:00:04.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.810+0000 7f352f400700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f35140a3fe0 0x7f35140a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.810+0000 7f3541230b80 1 Processor -- start 2024-04-25T19:00:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f352f400700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f35140a3fe0 0x7f35140a64a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3520009680 tx=0x7f3520009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:04.812 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f3541230b80 1 -- start start 2024-04-25T19:00:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f3541230b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4da00b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f3541230b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da010a0 0x55bf4d9fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f3541230b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4d9fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f3541230b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bf4d94ce40 con 0x55bf4da010a0 2024-04-25T19:00:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f3541230b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bf4d8f36e0 con 0x55bf4da00780 2024-04-25T19:00:04.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f3541230b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bf4d948e00 con 0x55bf4d9faef0 2024-04-25T19:00:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f352f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4da00b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f352fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4d9fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f352f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4da00b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41486/0 (socket says 172.21.15.45:41486) 2024-04-25T19:00:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f352f400700 1 -- 172.21.15.45:0/511533694 learned_addr learned my addr 172.21.15.45:0/511533694 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:04.814 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.811+0000 7f352ea00700 1 --2- 172.21.15.45:0/511533694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da010a0 0x55bf4d9fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.812+0000 7f352fe00700 1 -- 172.21.15.45:0/511533694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 msgr2=0x55bf4da00b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.812+0000 7f352fe00700 1 --2- 172.21.15.45:0/511533694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4da00b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.812+0000 7f352fe00700 1 -- 172.21.15.45:0/511533694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da010a0 msgr2=0x55bf4d9fa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.812+0000 7f352fe00700 1 --2- 172.21.15.45:0/511533694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da010a0 0x55bf4d9fa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.812+0000 7f352fe00700 1 -- 172.21.15.45:0/511533694 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf4d8e9630 con 0x55bf4d9faef0 2024-04-25T19:00:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.812+0000 7f352f400700 1 --2- 172.21.15.45:0/511533694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4da00b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.812+0000 7f352fe00700 1 --2- 172.21.15.45:0/511533694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4d9fb330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3528011580 tx=0x7f3528009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a995b8a8a2c85559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.812+0000 7f3524c00700 1 -- 172.21.15.45:0/511533694 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528011e80 con 0x55bf4d9faef0 2024-04-25T19:00:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.812+0000 7f3524c00700 1 -- 172.21.15.45:0/511533694 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3528004bb0 con 0x55bf4d9faef0 2024-04-25T19:00:04.815 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.812+0000 7f3524c00700 1 -- 172.21.15.45:0/511533694 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3528004720 con 0x55bf4d9faef0 2024-04-25T19:00:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.813+0000 7f3541230b80 1 -- 172.21.15.45:0/511533694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 msgr2=0x55bf4d9fb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.813+0000 7f3541230b80 1 --2- 172.21.15.45:0/511533694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4d9fb330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3528011580 tx=0x7f3528009920 comp rx=0 tx=0).stop 2024-04-25T19:00:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.814+0000 7f3541230b80 1 -- 172.21.15.45:0/511533694 shutdown_connections 2024-04-25T19:00:04.816 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.814+0000 7f3541230b80 1 --2- 172.21.15.45:0/511533694 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4da00b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.814+0000 7f3541230b80 1 --2- 172.21.15.45:0/511533694 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4d9fb330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.814+0000 7f3541230b80 1 --2- 172.21.15.45:0/511533694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da010a0 0x55bf4d9fa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.814+0000 7f3541230b80 1 -- 172.21.15.45:0/511533694 >> 172.21.15.45:0/511533694 conn(0x55bf4d928f00 msgr2=0x55bf4d9f58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.814+0000 7f3541230b80 1 -- 172.21.15.45:0/511533694 shutdown_connections 2024-04-25T19:00:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.814+0000 7f3541230b80 1 -- 172.21.15.45:0/511533694 wait complete. 2024-04-25T19:00:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.814+0000 7f3541230b80 1 -- start start 2024-04-25T19:00:04.817 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.815+0000 7f3541230b80 1 Processor -- start 2024-04-25T19:00:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f3541230b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4d9faef0 0x55bf4da06c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f3541230b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da00780 0x55bf4da03f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f3541230b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da04930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f3541230b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bf4da07920 con 0x55bf4da00780 2024-04-25T19:00:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f3541230b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bf4d906810 con 0x55bf4d9faef0 2024-04-25T19:00:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f3541230b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bf4da07aa0 con 0x55bf4da044b0 2024-04-25T19:00:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f3541230b80 10 client.0 _ll_register_callbacks cb 0x55bf4d8e56d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f352f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4d9faef0 0x55bf4da06c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f352f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4d9faef0 0x55bf4da06c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41502/0 (socket says 172.21.15.45:41502) 2024-04-25T19:00:04.818 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f352f400700 1 -- 172.21.15.45:0/1510600940 learned_addr learned my addr 172.21.15.45:0/1510600940 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f352fe00700 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da04930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.816+0000 7f352ea00700 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da00780 0x55bf4da03f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.819 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.817+0000 7f352fe00700 1 -- 172.21.15.45:0/1510600940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4d9faef0 msgr2=0x55bf4da06c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.817+0000 7f352fe00700 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4d9faef0 0x55bf4da06c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.817+0000 7f352fe00700 1 -- 172.21.15.45:0/1510600940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da00780 msgr2=0x55bf4da03f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.817+0000 7f352fe00700 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da00780 0x55bf4da03f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.817+0000 7f352fe00700 1 -- 172.21.15.45:0/1510600940 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf4d89a880 con 0x55bf4da044b0 2024-04-25T19:00:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.817+0000 7f352fe00700 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da04930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3528004300 tx=0x7f35280049c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3680103d498c07f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.817+0000 7f3519600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:00:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.817+0000 7f3519600700 1 -- 172.21.15.45:0/1510600940 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352802b060 con 0x55bf4da044b0 2024-04-25T19:00:04.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.818+0000 7f3519600700 1 -- 172.21.15.45:0/1510600940 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f3528003c00 con 0x55bf4da044b0 2024-04-25T19:00:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.818+0000 7f3541230b80 1 -- 172.21.15.45:0/1510600940 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3528004bb0 con 0x55bf4da044b0 2024-04-25T19:00:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.818+0000 7f3519600700 1 -- 172.21.15.45:0/1510600940 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3528011e80 con 0x55bf4da044b0 2024-04-25T19:00:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.818+0000 7f3541230b80 10 client.25303 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.818+0000 7f3541230b80 1 -- 172.21.15.45:0/1510600940 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf4d8e3aa0 con 0x55bf4da044b0 2024-04-25T19:00:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.818+0000 7f3519600700 1 -- 172.21.15.45:0/1510600940 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3528004bb0 con 0x55bf4da044b0 2024-04-25T19:00:04.821 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.819+0000 7f3519600700 1 -- 172.21.15.45:0/1510600940 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3528025030 con 0x55bf4da044b0 2024-04-25T19:00:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.819+0000 7f3541230b80 10 client.25303 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.819+0000 7f3541230b80 10 client.25303 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T19:00:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.819+0000 7f3541230b80 20 client.25303 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.819+0000 7f3541230b80 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bf4da18b90 0x55bf4da1a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.819+0000 7f3541230b80 4 client.25303 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:00:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.819+0000 7f3541230b80 1 -- 172.21.15.45:0/1510600940 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf4d95a630 con 0x55bf4da18b90 2024-04-25T19:00:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.819+0000 7f352f400700 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bf4da18b90 0x55bf4da1a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.820+0000 7f352f400700 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bf4da18b90 0x55bf4da1a000 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ab95cc794104bd3 server_cookie=6878ae9ef43f8a5e in_seq=0 out_seq=0 2024-04-25T19:00:04.822 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.820+0000 7f3519600700 10 client.25303 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.821+0000 7f3519600700 1 -- 172.21.15.45:0/1510600940 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf4d95a630 con 0x55bf4da18b90 2024-04-25T19:00:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.821+0000 7f3519600700 10 client.25303 handle_command_reply: tid=0 2024-04-25T19:00:04.823 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.821+0000 7f3541230b80 1 client.25303 shutdown 2024-04-25T19:00:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.821+0000 7f3541230b80 1 -- 172.21.15.45:0/1510600940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 msgr2=0x55bf4da04930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.821+0000 7f3541230b80 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da04930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3528004300 tx=0x7f35280049c0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.821+0000 7f3541230b80 1 -- 172.21.15.45:0/1510600940 shutdown_connections 2024-04-25T19:00:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.821+0000 7f3541230b80 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bf4da18b90 0x55bf4da1a000 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.821+0000 7f3519600700 0 client.25303 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.821+0000 7f3541230b80 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4d9faef0 0x55bf4da06c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.822+0000 7f3541230b80 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da04930 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.822+0000 7f3541230b80 1 --2- 172.21.15.45:0/1510600940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da00780 0x55bf4da03f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.824 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.822+0000 7f3541230b80 1 -- 172.21.15.45:0/1510600940 >> 172.21.15.45:0/1510600940 conn(0x55bf4d928f00 msgr2=0x55bf4d9f4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.822+0000 7f3541230b80 1 -- 172.21.15.45:0/1510600940 shutdown_connections 2024-04-25T19:00:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.822+0000 7f3541230b80 1 -- 172.21.15.45:0/1510600940 wait complete. 2024-04-25T19:00:04.825 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.822+0000 7f3541230b80 20 client.25303 trim_cache size 0 max 16384 2024-04-25T19:00:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.850+0000 7f3541230b80 1 Processor -- start 2024-04-25T19:00:04.852 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.850+0000 7f3541230b80 1 -- start start 2024-04-25T19:00:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f3541230b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4d9faef0 0x55bf4d9fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f3541230b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4d9f55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f3541230b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da01490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f3541230b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bf4da18340 con 0x55bf4d9faef0 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f3541230b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bf4da18a10 con 0x55bf4da00780 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f3541230b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bf4da190e0 con 0x55bf4da044b0 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f352ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4d9f55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f352fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da01490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f352ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4d9f55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41518/0 (socket says 172.21.15.45:41518) 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f352ea00700 1 -- 172.21.15.45:0/3813042719 learned_addr learned my addr 172.21.15.45:0/3813042719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.851+0000 7f352fe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da01490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:34712/0 (socket says 172.21.15.45:34712) 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.852+0000 7f352f400700 1 --2- 172.21.15.45:0/3813042719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4d9faef0 0x55bf4d9fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.852+0000 7f352ea00700 1 -- 172.21.15.45:0/3813042719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 msgr2=0x55bf4da01490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.852+0000 7f352ea00700 1 --2- 172.21.15.45:0/3813042719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da01490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.852+0000 7f352ea00700 1 -- 172.21.15.45:0/3813042719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4d9faef0 msgr2=0x55bf4d9fa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.852+0000 7f352ea00700 1 --2- 172.21.15.45:0/3813042719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4d9faef0 0x55bf4d9fa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.855 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.852+0000 7f352ea00700 1 -- 172.21.15.45:0/3813042719 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf4da024e0 con 0x55bf4da00780 2024-04-25T19:00:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.852+0000 7f352fe00700 1 --2- 172.21.15.45:0/3813042719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da01490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.852+0000 7f352f400700 1 --2- 172.21.15.45:0/3813042719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4d9faef0 0x55bf4d9fa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:00:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.852+0000 7f352ea00700 1 --2- 172.21.15.45:0/3813042719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4d9f55d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f351c0a5b50 tx=0x7f351c005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1da8710fc48c3eb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.853+0000 7f351be00700 1 -- 172.21.15.45:0/3813042719 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c09d920 con 0x55bf4da00780 2024-04-25T19:00:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.853+0000 7f351be00700 1 -- 172.21.15.45:0/3813042719 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f351c099860 con 0x55bf4da00780 2024-04-25T19:00:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.853+0000 7f351be00700 1 -- 172.21.15.45:0/3813042719 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c09c970 con 0x55bf4da00780 2024-04-25T19:00:04.856 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.854+0000 7f3541230b80 1 -- 172.21.15.45:0/3813042719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 msgr2=0x55bf4d9f55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:04.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.854+0000 7f3541230b80 1 --2- 172.21.15.45:0/3813042719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4d9f55d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f351c0a5b50 tx=0x7f351c005040 comp rx=0 tx=0).stop 2024-04-25T19:00:04.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.854+0000 7f3541230b80 1 -- 172.21.15.45:0/3813042719 shutdown_connections 2024-04-25T19:00:04.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.854+0000 7f3541230b80 1 --2- 172.21.15.45:0/3813042719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4da044b0 0x55bf4da01490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.854+0000 7f3541230b80 1 --2- 172.21.15.45:0/3813042719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4d9faef0 0x55bf4d9fa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.854+0000 7f3541230b80 1 --2- 172.21.15.45:0/3813042719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4d9f55d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.854+0000 7f3541230b80 1 -- 172.21.15.45:0/3813042719 >> 172.21.15.45:0/3813042719 conn(0x55bf4d928f00 msgr2=0x55bf4d9fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:04.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.854+0000 7f3541230b80 1 -- 172.21.15.45:0/3813042719 shutdown_connections 2024-04-25T19:00:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.854+0000 7f3541230b80 1 -- 172.21.15.45:0/3813042719 wait complete. 2024-04-25T19:00:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.855+0000 7f3541230b80 1 -- start start 2024-04-25T19:00:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.855+0000 7f3541230b80 1 Processor -- start 2024-04-25T19:00:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.855+0000 7f3541230b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4da23930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f3541230b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4d9f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f3541230b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da044b0 0x55bf4d9f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f3541230b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bf4da19e80 con 0x55bf4da044b0 2024-04-25T19:00:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f3541230b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bf4da197b0 con 0x55bf4da00780 2024-04-25T19:00:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f3541230b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bf4da1a550 con 0x55bf4d9faef0 2024-04-25T19:00:04.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f3541230b80 10 client.0 _ll_register_callbacks cb 0x55bf4d8e56d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f352f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4da23930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f352ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4d9f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f352f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4da23930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:34722/0 (socket says 172.21.15.45:34722) 2024-04-25T19:00:04.859 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f352f400700 1 -- 172.21.15.45:0/4223251179 learned_addr learned my addr 172.21.15.45:0/4223251179 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f352fe00700 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da044b0 0x55bf4d9f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f352f400700 1 -- 172.21.15.45:0/4223251179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 msgr2=0x55bf4d9f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f352f400700 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4d9f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f352f400700 1 -- 172.21.15.45:0/4223251179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da044b0 msgr2=0x55bf4d9f4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.856+0000 7f352f400700 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da044b0 0x55bf4d9f4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.857+0000 7f352f400700 1 -- 172.21.15.45:0/4223251179 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf4d9f5580 con 0x55bf4d9faef0 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.857+0000 7f352f400700 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4da23930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f352000e890 tx=0x7f35200047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a7c79c47a6dcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.857+0000 7f3519600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.857+0000 7f3519600700 1 -- 172.21.15.45:0/4223251179 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352002b2b0 con 0x55bf4d9faef0 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.857+0000 7f3519600700 1 -- 172.21.15.45:0/4223251179 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f352002b450 con 0x55bf4d9faef0 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.857+0000 7f3541230b80 1 -- 172.21.15.45:0/4223251179 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f351c099860 con 0x55bf4d9faef0 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.858+0000 7f3519600700 1 -- 172.21.15.45:0/4223251179 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f352002b650 con 0x55bf4d9faef0 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.858+0000 7f3541230b80 10 client.25309 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:04.860 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.858+0000 7f3541230b80 1 -- 172.21.15.45:0/4223251179 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf4da7fde0 con 0x55bf4d9faef0 2024-04-25T19:00:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.858+0000 7f3519600700 1 -- 172.21.15.45:0/4223251179 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f351c099860 con 0x55bf4d9faef0 2024-04-25T19:00:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.858+0000 7f3519600700 1 -- 172.21.15.45:0/4223251179 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f3520018040 con 0x55bf4d9faef0 2024-04-25T19:00:04.861 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.858+0000 7f3541230b80 10 client.25309 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.859+0000 7f3541230b80 10 client.25309 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T19:00:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.859+0000 7f3541230b80 20 client.25309 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.859+0000 7f3541230b80 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bf4da18b90 0x55bf4da83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.859+0000 7f3541230b80 4 client.25309 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T19:00:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.859+0000 7f3541230b80 1 -- 172.21.15.45:0/4223251179 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "status"}) -- 0x55bf4d9546d0 con 0x55bf4da18b90 2024-04-25T19:00:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.859+0000 7f352ea00700 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bf4da18b90 0x55bf4da83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f352ea00700 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bf4da18b90 0x55bf4da83660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56c1d133421532fa server_cookie=58037579361e8280 in_seq=0 out_seq=0 2024-04-25T19:00:04.862 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3519600700 10 client.25309 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3519600700 1 -- 172.21.15.45:0/4223251179 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bf4d9546d0 con 0x55bf4da18b90 2024-04-25T19:00:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3519600700 10 client.25309 handle_command_reply: tid=0 2024-04-25T19:00:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3541230b80 1 client.25309 shutdown 2024-04-25T19:00:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3541230b80 1 -- 172.21.15.45:0/4223251179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 msgr2=0x55bf4da23930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:04.863 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3541230b80 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4da23930 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f352000e890 tx=0x7f35200047c0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3541230b80 1 -- 172.21.15.45:0/4223251179 shutdown_connections 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3541230b80 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bf4da18b90 0x55bf4da83660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3541230b80 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bf4da044b0 0x55bf4d9f4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3541230b80 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bf4d9faef0 0x55bf4da23930 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3541230b80 1 --2- 172.21.15.45:0/4223251179 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bf4da00780 0x55bf4d9f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3541230b80 1 -- 172.21.15.45:0/4223251179 >> 172.21.15.45:0/4223251179 conn(0x55bf4d928f00 msgr2=0x55bf4da01490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.860+0000 7f3519600700 0 client.25309 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.861+0000 7f3541230b80 1 -- 172.21.15.45:0/4223251179 shutdown_connections 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.861+0000 7f3541230b80 1 -- 172.21.15.45:0/4223251179 wait complete. 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.861+0000 7f3541230b80 20 client.25309 trim_cache size 0 max 16384 2024-04-25T19:00:04.864 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "cluster_fsid": "ea2833ba-0333-11ef-bc93-c7b262605968", 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "whoami": 0, 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "id": 24524, 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "want_state": "up:active", 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "state": "up:active", 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "fs_name": "cephfs", 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "rank_uptime": 278.62101010600003, 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "mdsmap_epoch": 26, 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch": 95, 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_epoch_barrier": 95, 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout: "uptime": 299.45023208800001 2024-04-25T19:00:04.865 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:00:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.862+0000 7f3536800700 1 -- 172.21.15.45:0/475696177 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f35140a3fe0 msgr2=0x7f35140a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.862+0000 7f3536800700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f35140a3fe0 0x7f35140a64a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3520009680 tx=0x7f3520009260 comp rx=0 tx=0).stop 2024-04-25T19:00:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.862+0000 7f3536800700 1 -- 172.21.15.45:0/475696177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f353006b0f0 msgr2=0x7f35301996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.862+0000 7f3536800700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f353006b0f0 0x7f35301996d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f351c000c00 tx=0x7f351c0057c0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.862+0000 7f3536800700 1 -- 172.21.15.45:0/475696177 shutdown_connections 2024-04-25T19:00:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.862+0000 7f3536800700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f35140a3fe0 0x7f35140a64a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.862+0000 7f3536800700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f353006a750 0x7f3530199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.862+0000 7f3536800700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3530108a30 0x7f353019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.866 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.862+0000 7f3536800700 1 --2- 172.21.15.45:0/475696177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f353006b0f0 0x7f35301996d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.862+0000 7f3536800700 1 -- 172.21.15.45:0/475696177 >> 172.21.15.45:0/475696177 conn(0x7f35300788f0 msgr2=0x7f35300ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.863+0000 7f3536800700 1 -- 172.21.15.45:0/475696177 shutdown_connections 2024-04-25T19:00:04.867 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:04.864+0000 7f3536800700 1 -- 172.21.15.45:0/475696177 wait complete. 2024-04-25T19:00:05.737 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-25T19:00:05.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:05 smithi045 ceph-mon[27776]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T19:00:06.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:05 smithi079 ceph-mon[33002]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T19:00:06.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:05 smithi062 ceph-mon[32844]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T19:00:08.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:07 smithi079 ceph-mon[33002]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T19:00:08.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:07 smithi045 ceph-mon[27776]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T19:00:08.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:07 smithi062 ceph-mon[32844]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T19:00:10.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:09 smithi079 ceph-mon[33002]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T19:00:10.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:09 smithi045 ceph-mon[27776]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T19:00:10.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:09 smithi062 ceph-mon[32844]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T19:00:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:11 smithi079 ceph-mon[33002]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T19:00:12.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:11 smithi045 ceph-mon[27776]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T19:00:12.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:11 smithi062 ceph-mon[32844]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T19:00:14.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:13 smithi079 ceph-mon[33002]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-25T19:00:14.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:13 smithi045 ceph-mon[27776]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-25T19:00:14.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:13 smithi062 ceph-mon[32844]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-25T19:00:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.023+0000 7f8449400640 1 -- 172.21.15.45:0/1599183380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f844410c380 msgr2=0x7f844410e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.023+0000 7f8449400640 1 --2- 172.21.15.45:0/1599183380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f844410c380 0x7f844410e800 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f843c009be0 tx=0x7f843c02a750 comp rx=0 tx=0).stop 2024-04-25T19:00:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.024+0000 7f8449400640 1 -- 172.21.15.45:0/1599183380 shutdown_connections 2024-04-25T19:00:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.024+0000 7f8449400640 1 --2- 172.21.15.45:0/1599183380 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f844410c380 0x7f844410e800 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.024+0000 7f8449400640 1 --2- 172.21.15.45:0/1599183380 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84441038d0 0x7f844410bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.024+0000 7f8449400640 1 --2- 172.21.15.45:0/1599183380 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8444102fb0 0x7f8444103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.024+0000 7f8449400640 1 -- 172.21.15.45:0/1599183380 >> 172.21.15.45:0/1599183380 conn(0x7f84440fc750 msgr2=0x7f84440feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.024+0000 7f8449400640 1 -- 172.21.15.45:0/1599183380 shutdown_connections 2024-04-25T19:00:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.024+0000 7f8449400640 1 -- 172.21.15.45:0/1599183380 wait complete. 2024-04-25T19:00:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.025+0000 7f8449400640 1 Processor -- start 2024-04-25T19:00:15.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.025+0000 7f8449400640 1 -- start start 2024-04-25T19:00:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.025+0000 7f8449400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8444102fb0 0x7f8444100c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.025+0000 7f8449400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84441038d0 0x7f8444101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8449400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f844410c380 0x7f8444101680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8449400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8444111ff0 con 0x7f8444102fb0 2024-04-25T19:00:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8449400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f8444111e70 con 0x7f844410c380 2024-04-25T19:00:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8449400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8444112170 con 0x7f84441038d0 2024-04-25T19:00:15.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8442800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8444102fb0 0x7f8444100c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8443200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f844410c380 0x7f8444101680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8441e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84441038d0 0x7f8444101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8443200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f844410c380 0x7f8444101680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:46952/0 (socket says 172.21.15.45:46952) 2024-04-25T19:00:15.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8442800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8444102fb0 0x7f8444100c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54340/0 (socket says 172.21.15.45:54340) 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8441e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84441038d0 0x7f8444101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:57204/0 (socket says 172.21.15.45:57204) 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8443200640 1 -- 172.21.15.45:0/3875359734 learned_addr learned my addr 172.21.15.45:0/3875359734 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8443200640 1 -- 172.21.15.45:0/3875359734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84441038d0 msgr2=0x7f8444101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8443200640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84441038d0 0x7f8444101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8443200640 1 -- 172.21.15.45:0/3875359734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8444102fb0 msgr2=0x7f8444100c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8443200640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8444102fb0 0x7f8444100c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.026+0000 7f8443200640 1 -- 172.21.15.45:0/3875359734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84441a6a30 con 0x7f844410c380 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.027+0000 7f8443200640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f844410c380 0x7f8444101680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f843c009b30 tx=0x7f843c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.027+0000 7f843b400640 1 -- 172.21.15.45:0/3875359734 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c041020 con 0x7f844410c380 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.027+0000 7f843b400640 1 -- 172.21.15.45:0/3875359734 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843c0318f0 con 0x7f844410c380 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.027+0000 7f8449400640 1 -- 172.21.15.45:0/3875359734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84441a6cc0 con 0x7f844410c380 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.027+0000 7f843b400640 1 -- 172.21.15.45:0/3875359734 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c036040 con 0x7f844410c380 2024-04-25T19:00:15.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.027+0000 7f8449400640 1 -- 172.21.15.45:0/3875359734 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84441a7200 con 0x7f844410c380 2024-04-25T19:00:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.032+0000 7f843b400640 1 -- 172.21.15.45:0/3875359734 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f843c044020 con 0x7f844410c380 2024-04-25T19:00:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.032+0000 7f844c1e9740 1 Processor -- start 2024-04-25T19:00:15.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.033+0000 7f844c1e9740 1 -- start start 2024-04-25T19:00:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.033+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 0x565513d92d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.033+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d932d0 0x565513d9d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.033+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 0x565513d9ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.033+0000 7f844c1e9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x565513cf8900 con 0x565513d9db90 2024-04-25T19:00:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.033+0000 7f844c1e9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565513c9eef0 con 0x565513d932d0 2024-04-25T19:00:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.033+0000 7f844c1e9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565513cf48c0 con 0x565513d929b0 2024-04-25T19:00:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.033+0000 7f8441e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 0x565513d92d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f8441e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 0x565513d92d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:57220/0 (socket says 172.21.15.45:57220) 2024-04-25T19:00:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f8441e00640 1 -- 172.21.15.45:0/2286954287 learned_addr learned my addr 172.21.15.45:0/2286954287 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.033+0000 7f8442800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d932d0 0x565513d9d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f8441e00640 1 --2- 172.21.15.45:0/2286954287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 0x565513d9ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f843b400640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f84280a3f60 0x7f84280a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f8441e00640 1 -- 172.21.15.45:0/2286954287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d932d0 msgr2=0x565513d9d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f8441e00640 1 --2- 172.21.15.45:0/2286954287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d932d0 0x565513d9d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f8441e00640 1 -- 172.21.15.45:0/2286954287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 msgr2=0x565513d9ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f8441e00640 1 --2- 172.21.15.45:0/2286954287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 0x565513d9ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f8441e00640 1 -- 172.21.15.45:0/2286954287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565513a0b210 con 0x565513d929b0 2024-04-25T19:00:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f843b400640 1 -- 172.21.15.45:0/3875359734 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f843c0f1550 con 0x7f844410c380 2024-04-25T19:00:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.034+0000 7f8441e00640 1 --2- 172.21.15.45:0/2286954287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 0x565513d92d90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x565513d90050 tx=0x7f8430005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fccb250d06b23232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.035+0000 7f8443200640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f84280a3f60 0x7f84280a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.035+0000 7f842f400640 1 -- 172.21.15.45:0/2286954287 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843000f790 con 0x565513d929b0 2024-04-25T19:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.035+0000 7f842f400640 1 -- 172.21.15.45:0/2286954287 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843000fde0 con 0x565513d929b0 2024-04-25T19:00:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.035+0000 7f842f400640 1 -- 172.21.15.45:0/2286954287 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843001fcf0 con 0x565513d929b0 2024-04-25T19:00:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.036+0000 7f844c1e9740 1 -- 172.21.15.45:0/2286954287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 msgr2=0x565513d92d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.036+0000 7f8443200640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f84280a3f60 0x7f84280a6420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f843c0eeac0 tx=0x7f843c009280 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.036+0000 7f844c1e9740 1 --2- 172.21.15.45:0/2286954287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 0x565513d92d90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x565513d90050 tx=0x7f8430005850 comp rx=0 tx=0).stop 2024-04-25T19:00:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.036+0000 7f844c1e9740 1 -- 172.21.15.45:0/2286954287 shutdown_connections 2024-04-25T19:00:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.036+0000 7f844c1e9740 1 --2- 172.21.15.45:0/2286954287 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 0x565513d9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.036+0000 7f844c1e9740 1 --2- 172.21.15.45:0/2286954287 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d932d0 0x565513d9d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.036+0000 7f844c1e9740 1 --2- 172.21.15.45:0/2286954287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 0x565513d92d90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.036+0000 7f844c1e9740 1 -- 172.21.15.45:0/2286954287 >> 172.21.15.45:0/2286954287 conn(0x565513d8eec0 msgr2=0x565513d8f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.036+0000 7f844c1e9740 1 -- 172.21.15.45:0/2286954287 shutdown_connections 2024-04-25T19:00:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.036+0000 7f844c1e9740 1 -- 172.21.15.45:0/2286954287 wait complete. 2024-04-25T19:00:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.037+0000 7f844c1e9740 1 -- start start 2024-04-25T19:00:15.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.037+0000 7f844c1e9740 1 Processor -- start 2024-04-25T19:00:15.040 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d929b0 0x565513d9ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d932d0 0x565513d9b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d9db90 0x565513d9b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f844c1e9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x565513da07c0 con 0x565513d932d0 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f844c1e9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x565513cb2310 con 0x565513d929b0 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f844c1e9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x565513da0940 con 0x565513d9db90 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f844c1e9740 10 client.0 _ll_register_callbacks cb 0x565513c80cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f8442800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d932d0 0x565513d9b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f8441e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d929b0 0x565513d9ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f8441e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d929b0 0x565513d9ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:46976/0 (socket says 172.21.15.45:46976) 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f8441e00640 1 -- 172.21.15.45:0/526637757 learned_addr learned my addr 172.21.15.45:0/526637757 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.038+0000 7f8441e00640 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d9db90 0x565513d9b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.039+0000 7f8442800640 1 -- 172.21.15.45:0/526637757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d9db90 msgr2=0x565513d9b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:15.041 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.039+0000 7f8442800640 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d9db90 0x565513d9b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.039+0000 7f8442800640 1 -- 172.21.15.45:0/526637757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d929b0 msgr2=0x565513d9ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.039+0000 7f8442800640 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d929b0 0x565513d9ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.039+0000 7f8442800640 1 -- 172.21.15.45:0/526637757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565513ad7620 con 0x565513d932d0 2024-04-25T19:00:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.039+0000 7f8442800640 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d932d0 0x565513d9b150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8434005fd0 tx=0x7f8434012b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a0fe26849af5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.039+0000 7f8423e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:00:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.040+0000 7f8423e00640 1 -- 172.21.15.45:0/526637757 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843400f920 con 0x565513d932d0 2024-04-25T19:00:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.040+0000 7f844c1e9740 1 -- 172.21.15.45:0/526637757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843000fde0 con 0x565513d932d0 2024-04-25T19:00:15.042 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.040+0000 7f8423e00640 1 -- 172.21.15.45:0/526637757 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f843400fac0 con 0x565513d932d0 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.040+0000 7f8423e00640 1 -- 172.21.15.45:0/526637757 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f843400fcc0 con 0x565513d932d0 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.040+0000 7f844c1e9740 10 client.15192 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.040+0000 7f844c1e9740 1 -- 172.21.15.45:0/526637757 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565513dacba0 con 0x565513d932d0 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.041+0000 7f8423e00640 1 -- 172.21.15.45:0/526637757 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843000fde0 con 0x565513d932d0 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.041+0000 7f8423e00640 1 -- 172.21.15.45:0/526637757 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f843401c420 con 0x565513d932d0 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.041+0000 7f844c1e9740 10 client.15192 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.041+0000 7f844c1e9740 10 client.15192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.042+0000 7f844c1e9740 20 client.15192 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.042+0000 7f844c1e9740 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x565513dae010 0x565513db0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.042+0000 7f844c1e9740 4 client.15192 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.042+0000 7f844c1e9740 1 -- 172.21.15.45:0/526637757 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565513d0c8b0 con 0x565513dae010 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.042+0000 7f8441e00640 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x565513dae010 0x565513db0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.043+0000 7f8423e00640 10 client.15192 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:15.044 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.043+0000 7f8441e00640 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x565513dae010 0x565513db0460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8415d40559cfe9e server_cookie=bba6f392ace96f2b in_seq=0 out_seq=0 2024-04-25T19:00:15.045 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.043+0000 7f8423e00640 1 -- 172.21.15.45:0/526637757 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565513d0c8b0 con 0x565513dae010 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.043+0000 7f8423e00640 10 client.15192 handle_command_reply: tid=0 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.043+0000 7f844c1e9740 1 client.15192 shutdown 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 1 -- 172.21.15.45:0/526637757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d932d0 msgr2=0x565513d9b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d932d0 0x565513d9b150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8434005fd0 tx=0x7f8434012b40 comp rx=0 tx=0).stop 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 1 -- 172.21.15.45:0/526637757 shutdown_connections 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x565513dae010 0x565513db0460 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d9db90 0x565513d9b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d932d0 0x565513d9b150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 1 --2- 172.21.15.45:0/526637757 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d929b0 0x565513d9ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 1 -- 172.21.15.45:0/526637757 >> 172.21.15.45:0/526637757 conn(0x565513d8eec0 msgr2=0x565513d94bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f8423e00640 0 client.15192 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 1 -- 172.21.15.45:0/526637757 shutdown_connections 2024-04-25T19:00:15.046 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 1 -- 172.21.15.45:0/526637757 wait complete. 2024-04-25T19:00:15.047 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.044+0000 7f844c1e9740 20 client.15192 trim_cache size 0 max 16384 2024-04-25T19:00:15.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.071+0000 7f844c1e9740 1 Processor -- start 2024-04-25T19:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.071+0000 7f844c1e9740 1 -- start start 2024-04-25T19:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d929b0 0x565513da9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d9db90 0x565513d8e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513db5d10 0x565513db6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f844c1e9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84441b24f0 con 0x565513d929b0 2024-04-25T19:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f844c1e9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f84441afdc0 con 0x565513d9db90 2024-04-25T19:00:15.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f844c1e9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f84441adc10 con 0x565513db5d10 2024-04-25T19:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f8442800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d9db90 0x565513d8e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f8441e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d929b0 0x565513da9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f8442800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d9db90 0x565513d8e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:46980/0 (socket says 172.21.15.45:46980) 2024-04-25T19:00:15.074 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f8442800640 1 -- 172.21.15.45:0/114657755 learned_addr learned my addr 172.21.15.45:0/114657755 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.072+0000 7f8441e00640 1 --2- 172.21.15.45:0/114657755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513db5d10 0x565513db6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.073+0000 7f8442800640 1 -- 172.21.15.45:0/114657755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513db5d10 msgr2=0x565513db6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.073+0000 7f8442800640 1 --2- 172.21.15.45:0/114657755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513db5d10 0x565513db6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.073+0000 7f8442800640 1 -- 172.21.15.45:0/114657755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d929b0 msgr2=0x565513da9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.073+0000 7f8442800640 1 --2- 172.21.15.45:0/114657755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d929b0 0x565513da9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.073+0000 7f8442800640 1 -- 172.21.15.45:0/114657755 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565513d8a820 con 0x565513d9db90 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.073+0000 7f8441e00640 1 --2- 172.21.15.45:0/114657755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d929b0 0x565513da9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.073+0000 7f8442800640 1 --2- 172.21.15.45:0/114657755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d9db90 0x565513d8e7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8434000c00 tx=0x7f8434009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e5d35fcb992d567 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.073+0000 7f842ea00640 1 -- 172.21.15.45:0/114657755 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843400f920 con 0x565513d9db90 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.073+0000 7f842ea00640 1 -- 172.21.15.45:0/114657755 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843400fac0 con 0x565513d9db90 2024-04-25T19:00:15.075 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.073+0000 7f842ea00640 1 -- 172.21.15.45:0/114657755 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843401c420 con 0x565513d9db90 2024-04-25T19:00:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.074+0000 7f844c1e9740 1 -- 172.21.15.45:0/114657755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d9db90 msgr2=0x565513d8e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.074+0000 7f844c1e9740 1 --2- 172.21.15.45:0/114657755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d9db90 0x565513d8e7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8434000c00 tx=0x7f8434009fd0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.075+0000 7f844c1e9740 1 -- 172.21.15.45:0/114657755 shutdown_connections 2024-04-25T19:00:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.075+0000 7f844c1e9740 1 --2- 172.21.15.45:0/114657755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513db5d10 0x565513db6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.075+0000 7f844c1e9740 1 --2- 172.21.15.45:0/114657755 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513d9db90 0x565513d8e7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.075+0000 7f844c1e9740 1 --2- 172.21.15.45:0/114657755 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d929b0 0x565513da9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.075+0000 7f844c1e9740 1 -- 172.21.15.45:0/114657755 >> 172.21.15.45:0/114657755 conn(0x565513d8eec0 msgr2=0x565513d9b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.075+0000 7f844c1e9740 1 -- 172.21.15.45:0/114657755 shutdown_connections 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.075+0000 7f844c1e9740 1 -- 172.21.15.45:0/114657755 wait complete. 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.075+0000 7f844c1e9740 1 -- start start 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.075+0000 7f844c1e9740 1 Processor -- start 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 0x565513da60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 0x565513da34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f844c1e9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513db5d10 0x565513da3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f844c1e9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f84441ae280 con 0x565513d9db90 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f844c1e9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f84441af6f0 con 0x565513db5d10 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f844c1e9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f84441ae950 con 0x565513d929b0 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f844c1e9740 10 client.0 _ll_register_callbacks cb 0x7f8434009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f8442800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 0x565513da34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f8442800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 0x565513da34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54388/0 (socket says 172.21.15.45:54388) 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f8441e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 0x565513da60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f8442800640 1 -- 172.21.15.45:0/709656742 learned_addr learned my addr 172.21.15.45:0/709656742 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.076+0000 7f8441e00640 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513db5d10 0x565513da3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.077+0000 7f8442800640 1 -- 172.21.15.45:0/709656742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 msgr2=0x565513da60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.077+0000 7f8442800640 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 0x565513da60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.077+0000 7f8442800640 1 -- 172.21.15.45:0/709656742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513db5d10 msgr2=0x565513da3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.077+0000 7f8442800640 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513db5d10 0x565513da3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.077+0000 7f8442800640 1 -- 172.21.15.45:0/709656742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565513e13590 con 0x565513d9db90 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.077+0000 7f8441e00640 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513db5d10 0x565513da3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.077+0000 7f8442800640 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 0x565513da34d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f843401c890 tx=0x7f843401cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71edb7e893d4dfd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.077+0000 7f8423e00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.078+0000 7f8423e00640 1 -- 172.21.15.45:0/709656742 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8434032070 con 0x565513d9db90 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.078+0000 7f8423e00640 1 -- 172.21.15.45:0/709656742 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f8434018040 con 0x565513d9db90 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.078+0000 7f844c1e9740 1 -- 172.21.15.45:0/709656742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f843400fac0 con 0x565513d9db90 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.078+0000 7f8423e00640 1 -- 172.21.15.45:0/709656742 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f843400f920 con 0x565513d9db90 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.078+0000 7f844c1e9740 10 client.15198 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.078+0000 7f844c1e9740 1 -- 172.21.15.45:0/709656742 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x565513e14740 con 0x565513d9db90 2024-04-25T19:00:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.078+0000 7f8423e00640 1 -- 172.21.15.45:0/709656742 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f843400fac0 con 0x565513d9db90 2024-04-25T19:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.078+0000 7f8423e00640 1 -- 172.21.15.45:0/709656742 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f8434010430 con 0x565513d9db90 2024-04-25T19:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.078+0000 7f844c1e9740 10 client.15198 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.079+0000 7f844c1e9740 10 client.15198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.079+0000 7f844c1e9740 20 client.15198 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.079+0000 7f844c1e9740 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x565513e14db0 0x565513e171d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.079+0000 7f844c1e9740 4 client.15198 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.079+0000 7f844c1e9740 1 -- 172.21.15.45:0/709656742 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565513d06320 con 0x565513e14db0 2024-04-25T19:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.079+0000 7f8441e00640 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x565513e14db0 0x565513e171d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.080+0000 7f8423e00640 10 client.15198 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.080+0000 7f8441e00640 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x565513e14db0 0x565513e171d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b8a19f76136ef02 server_cookie=8ae96bc399a08b0b in_seq=0 out_seq=0 2024-04-25T19:00:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f8423e00640 1 -- 172.21.15.45:0/709656742 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565513d06320 con 0x565513e14db0 2024-04-25T19:00:15.085 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f8423e00640 10 client.15198 handle_command_reply: tid=0 2024-04-25T19:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 client.15198 shutdown 2024-04-25T19:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 -- 172.21.15.45:0/709656742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 msgr2=0x565513da34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 0x565513da34d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f843401c890 tx=0x7f843401cb00 comp rx=0 tx=0).stop 2024-04-25T19:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 -- 172.21.15.45:0/709656742 shutdown_connections 2024-04-25T19:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x565513e14db0 0x565513e171d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x565513db5d10 0x565513da3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.086 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x565513d9db90 0x565513da34d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 --2- 172.21.15.45:0/709656742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x565513d929b0 0x565513da60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f8423e00640 0 client.15198 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:15.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 -- 172.21.15.45:0/709656742 >> 172.21.15.45:0/709656742 conn(0x565513d8eec0 msgr2=0x565513db5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:15.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 -- 172.21.15.45:0/709656742 shutdown_connections 2024-04-25T19:00:15.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 1 -- 172.21.15.45:0/709656742 wait complete. 2024-04-25T19:00:15.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.084+0000 7f844c1e9740 20 client.15198 trim_cache size 0 max 16384 2024-04-25T19:00:15.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:00:15.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:00:15.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:00:15.087 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.085+0000 7f8449400640 1 -- 172.21.15.45:0/3875359734 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f84280a3f60 msgr2=0x7f84280a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.085+0000 7f8449400640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f84280a3f60 0x7f84280a6420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f843c0eeac0 tx=0x7f843c009280 comp rx=0 tx=0).stop 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.085+0000 7f8449400640 1 -- 172.21.15.45:0/3875359734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f844410c380 msgr2=0x7f8444101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.085+0000 7f8449400640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f844410c380 0x7f8444101680 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f843c009b30 tx=0x7f843c003ba0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.085+0000 7f8449400640 1 -- 172.21.15.45:0/3875359734 shutdown_connections 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.085+0000 7f8449400640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f84280a3f60 0x7f84280a6420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.085+0000 7f8449400640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f844410c380 0x7f8444101680 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.085+0000 7f8449400640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f84441038d0 0x7f8444101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.085+0000 7f8449400640 1 --2- 172.21.15.45:0/3875359734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8444102fb0 0x7f8444100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.085+0000 7f8449400640 1 -- 172.21.15.45:0/3875359734 >> 172.21.15.45:0/3875359734 conn(0x7f84440fc750 msgr2=0x7f84440fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.086+0000 7f8449400640 1 -- 172.21.15.45:0/3875359734 shutdown_connections 2024-04-25T19:00:15.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:15.086+0000 7f8449400640 1 -- 172.21.15.45:0/3875359734 wait complete. 2024-04-25T19:00:16.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:15 smithi079 ceph-mon[33002]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T19:00:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:15 smithi045 ceph-mon[27776]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T19:00:16.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:15 smithi062 ceph-mon[32844]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T19:00:18.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:17 smithi079 ceph-mon[33002]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T19:00:18.186 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:17 smithi062 ceph-mon[32844]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T19:00:18.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:17 smithi045 ceph-mon[27776]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 24 op/s 2024-04-25T19:00:20.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:19 smithi045 ceph-mon[27776]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T19:00:20.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:19 smithi062 ceph-mon[32844]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T19:00:20.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:19 smithi079 ceph-mon[33002]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T19:00:20.738 INFO:tasks.quiescer.fs.[cephfs]:Total duration (58.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T19:00:20.738 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.4 2024-04-25T19:00:20.738 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:00:21.227 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:21.939 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:21 smithi045 ceph-mon[27776]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T19:00:22.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:21 smithi062 ceph-mon[32844]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T19:00:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:21 smithi079 ceph-mon[33002]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T19:00:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.336+0000 7fe697200700 1 -- 172.21.15.45:0/847550957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe69006b0f0 msgr2=0x7fe69010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.336+0000 7fe697200700 1 --2- 172.21.15.45:0/847550957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe69006b0f0 0x7fe69010f900 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe68800b0a0 tx=0x7fe68800b3b0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.336+0000 7fe697200700 1 -- 172.21.15.45:0/847550957 shutdown_connections 2024-04-25T19:00:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.336+0000 7fe697200700 1 --2- 172.21.15.45:0/847550957 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe69006b0f0 0x7fe69010f900 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.336+0000 7fe697200700 1 --2- 172.21.15.45:0/847550957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe690108a30 0x7fe690108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.336+0000 7fe697200700 1 --2- 172.21.15.45:0/847550957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe69006a750 0x7fe69006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.336+0000 7fe697200700 1 -- 172.21.15.45:0/847550957 >> 172.21.15.45:0/847550957 conn(0x7fe6900788f0 msgr2=0x7fe690078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.336+0000 7fe697200700 1 -- 172.21.15.45:0/847550957 shutdown_connections 2024-04-25T19:00:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.336+0000 7fe697200700 1 -- 172.21.15.45:0/847550957 wait complete. 2024-04-25T19:00:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.338+0000 7fe697200700 1 Processor -- start 2024-04-25T19:00:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.338+0000 7fe697200700 1 -- start start 2024-04-25T19:00:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.338+0000 7fe697200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69006a750 0x7fe69019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:23.339 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.338+0000 7fe697200700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe69006b0f0 0x7fe69019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.338+0000 7fe697200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690108a30 0x7fe6901a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.338+0000 7fe697200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe690110a60 con 0x7fe69006a750 2024-04-25T19:00:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.338+0000 7fe697200700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe6901108e0 con 0x7fe690108a30 2024-04-25T19:00:23.340 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.338+0000 7fe697200700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe690110be0 con 0x7fe69006b0f0 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.339+0000 7fe68fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69006a750 0x7fe69019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.339+0000 7fe695200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690108a30 0x7fe6901a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.339+0000 7fe695200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690108a30 0x7fe6901a4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:47002/0 (socket says 172.21.15.45:47002) 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.339+0000 7fe695200700 1 -- 172.21.15.45:0/1350842834 learned_addr learned my addr 172.21.15.45:0/1350842834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.339+0000 7fe68f400700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe69006b0f0 0x7fe69019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.340+0000 7fe695200700 1 -- 172.21.15.45:0/1350842834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe69006b0f0 msgr2=0x7fe69019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.340+0000 7fe695200700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe69006b0f0 0x7fe69019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.340+0000 7fe695200700 1 -- 172.21.15.45:0/1350842834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69006a750 msgr2=0x7fe69019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.340+0000 7fe695200700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69006a750 0x7fe69019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.340+0000 7fe695200700 1 -- 172.21.15.45:0/1350842834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6901a4750 con 0x7fe690108a30 2024-04-25T19:00:23.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.340+0000 7fe695200700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690108a30 0x7fe6901a4010 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe68800b780 tx=0x7fe688004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.340+0000 7fe68cc00700 1 -- 172.21.15.45:0/1350842834 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688021070 con 0x7fe690108a30 2024-04-25T19:00:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.340+0000 7fe68cc00700 1 -- 172.21.15.45:0/1350842834 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe688007e30 con 0x7fe690108a30 2024-04-25T19:00:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.340+0000 7fe68cc00700 1 -- 172.21.15.45:0/1350842834 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688004700 con 0x7fe690108a30 2024-04-25T19:00:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.341+0000 7fe697200700 1 -- 172.21.15.45:0/1350842834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6901a49e0 con 0x7fe690108a30 2024-04-25T19:00:23.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.341+0000 7fe697200700 1 -- 172.21.15.45:0/1350842834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6901ac2c0 con 0x7fe690108a30 2024-04-25T19:00:23.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.342+0000 7fe697200700 1 -- 172.21.15.45:0/1350842834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe690104170 con 0x7fe690108a30 2024-04-25T19:00:23.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.346+0000 7fe68cc00700 1 -- 172.21.15.45:0/1350842834 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7fe688007980 con 0x7fe690108a30 2024-04-25T19:00:23.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.348+0000 7fe68cc00700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe6740a4060 0x7fe6740a6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.349+0000 7fe68cc00700 1 -- 172.21.15.45:0/1350842834 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fe68800f020 con 0x7fe690108a30 2024-04-25T19:00:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.349+0000 7fe68fe00700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe6740a4060 0x7fe6740a6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:23.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.350+0000 7fe68fe00700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe6740a4060 0x7fe6740a6520 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe680009be0 tx=0x7fe680009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:23.351 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.349+0000 7fe68cc00700 1 -- 172.21.15.45:0/1350842834 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe68802a020 con 0x7fe690108a30 2024-04-25T19:00:23.625 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.624+0000 7fe697200700 1 -- 172.21.15.45:0/1350842834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe69002df80 con 0x7fe690108a30 2024-04-25T19:00:23.627 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:00:23.627 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:59:18.321335+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24512,24505]},"id":1}]} 2024-04-25T19:00:23.627 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.625+0000 7fe68cc00700 1 -- 172.21.15.45:0/1350842834 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7fe688098e20 con 0x7fe690108a30 2024-04-25T19:00:23.629 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.628+0000 7fe697200700 1 -- 172.21.15.45:0/1350842834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe6740a4060 msgr2=0x7fe6740a6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.628+0000 7fe697200700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe6740a4060 0x7fe6740a6520 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe680009be0 tx=0x7fe680009260 comp rx=0 tx=0).stop 2024-04-25T19:00:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.628+0000 7fe697200700 1 -- 172.21.15.45:0/1350842834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690108a30 msgr2=0x7fe6901a4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.628+0000 7fe697200700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690108a30 0x7fe6901a4010 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe68800b780 tx=0x7fe688004d00 comp rx=0 tx=0).stop 2024-04-25T19:00:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.629+0000 7fe697200700 1 -- 172.21.15.45:0/1350842834 shutdown_connections 2024-04-25T19:00:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.629+0000 7fe697200700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe6740a4060 0x7fe6740a6520 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.629+0000 7fe697200700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe690108a30 0x7fe6901a4010 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.629+0000 7fe697200700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe69006a750 0x7fe69019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.629+0000 7fe697200700 1 --2- 172.21.15.45:0/1350842834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe69006b0f0 0x7fe69019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.629+0000 7fe697200700 1 -- 172.21.15.45:0/1350842834 >> 172.21.15.45:0/1350842834 conn(0x7fe6900788f0 msgr2=0x7fe6900ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:23.630 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.629+0000 7fe697200700 1 -- 172.21.15.45:0/1350842834 shutdown_connections 2024-04-25T19:00:23.631 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:23.629+0000 7fe697200700 1 -- 172.21.15.45:0/1350842834 wait complete. 2024-04-25T19:00:23.632 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-04-25T19:00:24.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:23 smithi045 ceph-mon[27776]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T19:00:24.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:23 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1350842834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:00:24.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:23 smithi062 ceph-mon[32844]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T19:00:24.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:23 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1350842834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:00:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:23 smithi079 ceph-mon[33002]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T19:00:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:23 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1350842834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:00:24.546 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 60 --expiration 89.4 --await-for 42.0' 2024-04-25T19:00:24.546 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24524 quiesce db / --timeout 60 --expiration 89.4 --await-for 42.0 2024-04-25T19:00:25.119 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:26.197 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:25 smithi045 ceph-mon[27776]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T19:00:26.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:25 smithi062 ceph-mon[32844]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T19:00:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:25 smithi079 ceph-mon[33002]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T19:00:27.484 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.482+0000 7f5209600700 1 -- 172.21.15.45:0/1918367703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5204108a60 msgr2=0x7f5204108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:27.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.482+0000 7f5209600700 1 --2- 172.21.15.45:0/1918367703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5204108a60 0x7f5204108e40 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f51f4009990 tx=0x7f51f4009ca0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.483+0000 7f5209600700 1 -- 172.21.15.45:0/1918367703 shutdown_connections 2024-04-25T19:00:27.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.483+0000 7f5209600700 1 --2- 172.21.15.45:0/1918367703 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f5204108a60 0x7f5204108e40 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.483+0000 7f5209600700 1 --2- 172.21.15.45:0/1918367703 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520406b100 0x7f520410f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.483+0000 7f5209600700 1 --2- 172.21.15.45:0/1918367703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f520406a740 0x7f520406abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.483+0000 7f5209600700 1 -- 172.21.15.45:0/1918367703 >> 172.21.15.45:0/1918367703 conn(0x7f52040788e0 msgr2=0x7f5204078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:27.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.483+0000 7f5209600700 1 -- 172.21.15.45:0/1918367703 shutdown_connections 2024-04-25T19:00:27.485 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.483+0000 7f5209600700 1 -- 172.21.15.45:0/1918367703 wait complete. 2024-04-25T19:00:27.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.485+0000 7f5209600700 1 Processor -- start 2024-04-25T19:00:27.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.485+0000 7f5209600700 1 -- start start 2024-04-25T19:00:27.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.485+0000 7f5209600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520406a740 0x7f520419fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.488 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.485+0000 7f5209600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520406b100 0x7f52041a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.485+0000 7f5209600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204108a60 0x7f520410e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.485+0000 7f5209600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5204112c40 con 0x7f5204108a60 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.485+0000 7f5209600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5204112ac0 con 0x7f520406a740 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.485+0000 7f5209600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5204112dc0 con 0x7f520406b100 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5202a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520406a740 0x7f520419fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5202a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520406a740 0x7f520419fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48574/0 (socket says 172.21.15.45:48574) 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5202a00700 1 -- 172.21.15.45:0/1675528769 learned_addr learned my addr 172.21.15.45:0/1675528769 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5202000700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520406b100 0x7f52041a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5203400700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204108a60 0x7f520410e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5202a00700 1 -- 172.21.15.45:0/1675528769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520406b100 msgr2=0x7f52041a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5202a00700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520406b100 0x7f52041a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5202a00700 1 -- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204108a60 msgr2=0x7f520410e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5202a00700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204108a60 0x7f520410e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5202a00700 1 -- 172.21.15.45:0/1675528769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520410f150 con 0x7f520406a740 2024-04-25T19:00:27.489 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5202000700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520406b100 0x7f52041a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:00:27.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.486+0000 7f5203400700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204108a60 0x7f520410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:00:27.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.487+0000 7f5202a00700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520406a740 0x7f520419fc00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51f4005910 tx=0x7f51f400b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:27.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.487+0000 7f51fb400700 1 -- 172.21.15.45:0/1675528769 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4027070 con 0x7f520406a740 2024-04-25T19:00:27.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.487+0000 7f5209600700 1 -- 172.21.15.45:0/1675528769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f520410f3e0 con 0x7f520406a740 2024-04-25T19:00:27.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.487+0000 7f5209600700 1 -- 172.21.15.45:0/1675528769 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f520419b740 con 0x7f520406a740 2024-04-25T19:00:27.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.488+0000 7f51fb400700 1 -- 172.21.15.45:0/1675528769 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f4022470 con 0x7f520406a740 2024-04-25T19:00:27.490 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.488+0000 7f51fb400700 1 -- 172.21.15.45:0/1675528769 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f4017650 con 0x7f520406a740 2024-04-25T19:00:27.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.493+0000 7f51fb400700 1 -- 172.21.15.45:0/1675528769 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f51f402a070 con 0x7f520406a740 2024-04-25T19:00:27.494 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.494+0000 7f5213ffbb80 1 Processor -- start 2024-04-25T19:00:27.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.494+0000 7f5213ffbb80 1 -- start start 2024-04-25T19:00:27.495 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.494+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27009b0 0x55f1c2700d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27012d0 0x55f1c270bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c270c180 0x55f1c270e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f1c2654020 con 0x55f1c270c180 2024-04-25T19:00:27.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f1c25fa8c0 con 0x55f1c27009b0 2024-04-25T19:00:27.496 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f1c264ffe0 con 0x55f1c27012d0 2024-04-25T19:00:27.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f51fb400700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51e80a4100 0x7f51e80a65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f5203400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27009b0 0x55f1c2700d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.497 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f5202a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27012d0 0x55f1c270bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f5202a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27012d0 0x55f1c270bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32836/0 (socket says 172.21.15.45:32836) 2024-04-25T19:00:27.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f5202a00700 1 -- 172.21.15.45:0/240712481 learned_addr learned my addr 172.21.15.45:0/240712481 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:27.498 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.495+0000 7f5202000700 1 --2- 172.21.15.45:0/240712481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c270c180 0x55f1c270e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.496+0000 7f51fb400700 1 -- 172.21.15.45:0/1675528769 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f51f40d1730 con 0x7f520406a740 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.496+0000 7f5203400700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51e80a4100 0x7f51e80a65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.496+0000 7f5202000700 1 -- 172.21.15.45:0/240712481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27012d0 msgr2=0x55f1c270bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.496+0000 7f5202000700 1 --2- 172.21.15.45:0/240712481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27012d0 0x55f1c270bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.496+0000 7f5202000700 1 -- 172.21.15.45:0/240712481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27009b0 msgr2=0x55f1c2700d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.496+0000 7f5202000700 1 --2- 172.21.15.45:0/240712481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27009b0 0x55f1c2700d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.496+0000 7f5202000700 1 -- 172.21.15.45:0/240712481 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c25f0810 con 0x55f1c270c180 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.496+0000 7f5202a00700 1 --2- 172.21.15.45:0/240712481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27012d0 0x55f1c270bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.497+0000 7f5202000700 1 --2- 172.21.15.45:0/240712481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c270c180 0x55f1c270e570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009cc0 tx=0x7f51f00113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7b61476a72640b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.497+0000 7f51efe00700 1 -- 172.21.15.45:0/240712481 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0011b70 con 0x55f1c270c180 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.498+0000 7f5203400700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51e80a4100 0x7f51e80a65c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009fd0 tx=0x7f51fc004b40 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.498+0000 7f51efe00700 1 -- 172.21.15.45:0/240712481 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f0011d10 con 0x55f1c270c180 2024-04-25T19:00:27.499 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.498+0000 7f51efe00700 1 -- 172.21.15.45:0/240712481 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f000f6c0 con 0x55f1c270c180 2024-04-25T19:00:27.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.498+0000 7f5213ffbb80 1 -- 172.21.15.45:0/240712481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c270c180 msgr2=0x55f1c270e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:27.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.499+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/240712481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c270c180 0x55f1c270e570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009cc0 tx=0x7f51f00113e0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.499+0000 7f5213ffbb80 1 -- 172.21.15.45:0/240712481 shutdown_connections 2024-04-25T19:00:27.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.499+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/240712481 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27009b0 0x55f1c2700d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.499+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/240712481 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c270c180 0x55f1c270e570 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.499+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/240712481 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27012d0 0x55f1c270bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.499+0000 7f5213ffbb80 1 -- 172.21.15.45:0/240712481 >> 172.21.15.45:0/240712481 conn(0x55f1c262f730 msgr2=0x55f1c26fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:27.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.499+0000 7f5213ffbb80 1 -- 172.21.15.45:0/240712481 shutdown_connections 2024-04-25T19:00:27.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.499+0000 7f5213ffbb80 1 -- 172.21.15.45:0/240712481 wait complete. 2024-04-25T19:00:27.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.500+0000 7f5213ffbb80 1 -- start start 2024-04-25T19:00:27.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.500+0000 7f5213ffbb80 1 Processor -- start 2024-04-25T19:00:27.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.500+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 0x55f1c270ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.500+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 0x55f1c2708d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.500+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27092c0 0x55f1c2709740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.500+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f1c270eca0 con 0x55f1c27092c0 2024-04-25T19:00:27.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.500+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f1c260d9f0 con 0x55f1c27012d0 2024-04-25T19:00:27.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.500+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f1c270ee20 con 0x55f1c27009b0 2024-04-25T19:00:27.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.501+0000 7f5213ffbb80 10 client.0 _ll_register_callbacks cb 0x55f1c25ec8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:27.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.501+0000 7f5202a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 0x55f1c270ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.501+0000 7f5202a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 0x55f1c270ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32840/0 (socket says 172.21.15.45:32840) 2024-04-25T19:00:27.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.501+0000 7f5202a00700 1 -- 172.21.15.45:0/539362363 learned_addr learned my addr 172.21.15.45:0/539362363 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:27.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f5202a00700 1 -- 172.21.15.45:0/539362363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 msgr2=0x55f1c2708d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:27.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f5202a00700 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 0x55f1c2708d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f5202a00700 1 -- 172.21.15.45:0/539362363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27092c0 msgr2=0x55f1c2709740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:27.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f5202a00700 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27092c0 0x55f1c2709740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f5202a00700 1 -- 172.21.15.45:0/539362363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c25a1a60 con 0x55f1c27009b0 2024-04-25T19:00:27.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f5202a00700 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 0x55f1c270ba80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f51f4099ef0 tx=0x7f51f4009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be75941638c7228f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:27.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f51ecc00700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:00:27.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f51ecc00700 1 -- 172.21.15.45:0/539362363 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f40cae60 con 0x55f1c27009b0 2024-04-25T19:00:27.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f51ecc00700 1 -- 172.21.15.45:0/539362363 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f51f409cb70 con 0x55f1c27009b0 2024-04-25T19:00:27.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f5213ffbb80 1 -- 172.21.15.45:0/539362363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51f0011d10 con 0x55f1c27009b0 2024-04-25T19:00:27.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.502+0000 7f51ecc00700 1 -- 172.21.15.45:0/539362363 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51f40095c0 con 0x55f1c27009b0 2024-04-25T19:00:27.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.503+0000 7f5213ffbb80 10 client.25345 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:27.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.503+0000 7f5213ffbb80 1 -- 172.21.15.45:0/539362363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1c25eac80 con 0x55f1c27009b0 2024-04-25T19:00:27.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.503+0000 7f51ecc00700 1 -- 172.21.15.45:0/539362363 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f0011d10 con 0x55f1c27009b0 2024-04-25T19:00:27.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.504+0000 7f51ecc00700 1 -- 172.21.15.45:0/539362363 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f51f40cef50 con 0x55f1c27009b0 2024-04-25T19:00:27.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.504+0000 7f5213ffbb80 10 client.25345 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:27.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.504+0000 7f5213ffbb80 10 client.25345 resolve_mds: validated gid 24524 aka daemon mds.b 2024-04-25T19:00:27.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.504+0000 7f5213ffbb80 20 client.25345 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:27.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.504+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f1c271bea0 0x55f1c271e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.504+0000 7f5213ffbb80 4 client.25345 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:00:27.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.504+0000 7f5213ffbb80 1 -- 172.21.15.45:0/539362363 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1c2661810 con 0x55f1c271bea0 2024-04-25T19:00:27.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.505+0000 7f5202000700 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f1c271bea0 0x55f1c271e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.505+0000 7f5202000700 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f1c271bea0 0x55f1c271e360 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42df2b2b879b0844 server_cookie=287d75d0e78eae3f in_seq=0 out_seq=0 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.505+0000 7f51ecc00700 10 client.25345 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.506+0000 7f51ecc00700 1 -- 172.21.15.45:0/539362363 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1c2661810 con 0x55f1c271bea0 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.506+0000 7f51ecc00700 10 client.25345 handle_command_reply: tid=0 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.506+0000 7f5213ffbb80 1 client.25345 shutdown 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.506+0000 7f5213ffbb80 1 -- 172.21.15.45:0/539362363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 msgr2=0x55f1c270ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.506+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 0x55f1c270ba80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f51f4099ef0 tx=0x7f51f4009310 comp rx=0 tx=0).stop 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.507+0000 7f5213ffbb80 1 -- 172.21.15.45:0/539362363 shutdown_connections 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.507+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f1c271bea0 0x55f1c271e360 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.507+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27092c0 0x55f1c2709740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.507+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 0x55f1c270ba80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.507+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/539362363 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 0x55f1c2708d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.507+0000 7f5213ffbb80 1 -- 172.21.15.45:0/539362363 >> 172.21.15.45:0/539362363 conn(0x55f1c262f730 msgr2=0x55f1c26fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:27.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.507+0000 7f51ecc00700 0 client.25345 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:27.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.507+0000 7f5213ffbb80 1 -- 172.21.15.45:0/539362363 shutdown_connections 2024-04-25T19:00:27.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.507+0000 7f5213ffbb80 1 -- 172.21.15.45:0/539362363 wait complete. 2024-04-25T19:00:27.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.508+0000 7f5213ffbb80 20 client.25345 trim_cache size 0 max 16384 2024-04-25T19:00:27.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.529+0000 7f5213ffbb80 1 Processor -- start 2024-04-25T19:00:27.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.529+0000 7f5213ffbb80 1 -- start start 2024-04-25T19:00:27.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 0x55f1c2717c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 0x55f1c2704450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27092c0 0x55f1c2707d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51e400fef0 con 0x55f1c27092c0 2024-04-25T19:00:27.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51e400d780 con 0x55f1c27012d0 2024-04-25T19:00:27.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51e400b5d0 con 0x55f1c27009b0 2024-04-25T19:00:27.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5202000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 0x55f1c2704450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5202000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 0x55f1c2704450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48616/0 (socket says 172.21.15.45:48616) 2024-04-25T19:00:27.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5202000700 1 -- 172.21.15.45:0/1161482714 learned_addr learned my addr 172.21.15.45:0/1161482714 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:27.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5203400700 1 --2- 172.21.15.45:0/1161482714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27092c0 0x55f1c2707d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.530+0000 7f5202a00700 1 --2- 172.21.15.45:0/1161482714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 0x55f1c2717c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.531+0000 7f5202000700 1 -- 172.21.15.45:0/1161482714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 msgr2=0x55f1c2717c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:27.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.531+0000 7f5202000700 1 --2- 172.21.15.45:0/1161482714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 0x55f1c2717c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.531+0000 7f5202000700 1 -- 172.21.15.45:0/1161482714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27092c0 msgr2=0x55f1c2707d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:27.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.531+0000 7f5202000700 1 --2- 172.21.15.45:0/1161482714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27092c0 0x55f1c2707d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.531+0000 7f5202000700 1 -- 172.21.15.45:0/1161482714 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c27072f0 con 0x55f1c27012d0 2024-04-25T19:00:27.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.531+0000 7f5203400700 1 --2- 172.21.15.45:0/1161482714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27092c0 0x55f1c2707d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:27.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.532+0000 7f5202000700 1 --2- 172.21.15.45:0/1161482714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 0x55f1c2704450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f51f0013e60 tx=0x7f51f0012d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=880a8a1f37bc210c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:27.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.532+0000 7f51ef400700 1 -- 172.21.15.45:0/1161482714 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0009260 con 0x55f1c27012d0 2024-04-25T19:00:27.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.532+0000 7f51ef400700 1 -- 172.21.15.45:0/1161482714 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51f0011970 con 0x55f1c27012d0 2024-04-25T19:00:27.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.532+0000 7f51ef400700 1 -- 172.21.15.45:0/1161482714 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f00126b0 con 0x55f1c27012d0 2024-04-25T19:00:27.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.533+0000 7f5213ffbb80 1 -- 172.21.15.45:0/1161482714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 msgr2=0x55f1c2704450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:27.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.533+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/1161482714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 0x55f1c2704450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f51f0013e60 tx=0x7f51f0012d40 comp rx=0 tx=0).stop 2024-04-25T19:00:27.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.533+0000 7f5213ffbb80 1 -- 172.21.15.45:0/1161482714 shutdown_connections 2024-04-25T19:00:27.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.533+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/1161482714 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27092c0 0x55f1c2707d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.533+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/1161482714 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27009b0 0x55f1c2717c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.534 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.533+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/1161482714 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27012d0 0x55f1c2704450 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.533+0000 7f5213ffbb80 1 -- 172.21.15.45:0/1161482714 >> 172.21.15.45:0/1161482714 conn(0x55f1c262f730 msgr2=0x55f1c2707670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.533+0000 7f5213ffbb80 1 -- 172.21.15.45:0/1161482714 shutdown_connections 2024-04-25T19:00:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.533+0000 7f5213ffbb80 1 -- 172.21.15.45:0/1161482714 wait complete. 2024-04-25T19:00:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.534+0000 7f5213ffbb80 1 -- start start 2024-04-25T19:00:27.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.534+0000 7f5213ffbb80 1 Processor -- start 2024-04-25T19:00:27.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.534+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27009b0 0x55f1c2711110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.534+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27012d0 0x55f1c2711650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.534+0000 7f5213ffbb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27092c0 0x55f1c27824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.534+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51e400bc40 con 0x55f1c27012d0 2024-04-25T19:00:27.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.534+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51e400d0b0 con 0x55f1c27009b0 2024-04-25T19:00:27.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.534+0000 7f5213ffbb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51e400c310 con 0x55f1c27092c0 2024-04-25T19:00:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.534+0000 7f5213ffbb80 10 client.0 _ll_register_callbacks cb 0x55f1c26fd4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.535+0000 7f5202000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27012d0 0x55f1c2711650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.535+0000 7f5202000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27012d0 0x55f1c2711650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32880/0 (socket says 172.21.15.45:32880) 2024-04-25T19:00:27.537 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.535+0000 7f5202000700 1 -- 172.21.15.45:0/3810138872 learned_addr learned my addr 172.21.15.45:0/3810138872 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.535+0000 7f5202a00700 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27009b0 0x55f1c2711110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.535+0000 7f5203400700 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27092c0 0x55f1c27824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.536+0000 7f5202000700 1 -- 172.21.15.45:0/3810138872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27092c0 msgr2=0x55f1c27824f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.536+0000 7f5202000700 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27092c0 0x55f1c27824f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.536+0000 7f5202000700 1 -- 172.21.15.45:0/3810138872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27009b0 msgr2=0x55f1c2711110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.536+0000 7f5202000700 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27009b0 0x55f1c2711110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.536+0000 7f5202000700 1 -- 172.21.15.45:0/3810138872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c2711ef0 con 0x55f1c27012d0 2024-04-25T19:00:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.536+0000 7f5203400700 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27092c0 0x55f1c27824f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.536+0000 7f5202000700 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27012d0 0x55f1c2711650 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f51f001c040 tx=0x7f51f0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2275e32fc955a0e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:27.538 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.536+0000 7f51ecc00700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:00:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.537+0000 7f51ecc00700 1 -- 172.21.15.45:0/3810138872 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0011bb0 con 0x55f1c27012d0 2024-04-25T19:00:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.537+0000 7f51ecc00700 1 -- 172.21.15.45:0/3810138872 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f51f0005040 con 0x55f1c27012d0 2024-04-25T19:00:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.537+0000 7f5213ffbb80 1 -- 172.21.15.45:0/3810138872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f1c265b8b0 con 0x55f1c27012d0 2024-04-25T19:00:27.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.537+0000 7f51ecc00700 1 -- 172.21.15.45:0/3810138872 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f51f0011d50 con 0x55f1c27012d0 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.537+0000 7f5213ffbb80 10 client.15219 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.537+0000 7f5213ffbb80 1 -- 172.21.15.45:0/3810138872 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f1c2783ae0 con 0x55f1c27012d0 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.538+0000 7f51ecc00700 1 -- 172.21.15.45:0/3810138872 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f1c265b8b0 con 0x55f1c27012d0 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.538+0000 7f51ecc00700 1 -- 172.21.15.45:0/3810138872 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f51f0003680 con 0x55f1c27012d0 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.538+0000 7f5213ffbb80 10 client.15219 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.538+0000 7f5213ffbb80 10 client.15219 resolve_mds: validated gid 24524 aka daemon mds.b 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.538+0000 7f5213ffbb80 20 client.15219 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.538+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f1c2783f30 0x55f1c2786350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.538+0000 7f5213ffbb80 4 client.15219 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.4, "await_for": 42.0}] 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.538+0000 7f5213ffbb80 1 -- 172.21.15.45:0/3810138872 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.4, "await_for": 42.0}) -- 0x55f1c24d43e0 con 0x55f1c2783f30 2024-04-25T19:00:27.540 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.538+0000 7f5202a00700 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f1c2783f30 0x55f1c2786350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:27.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.540+0000 7f5202a00700 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f1c2783f30 0x55f1c2786350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c42d52b0666d5097 server_cookie=f53829641f6df093 in_seq=0 out_seq=0 2024-04-25T19:00:27.541 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.540+0000 7f51ecc00700 10 client.15219 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:27.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.549+0000 7f51ecc00700 1 -- 172.21.15.45:0/3810138872 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55f1c24d43e0 con 0x55f1c2783f30 2024-04-25T19:00:27.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.549+0000 7f51ecc00700 10 client.15219 handle_command_reply: tid=0 2024-04-25T19:00:27.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.549+0000 7f5213ffbb80 1 client.15219 shutdown 2024-04-25T19:00:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.550+0000 7f5213ffbb80 1 -- 172.21.15.45:0/3810138872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27012d0 msgr2=0x55f1c2711650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.550+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27012d0 0x55f1c2711650 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f51f001c040 tx=0x7f51f0009260 comp rx=0 tx=0).stop 2024-04-25T19:00:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.550+0000 7f5213ffbb80 1 -- 172.21.15.45:0/3810138872 shutdown_connections 2024-04-25T19:00:27.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.550+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f1c2783f30 0x55f1c2786350 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.550+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f1c27009b0 0x55f1c2711110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.550+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f1c27092c0 0x55f1c27824f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.550+0000 7f5213ffbb80 1 --2- 172.21.15.45:0/3810138872 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f1c27012d0 0x55f1c2711650 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.550+0000 7f5213ffbb80 1 -- 172.21.15.45:0/3810138872 >> 172.21.15.45:0/3810138872 conn(0x55f1c262f730 msgr2=0x55f1c2704290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.550+0000 7f51ecc00700 0 client.15219 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:27.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.550+0000 7f5213ffbb80 1 -- 172.21.15.45:0/3810138872 shutdown_connections 2024-04-25T19:00:27.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.554+0000 7f5213ffbb80 1 -- 172.21.15.45:0/3810138872 wait complete. 2024-04-25T19:00:27.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.554+0000 7f5213ffbb80 20 client.15219 trim_cache size 0 max 16384 2024-04-25T19:00:27.555 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:00:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 26, 2024-04-25T19:00:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24524, 2024-04-25T19:00:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 2, 2024-04-25T19:00:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-04-25T19:00:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "213d97e7": { 2024-04-25T19:00:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 2, 2024-04-25T19:00:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 60.0, 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 89.4, 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCED", 2024-04-25T19:00:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-04-25T19:00:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:00:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:00:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:00:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:00:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:00:27.558 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.555+0000 7f5209600700 1 -- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51e80a4100 msgr2=0x7f51e80a65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.555+0000 7f5209600700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51e80a4100 0x7f51e80a65c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009fd0 tx=0x7f51fc004b40 comp rx=0 tx=0).stop 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.555+0000 7f5209600700 1 -- 172.21.15.45:0/1675528769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520406a740 msgr2=0x7f520419fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.555+0000 7f5209600700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520406a740 0x7f520419fc00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51f4005910 tx=0x7f51f400b670 comp rx=0 tx=0).stop 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.555+0000 7f5209600700 1 -- 172.21.15.45:0/1675528769 shutdown_connections 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.555+0000 7f5209600700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51e80a4100 0x7f51e80a65c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.555+0000 7f5209600700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f520406a740 0x7f520419fc00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.555+0000 7f5209600700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5204108a60 0x7f520410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.555+0000 7f5209600700 1 --2- 172.21.15.45:0/1675528769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f520406b100 0x7f52041a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.555+0000 7f5209600700 1 -- 172.21.15.45:0/1675528769 >> 172.21.15.45:0/1675528769 conn(0x7f52040788e0 msgr2=0x7f5204100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.556+0000 7f5209600700 1 -- 172.21.15.45:0/1675528769 shutdown_connections 2024-04-25T19:00:27.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:00:27.556+0000 7f5209600700 1 -- 172.21.15.45:0/1675528769 wait complete. 2024-04-25T19:00:27.952 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:27 smithi045 ceph-mon[27776]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T19:00:27.952 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:27 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:00:28.274 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.273+0000 7f7407200640 1 -- 172.21.15.45:0/3253619890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f740810e620 msgr2=0x7f7408110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.273+0000 7f7407200640 1 --2- 172.21.15.45:0/3253619890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f740810e620 0x7f7408110a10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7400009be0 tx=0x7f740002a750 comp rx=0 tx=0).stop 2024-04-25T19:00:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.274+0000 7f7407200640 1 -- 172.21.15.45:0/3253619890 shutdown_connections 2024-04-25T19:00:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.274+0000 7f7407200640 1 --2- 172.21.15.45:0/3253619890 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f740810e620 0x7f7408110a10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.274+0000 7f7407200640 1 --2- 172.21.15.45:0/3253619890 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7408074c20 0x7f740810e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.274+0000 7f7407200640 1 --2- 172.21.15.45:0/3253619890 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7408074300 0x7f74080746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.275 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.274+0000 7f7407200640 1 -- 172.21.15.45:0/3253619890 >> 172.21.15.45:0/3253619890 conn(0x7f74080fd890 msgr2=0x7f74080ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.274+0000 7f7407200640 1 -- 172.21.15.45:0/3253619890 shutdown_connections 2024-04-25T19:00:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.274+0000 7f7407200640 1 -- 172.21.15.45:0/3253619890 wait complete. 2024-04-25T19:00:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.275+0000 7f7407200640 1 Processor -- start 2024-04-25T19:00:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.275+0000 7f7407200640 1 -- start start 2024-04-25T19:00:28.276 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.276+0000 7f7407200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7408074300 0x7f740810e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.276+0000 7f7407200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7408074c20 0x7f740810b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.276+0000 7f7405e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7408074300 0x7f740810e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.276+0000 7f7405e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7408074300 0x7f740810e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48644/0 (socket says 172.21.15.45:48644) 2024-04-25T19:00:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.276+0000 7f7405e00640 1 -- 172.21.15.45:0/2850138852 learned_addr learned my addr 172.21.15.45:0/2850138852 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.276+0000 7f7407200640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f740810e620 0x7f740810b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.277 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.276+0000 7f7405400640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7408074c20 0x7f740810b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.278 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.276+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74081141b0 con 0x7f740810e620 2024-04-25T19:00:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.277+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7408114030 con 0x7f7408074300 2024-04-25T19:00:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.277+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7408114330 con 0x7f7408074c20 2024-04-25T19:00:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.277+0000 7f7406800640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f740810e620 0x7f740810b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.277+0000 7f7405e00640 1 -- 172.21.15.45:0/2850138852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7408074c20 msgr2=0x7f740810b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.277+0000 7f7405e00640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7408074c20 0x7f740810b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.277+0000 7f7405e00640 1 -- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f740810e620 msgr2=0x7f740810b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.277+0000 7f7405e00640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f740810e620 0x7f740810b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.277+0000 7f7405e00640 1 -- 172.21.15.45:0/2850138852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74081aaef0 con 0x7f7408074300 2024-04-25T19:00:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.277+0000 7f7406800640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f740810e620 0x7f740810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:28.279 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.278+0000 7f7405e00640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7408074300 0x7f740810e0b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e9f0 tx=0x7f73f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.278+0000 7f73fea00640 1 -- 172.21.15.45:0/2850138852 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f800ccc0 con 0x7f7408074300 2024-04-25T19:00:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.278+0000 7f73fea00640 1 -- 172.21.15.45:0/2850138852 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73f800ce60 con 0x7f7408074300 2024-04-25T19:00:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.278+0000 7f73fea00640 1 -- 172.21.15.45:0/2850138852 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8010430 con 0x7f7408074300 2024-04-25T19:00:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.278+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74081ab120 con 0x7f7408074300 2024-04-25T19:00:28.280 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.278+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74081ab620 con 0x7f7408074300 2024-04-25T19:00:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.283+0000 7f740fdf7740 1 Processor -- start 2024-04-25T19:00:28.284 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.283+0000 7f740fdf7740 1 -- start start 2024-04-25T19:00:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.284+0000 7f740fdf7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 0x55e5bff4bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f740fdf7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 0x55e5bff56650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f740fdf7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff56b90 0x55e5bff58f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f740fdf7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e5bfeb1900 con 0x55e5bff4c2d0 2024-04-25T19:00:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f740fdf7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e5bfe57ef0 con 0x55e5bff4b9b0 2024-04-25T19:00:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f740fdf7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e5bfead8c0 con 0x55e5bff56b90 2024-04-25T19:00:28.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f7406800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 0x55e5bff4bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f7406800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 0x55e5bff4bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48654/0 (socket says 172.21.15.45:48654) 2024-04-25T19:00:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f7406800640 1 -- 172.21.15.45:0/2205188062 learned_addr learned my addr 172.21.15.45:0/2205188062 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f7405400640 1 --2- 172.21.15.45:0/2205188062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff56b90 0x55e5bff58f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f73fea00640 1 -- 172.21.15.45:0/2850138852 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7f73f80046c0 con 0x7f7408074300 2024-04-25T19:00:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f7406800640 1 -- 172.21.15.45:0/2205188062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff56b90 msgr2=0x55e5bff58f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:28.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.285+0000 7f7406800640 1 --2- 172.21.15.45:0/2205188062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff56b90 0x55e5bff58f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.286+0000 7f7405e00640 1 --2- 172.21.15.45:0/2205188062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 0x55e5bff56650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.286+0000 7f7406800640 1 -- 172.21.15.45:0/2205188062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 msgr2=0x55e5bff56650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.286+0000 7f7406800640 1 --2- 172.21.15.45:0/2205188062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 0x55e5bff56650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.287+0000 7f7406800640 1 -- 172.21.15.45:0/2205188062 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5bfbc4210 con 0x55e5bff4b9b0 2024-04-25T19:00:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.287+0000 7f7405e00640 1 --2- 172.21.15.45:0/2205188062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 0x55e5bff56650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:00:28.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.288+0000 7f7406800640 1 --2- 172.21.15.45:0/2205188062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 0x55e5bff4bd90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7400009be0 tx=0x7f74000307c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5adb8c9c9ca4479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:28.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.289+0000 7f73f7400640 1 -- 172.21.15.45:0/2205188062 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400041030 con 0x55e5bff4b9b0 2024-04-25T19:00:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.289+0000 7f73f7400640 1 -- 172.21.15.45:0/2205188062 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7400003e20 con 0x55e5bff4b9b0 2024-04-25T19:00:28.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.289+0000 7f73f7400640 1 -- 172.21.15.45:0/2205188062 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400036040 con 0x55e5bff4b9b0 2024-04-25T19:00:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.289+0000 7f73fea00640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73ec0a4000 0x7f73ec0a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.290+0000 7f7405e00640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73ec0a4000 0x7f73ec0a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.291+0000 7f73fea00640 1 -- 172.21.15.45:0/2850138852 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f73f80d0c90 con 0x7f7408074300 2024-04-25T19:00:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.292+0000 7f740fdf7740 1 -- 172.21.15.45:0/2205188062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 msgr2=0x55e5bff4bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.292+0000 7f740fdf7740 1 --2- 172.21.15.45:0/2205188062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 0x55e5bff4bd90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7400009be0 tx=0x7f74000307c0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.292+0000 7f740fdf7740 1 -- 172.21.15.45:0/2205188062 shutdown_connections 2024-04-25T19:00:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.292+0000 7f740fdf7740 1 --2- 172.21.15.45:0/2205188062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff56b90 0x55e5bff58f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.292+0000 7f740fdf7740 1 --2- 172.21.15.45:0/2205188062 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 0x55e5bff56650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.292+0000 7f740fdf7740 1 --2- 172.21.15.45:0/2205188062 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 0x55e5bff4bd90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.292+0000 7f740fdf7740 1 -- 172.21.15.45:0/2205188062 >> 172.21.15.45:0/2205188062 conn(0x55e5bff47ec0 msgr2=0x55e5bff482a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.292+0000 7f7405e00640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73ec0a4000 0x7f73ec0a64c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f73f80ce200 tx=0x7f73f8098560 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.293+0000 7f740fdf7740 1 -- 172.21.15.45:0/2205188062 shutdown_connections 2024-04-25T19:00:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.293+0000 7f740fdf7740 1 -- 172.21.15.45:0/2205188062 wait complete. 2024-04-25T19:00:28.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.293+0000 7f740fdf7740 1 -- start start 2024-04-25T19:00:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.294+0000 7f740fdf7740 1 Processor -- start 2024-04-25T19:00:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.294+0000 7f740fdf7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4b9b0 0x55e5bff53b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.294+0000 7f740fdf7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 0x55e5bff540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.295+0000 7f7406800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4b9b0 0x55e5bff53b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.295+0000 7f7406800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4b9b0 0x55e5bff53b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32908/0 (socket says 172.21.15.45:32908) 2024-04-25T19:00:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.295+0000 7f740fdf7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff56b90 0x55e5bff54600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.295+0000 7f7406800640 1 -- 172.21.15.45:0/3338304094 learned_addr learned my addr 172.21.15.45:0/3338304094 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:28.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.295+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e5bff59760 con 0x55e5bff4c2d0 2024-04-25T19:00:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.295+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e5bfe6b310 con 0x55e5bff56b90 2024-04-25T19:00:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.295+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e5bff598e0 con 0x55e5bff4b9b0 2024-04-25T19:00:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.295+0000 7f740fdf7740 10 client.0 _ll_register_callbacks cb 0x55e5bfe39cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.296+0000 7f7405400640 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 0x55e5bff540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.296+0000 7f7406800640 1 -- 172.21.15.45:0/3338304094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff56b90 msgr2=0x55e5bff54600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.296+0000 7f7406800640 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff56b90 0x55e5bff54600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.296+0000 7f7406800640 1 -- 172.21.15.45:0/3338304094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 msgr2=0x55e5bff540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:28.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.296+0000 7f7406800640 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 0x55e5bff540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.296+0000 7f7406800640 1 -- 172.21.15.45:0/3338304094 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5bfc90620 con 0x55e5bff4b9b0 2024-04-25T19:00:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.297+0000 7f73ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:00:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.297+0000 7f7406800640 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4b9b0 0x55e5bff53b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7400006010 tx=0x7f7400009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca3cf4d6a6ebe970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:28.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.297+0000 7f73ebe00640 1 -- 172.21.15.45:0/3338304094 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400036050 con 0x55e5bff4b9b0 2024-04-25T19:00:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.297+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7400003e20 con 0x55e5bff4b9b0 2024-04-25T19:00:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.298+0000 7f73ebe00640 1 -- 172.21.15.45:0/3338304094 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f7400030a30 con 0x55e5bff4b9b0 2024-04-25T19:00:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.298+0000 7f73ebe00640 1 -- 172.21.15.45:0/3338304094 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7400036050 con 0x55e5bff4b9b0 2024-04-25T19:00:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.298+0000 7f740fdf7740 10 client.25363 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:28.299 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.298+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5bff59760 con 0x55e5bff4b9b0 2024-04-25T19:00:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.299+0000 7f73ebe00640 1 -- 172.21.15.45:0/3338304094 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7400003e20 con 0x55e5bff4b9b0 2024-04-25T19:00:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.299+0000 7f73ebe00640 1 -- 172.21.15.45:0/3338304094 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f740003d460 con 0x55e5bff4b9b0 2024-04-25T19:00:28.300 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.299+0000 7f740fdf7740 10 client.25363 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.299+0000 7f740fdf7740 10 client.25363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:00:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.299+0000 7f740fdf7740 20 client.25363 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.299+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e5bff66ef0 0x55e5bff69370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.299+0000 7f740fdf7740 4 client.25363 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:00:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.299+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5bfec58b0 con 0x55e5bff66ef0 2024-04-25T19:00:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.300+0000 7f7405400640 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e5bff66ef0 0x55e5bff69370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.300+0000 7f73ebe00640 10 client.25363 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:28.301 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.300+0000 7f7405400640 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e5bff66ef0 0x55e5bff69370 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8eea1d691ce5663 server_cookie=fc227ce507e51df5 in_seq=0 out_seq=0 2024-04-25T19:00:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.301+0000 7f73ebe00640 1 -- 172.21.15.45:0/3338304094 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e5bfec58b0 con 0x55e5bff66ef0 2024-04-25T19:00:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.301+0000 7f73ebe00640 10 client.25363 handle_command_reply: tid=0 2024-04-25T19:00:28.302 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.302+0000 7f740fdf7740 1 client.25363 shutdown 2024-04-25T19:00:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.302+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4b9b0 msgr2=0x55e5bff53b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.302+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4b9b0 0x55e5bff53b80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7400006010 tx=0x7f7400009800 comp rx=0 tx=0).stop 2024-04-25T19:00:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.302+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 shutdown_connections 2024-04-25T19:00:28.303 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.302+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e5bff66ef0 0x55e5bff69370 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.302+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff56b90 0x55e5bff54600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.302+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4c2d0 0x55e5bff540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.302+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3338304094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4b9b0 0x55e5bff53b80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.302+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 >> 172.21.15.45:0/3338304094 conn(0x55e5bff47ec0 msgr2=0x55e5bff4dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.302+0000 7f73ebe00640 0 client.25363 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.303+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 shutdown_connections 2024-04-25T19:00:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.303+0000 7f740fdf7740 1 -- 172.21.15.45:0/3338304094 wait complete. 2024-04-25T19:00:28.304 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.303+0000 7f740fdf7740 20 client.25363 trim_cache size 0 max 16384 2024-04-25T19:00:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.337+0000 7f740fdf7740 1 Processor -- start 2024-04-25T19:00:28.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.337+0000 7f740fdf7740 1 -- start start 2024-04-25T19:00:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f740fdf7740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 0x55e5bff4e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f740fdf7740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f740fdf7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff56b90 0x55e5bff6eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f740fdf7740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b69e0 con 0x55e5bff56b90 2024-04-25T19:00:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f740fdf7740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b42b0 con 0x55e5bff4b9b0 2024-04-25T19:00:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f740fdf7740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b20a0 con 0x55e5bff4c2d0 2024-04-25T19:00:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f7405400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f7405400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:32912/0 (socket says 172.21.15.45:32912) 2024-04-25T19:00:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f7405400640 1 -- 172.21.15.45:0/949581483 learned_addr learned my addr 172.21.15.45:0/949581483 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:28.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f7406800640 1 --2- 172.21.15.45:0/949581483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 0x55e5bff4e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f7405400640 1 -- 172.21.15.45:0/949581483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 msgr2=0x55e5bff4e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f7405400640 1 --2- 172.21.15.45:0/949581483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 0x55e5bff4e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f7405400640 1 -- 172.21.15.45:0/949581483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff56b90 msgr2=0x55e5bff6eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.338+0000 7f7405400640 1 --2- 172.21.15.45:0/949581483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff56b90 0x55e5bff6eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.339+0000 7f7405400640 1 -- 172.21.15.45:0/949581483 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5bff520d0 con 0x55e5bff4c2d0 2024-04-25T19:00:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.339+0000 7f7405400640 1 --2- 172.21.15.45:0/949581483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6e980 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f740810c9a0 tx=0x7f73f0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fb38a7a09c31915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.339+0000 7f73f6a00640 1 -- 172.21.15.45:0/949581483 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0024070 con 0x55e5bff4c2d0 2024-04-25T19:00:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.339+0000 7f73f6a00640 1 -- 172.21.15.45:0/949581483 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e5bfec58b0 con 0x55e5bff4c2d0 2024-04-25T19:00:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.339+0000 7f73f6a00640 1 -- 172.21.15.45:0/949581483 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f001f710 con 0x55e5bff4c2d0 2024-04-25T19:00:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.340+0000 7f740fdf7740 1 -- 172.21.15.45:0/949581483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4c2d0 msgr2=0x55e5bff6e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.340+0000 7f740fdf7740 1 --2- 172.21.15.45:0/949581483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6e980 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f740810c9a0 tx=0x7f73f0005f90 comp rx=0 tx=0).stop 2024-04-25T19:00:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.341+0000 7f740fdf7740 1 -- 172.21.15.45:0/949581483 shutdown_connections 2024-04-25T19:00:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.341+0000 7f740fdf7740 1 --2- 172.21.15.45:0/949581483 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff56b90 0x55e5bff6eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.341+0000 7f740fdf7740 1 --2- 172.21.15.45:0/949581483 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6e980 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.341+0000 7f740fdf7740 1 --2- 172.21.15.45:0/949581483 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4b9b0 0x55e5bff4e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.341+0000 7f740fdf7740 1 -- 172.21.15.45:0/949581483 >> 172.21.15.45:0/949581483 conn(0x55e5bff47ec0 msgr2=0x55e5bff4ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.341+0000 7f740fdf7740 1 -- 172.21.15.45:0/949581483 shutdown_connections 2024-04-25T19:00:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.341+0000 7f740fdf7740 1 -- 172.21.15.45:0/949581483 wait complete. 2024-04-25T19:00:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.341+0000 7f740fdf7740 1 -- start start 2024-04-25T19:00:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.342+0000 7f740fdf7740 1 Processor -- start 2024-04-25T19:00:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.343+0000 7f740fdf7740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4b9b0 0x55e5bff6d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.344+0000 7f7406800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4b9b0 0x55e5bff6d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.344+0000 7f7406800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4b9b0 0x55e5bff6d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:32908/0 (socket says 172.21.15.45:32908) 2024-04-25T19:00:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.344+0000 7f7406800640 1 -- 172.21.15.45:0/3510332298 learned_addr learned my addr 172.21.15.45:0/3510332298 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.344+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.344+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff56b90 0x55e5bff6e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.345+0000 7f7405400640 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.345+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b2770 con 0x55e5bff4b9b0 2024-04-25T19:00:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.345+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b3be0 con 0x55e5bff4c2d0 2024-04-25T19:00:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.345+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f74081b2e40 con 0x55e5bff56b90 2024-04-25T19:00:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.345+0000 7f740fdf7740 10 client.0 _ll_register_callbacks cb 0x7f7400030da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.346+0000 7f7406800640 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff56b90 0x55e5bff6e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.347+0000 7f7406800640 1 -- 172.21.15.45:0/3510332298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff56b90 msgr2=0x55e5bff6e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.347+0000 7f7406800640 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff56b90 0x55e5bff6e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.347+0000 7f7406800640 1 -- 172.21.15.45:0/3510332298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4c2d0 msgr2=0x55e5bff6db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.347+0000 7f7406800640 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.347+0000 7f7406800640 1 -- 172.21.15.45:0/3510332298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5bffcb600 con 0x55e5bff4b9b0 2024-04-25T19:00:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.347+0000 7f7405400640 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.348+0000 7f7406800640 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4b9b0 0x55e5bff6d5e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f740003db00 tx=0x7f740002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbafe9e345c2732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:28.349 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.348+0000 7f73ebe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:00:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.349+0000 7f73ebe00640 1 -- 172.21.15.45:0/3510332298 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400041040 con 0x55e5bff4b9b0 2024-04-25T19:00:28.350 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.349+0000 7f73ebe00640 1 -- 172.21.15.45:0/3510332298 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f7400003b80 con 0x55e5bff4b9b0 2024-04-25T19:00:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.350+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e5bfec58b0 con 0x55e5bff4b9b0 2024-04-25T19:00:28.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.350+0000 7f73ebe00640 1 -- 172.21.15.45:0/3510332298 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7400003d80 con 0x55e5bff4b9b0 2024-04-25T19:00:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.351+0000 7f740fdf7740 10 client.15228 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.351+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e5bffcc970 con 0x55e5bff4b9b0 2024-04-25T19:00:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.351+0000 7f73ebe00640 1 -- 172.21.15.45:0/3510332298 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7400062070 con 0x55e5bff4b9b0 2024-04-25T19:00:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.351+0000 7f73ebe00640 1 -- 172.21.15.45:0/3510332298 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f74000314d0 con 0x55e5bff4b9b0 2024-04-25T19:00:28.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.352+0000 7f740fdf7740 10 client.15228 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.352+0000 7f740fdf7740 10 client.15228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:00:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.352+0000 7f740fdf7740 20 client.15228 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.352+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e5bffccff0 0x55e5bffcf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.352+0000 7f740fdf7740 4 client.15228 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:00:28.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.352+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5bfebf320 con 0x55e5bffccff0 2024-04-25T19:00:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.353+0000 7f7405400640 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e5bffccff0 0x55e5bffcf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.353+0000 7f7405400640 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e5bffccff0 0x55e5bffcf410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98981cd2f2bab60f server_cookie=9f4e9c3b46ae0aa7 in_seq=0 out_seq=0 2024-04-25T19:00:28.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.353+0000 7f73ebe00640 10 client.15228 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.362+0000 7f73ebe00640 1 -- 172.21.15.45:0/3510332298 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5bfebf320 con 0x55e5bffccff0 2024-04-25T19:00:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.362+0000 7f73ebe00640 10 client.15228 handle_command_reply: tid=0 2024-04-25T19:00:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.362+0000 7f740fdf7740 1 client.15228 shutdown 2024-04-25T19:00:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.362+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4b9b0 msgr2=0x55e5bff6d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.362+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4b9b0 0x55e5bff6d5e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f740003db00 tx=0x7f740002a750 comp rx=0 tx=0).stop 2024-04-25T19:00:28.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:27 smithi062 ceph-mon[32844]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T19:00:28.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:27 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:00:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.363+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 shutdown_connections 2024-04-25T19:00:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.363+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e5bffccff0 0x55e5bffcf410 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.363+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e5bff56b90 0x55e5bff6e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.363+0000 7f73ebe00640 0 client.15228 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.363+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e5bff4c2d0 0x55e5bff6db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.363+0000 7f740fdf7740 1 --2- 172.21.15.45:0/3510332298 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e5bff4b9b0 0x55e5bff6d5e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.363+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 >> 172.21.15.45:0/3510332298 conn(0x55e5bff47ec0 msgr2=0x55e5bff61b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.364+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 shutdown_connections 2024-04-25T19:00:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.364+0000 7f740fdf7740 1 -- 172.21.15.45:0/3510332298 wait complete. 2024-04-25T19:00:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.364+0000 7f740fdf7740 20 client.15228 trim_cache size 0 max 16384 2024-04-25T19:00:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:00:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:00:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:00:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:00:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.365+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73ec0a4000 msgr2=0x7f73ec0a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.365+0000 7f7407200640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73ec0a4000 0x7f73ec0a64c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f73f80ce200 tx=0x7f73f8098560 comp rx=0 tx=0).stop 2024-04-25T19:00:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.365+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7408074300 msgr2=0x7f740810e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.365+0000 7f7407200640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7408074300 0x7f740810e0b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73f800e9f0 tx=0x7f73f800eeb0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.365+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 shutdown_connections 2024-04-25T19:00:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.365+0000 7f7407200640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73ec0a4000 0x7f73ec0a64c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.365+0000 7f7407200640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f740810e620 0x7f740810b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.365+0000 7f7407200640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7408074c20 0x7f740810b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.365+0000 7f7407200640 1 --2- 172.21.15.45:0/2850138852 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7408074300 0x7f740810e0b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:28.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.365+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 >> 172.21.15.45:0/2850138852 conn(0x7f74080fd890 msgr2=0x7f74080fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.366+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 shutdown_connections 2024-04-25T19:00:28.368 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:28.366+0000 7f7407200640 1 -- 172.21.15.45:0/2850138852 wait complete. 2024-04-25T19:00:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:27 smithi079 ceph-mon[33002]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-04-25T19:00:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:27 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:00:28.527 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '213d97e7', quiesce took 7.8 seconds. Will release after: 29.4 2024-04-25T19:00:30.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:29 smithi045 ceph-mon[27776]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T19:00:30.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:29 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:00:30.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:29 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:00:30.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:29 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:00:30.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:29 smithi062 ceph-mon[32844]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T19:00:30.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:29 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:00:30.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:29 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:00:30.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:29 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:00:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:29 smithi079 ceph-mon[33002]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-25T19:00:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:29 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:00:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:29 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:00:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:29 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:00:32.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:31 smithi045 ceph-mon[27776]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-25T19:00:32.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:31 smithi062 ceph-mon[32844]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-25T19:00:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:31 smithi079 ceph-mon[33002]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-04-25T19:00:34.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:33 smithi045 ceph-mon[27776]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T19:00:34.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:33 smithi062 ceph-mon[32844]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T19:00:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:33 smithi079 ceph-mon[33002]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T19:00:36.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:35 smithi045 ceph-mon[27776]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T19:00:36.235 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:35 smithi079 ceph-mon[33002]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T19:00:36.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:35 smithi062 ceph-mon[32844]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-04-25T19:00:38.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:37 smithi062 ceph-mon[32844]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-25T19:00:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:37 smithi079 ceph-mon[33002]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-25T19:00:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:37 smithi045 ceph-mon[27776]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.5 KiB/s wr, 9 op/s 2024-04-25T19:00:40.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:39 smithi062 ceph-mon[32844]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T19:00:40.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:39 smithi079 ceph-mon[33002]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T19:00:40.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:39 smithi045 ceph-mon[27776]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T19:00:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.558+0000 7fbc4b200640 1 -- 172.21.15.45:0/1003774857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc441041f0 msgr2=0x7fbc441045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.558+0000 7fbc4b200640 1 --2- 172.21.15.45:0/1003774857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc441041f0 0x7fbc441045d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009900 tx=0x7fbc3802a410 comp rx=0 tx=0).stop 2024-04-25T19:00:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.558+0000 7fbc4b200640 1 -- 172.21.15.45:0/1003774857 shutdown_connections 2024-04-25T19:00:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.558+0000 7fbc4b200640 1 --2- 172.21.15.45:0/1003774857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc441092c0 0x7fbc44110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.558+0000 7fbc4b200640 1 --2- 172.21.15.45:0/1003774857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc44104ba0 0x7fbc44108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.558+0000 7fbc4b200640 1 --2- 172.21.15.45:0/1003774857 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc441041f0 0x7fbc441045d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.558+0000 7fbc4b200640 1 -- 172.21.15.45:0/1003774857 >> 172.21.15.45:0/1003774857 conn(0x7fbc440ffa50 msgr2=0x7fbc44101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.558+0000 7fbc4b200640 1 -- 172.21.15.45:0/1003774857 shutdown_connections 2024-04-25T19:00:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.558+0000 7fbc4b200640 1 -- 172.21.15.45:0/1003774857 wait complete. 2024-04-25T19:00:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.559+0000 7fbc4b200640 1 Processor -- start 2024-04-25T19:00:41.561 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.560+0000 7fbc4b200640 1 -- start start 2024-04-25T19:00:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.560+0000 7fbc4b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc441041f0 0x7fbc441a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.560+0000 7fbc4b200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc441092c0 0x7fbc441a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.560+0000 7fbc4b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc4419a5f0 0x7fbc4419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.560+0000 7fbc4b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc44114270 con 0x7fbc441041f0 2024-04-25T19:00:41.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.560+0000 7fbc4b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441140f0 con 0x7fbc4419a5f0 2024-04-25T19:00:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.560+0000 7fbc4b200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441143f0 con 0x7fbc441092c0 2024-04-25T19:00:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc441041f0 0x7fbc441a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc49400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc441092c0 0x7fbc441a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc441041f0 0x7fbc441a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37096/0 (socket says 172.21.15.45:37096) 2024-04-25T19:00:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc49e00640 1 -- 172.21.15.45:0/1028973412 learned_addr learned my addr 172.21.15.45:0/1028973412 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:41.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc4a800640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc4419a5f0 0x7fbc4419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc49e00640 1 -- 172.21.15.45:0/1028973412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc441092c0 msgr2=0x7fbc441a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc49e00640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc441092c0 0x7fbc441a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc49e00640 1 -- 172.21.15.45:0/1028973412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc4419a5f0 msgr2=0x7fbc4419aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc49e00640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc4419a5f0 0x7fbc4419aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc49e00640 1 -- 172.21.15.45:0/1028973412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc4419b110 con 0x7fbc441041f0 2024-04-25T19:00:41.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.561+0000 7fbc49400640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc441092c0 0x7fbc441a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.562+0000 7fbc49e00640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc441041f0 0x7fbc441a0440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbc38005ab0 tx=0x7fbc38003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.562+0000 7fbc3ea00640 1 -- 172.21.15.45:0/1028973412 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3803f070 con 0x7fbc441041f0 2024-04-25T19:00:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.562+0000 7fbc3ea00640 1 -- 172.21.15.45:0/1028973412 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc38031b60 con 0x7fbc441041f0 2024-04-25T19:00:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.562+0000 7fbc3ea00640 1 -- 172.21.15.45:0/1028973412 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc38035030 con 0x7fbc441041f0 2024-04-25T19:00:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.562+0000 7fbc4b200640 1 -- 172.21.15.45:0/1028973412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc4419b3a0 con 0x7fbc441041f0 2024-04-25T19:00:41.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.563+0000 7fbc4b200640 1 -- 172.21.15.45:0/1028973412 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc441a7540 con 0x7fbc441041f0 2024-04-25T19:00:41.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.566+0000 7fbc4f576740 1 Processor -- start 2024-04-25T19:00:41.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.568+0000 7fbc4f576740 1 -- start start 2024-04-25T19:00:41.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.568+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 0x558c7e6305a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.568+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e630ae0 0x558c7e63ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.568+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e63d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.568+0000 7fbc4f576740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558c7e58d900 con 0x558c7e630ae0 2024-04-25T19:00:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.568+0000 7fbc4f576740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558c7e533ef0 con 0x558c7e6301c0 2024-04-25T19:00:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.568+0000 7fbc4f576740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558c7e5898c0 con 0x558c7e63b3a0 2024-04-25T19:00:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.568+0000 7fbc49400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e63d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.569+0000 7fbc4a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 0x558c7e6305a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.569+0000 7fbc49400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e63d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:37618/0 (socket says 172.21.15.45:37618) 2024-04-25T19:00:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.569+0000 7fbc49400640 1 -- 172.21.15.45:0/3897917981 learned_addr learned my addr 172.21.15.45:0/3897917981 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:41.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.569+0000 7fbc3ea00640 1 -- 172.21.15.45:0/1028973412 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138930+0+0 (secure 0 0 0) 0x7fbc38042070 con 0x7fbc441041f0 2024-04-25T19:00:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.569+0000 7fbc49400640 1 -- 172.21.15.45:0/3897917981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 msgr2=0x558c7e6305a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.569+0000 7fbc49400640 1 --2- 172.21.15.45:0/3897917981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 0x558c7e6305a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.569+0000 7fbc49400640 1 -- 172.21.15.45:0/3897917981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e630ae0 msgr2=0x558c7e63ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.569+0000 7fbc49400640 1 --2- 172.21.15.45:0/3897917981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e630ae0 0x558c7e63ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.569+0000 7fbc49400640 1 -- 172.21.15.45:0/3897917981 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c7e2a0210 con 0x558c7e63b3a0 2024-04-25T19:00:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.569+0000 7fbc49400640 1 --2- 172.21.15.45:0/3897917981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e63d790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbc34011470 tx=0x7fbc34011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a4c6a67548a2575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.570+0000 7fbc33000640 1 -- 172.21.15.45:0/3897917981 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3401e070 con 0x558c7e63b3a0 2024-04-25T19:00:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.570+0000 7fbc33000640 1 -- 172.21.15.45:0/3897917981 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc34003aa0 con 0x558c7e63b3a0 2024-04-25T19:00:41.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.570+0000 7fbc33000640 1 -- 172.21.15.45:0/3897917981 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400fbe0 con 0x558c7e63b3a0 2024-04-25T19:00:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.571+0000 7fbc4f576740 1 -- 172.21.15.45:0/3897917981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 msgr2=0x558c7e63d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.571+0000 7fbc4f576740 1 --2- 172.21.15.45:0/3897917981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e63d790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbc34011470 tx=0x7fbc34011930 comp rx=0 tx=0).stop 2024-04-25T19:00:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.571+0000 7fbc4f576740 1 -- 172.21.15.45:0/3897917981 shutdown_connections 2024-04-25T19:00:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.571+0000 7fbc4f576740 1 --2- 172.21.15.45:0/3897917981 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e63d790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.571+0000 7fbc4f576740 1 --2- 172.21.15.45:0/3897917981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e630ae0 0x558c7e63ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.571+0000 7fbc4f576740 1 --2- 172.21.15.45:0/3897917981 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 0x558c7e6305a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.571+0000 7fbc4f576740 1 -- 172.21.15.45:0/3897917981 >> 172.21.15.45:0/3897917981 conn(0x558c7e62c6d0 msgr2=0x558c7e62cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.571+0000 7fbc4f576740 1 -- 172.21.15.45:0/3897917981 shutdown_connections 2024-04-25T19:00:41.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.571+0000 7fbc4f576740 1 -- 172.21.15.45:0/3897917981 wait complete. 2024-04-25T19:00:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.571+0000 7fbc4f576740 1 -- start start 2024-04-25T19:00:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.572+0000 7fbc4f576740 1 Processor -- start 2024-04-25T19:00:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.572+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e6301c0 0x558c7e6383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.572+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e630ae0 0x558c7e638900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.572+0000 7fbc3ea00640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc2c0a3ec0 0x7fbc2c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.572+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e63b3a0 0x558c7e638e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.572+0000 7fbc4f576740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x558c7e63dfd0 con 0x558c7e6301c0 2024-04-25T19:00:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.572+0000 7fbc4f576740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x558c7e547310 con 0x558c7e63b3a0 2024-04-25T19:00:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.572+0000 7fbc4f576740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x558c7e63e150 con 0x558c7e630ae0 2024-04-25T19:00:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.572+0000 7fbc4f576740 10 client.0 _ll_register_callbacks cb 0x558c7e515cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.572+0000 7fbc4a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e6301c0 0x558c7e6383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.573+0000 7fbc4a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e6301c0 0x558c7e6383c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:37114/0 (socket says 172.21.15.45:37114) 2024-04-25T19:00:41.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.573+0000 7fbc4a800640 1 -- 172.21.15.45:0/7265939 learned_addr learned my addr 172.21.15.45:0/7265939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.573+0000 7fbc49e00640 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e630ae0 0x558c7e638900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.573+0000 7fbc4a800640 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e63b3a0 0x558c7e638e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.573+0000 7fbc4a800640 1 -- 172.21.15.45:0/7265939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e630ae0 msgr2=0x558c7e638900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.573+0000 7fbc4a800640 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e630ae0 0x558c7e638900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.573+0000 7fbc4a800640 1 -- 172.21.15.45:0/7265939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e63b3a0 msgr2=0x558c7e638e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.573+0000 7fbc4a800640 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e63b3a0 0x558c7e638e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.573+0000 7fbc4a800640 1 -- 172.21.15.45:0/7265939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c7e36c620 con 0x558c7e6301c0 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.573+0000 7fbc3ea00640 1 -- 172.21.15.45:0/1028973412 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fbc380f10b0 con 0x7fbc441041f0 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.574+0000 7fbc4a800640 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e6301c0 0x558c7e6383c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbc40005c40 tx=0x7fbc40005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5591877834a06d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.574+0000 7fbc49400640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc2c0a3ec0 0x7fbc2c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.574+0000 7fbc27400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.574+0000 7fbc27400640 1 -- 172.21.15.45:0/7265939 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc40012110 con 0x558c7e6301c0 2024-04-25T19:00:41.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.574+0000 7fbc27400640 1 -- 172.21.15.45:0/7265939 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fbc40003970 con 0x558c7e6301c0 2024-04-25T19:00:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.574+0000 7fbc4f576740 1 -- 172.21.15.45:0/7265939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc34003aa0 con 0x558c7e6301c0 2024-04-25T19:00:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.575+0000 7fbc27400640 1 -- 172.21.15.45:0/7265939 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc40026050 con 0x558c7e6301c0 2024-04-25T19:00:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.575+0000 7fbc4f576740 10 client.15237 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.575+0000 7fbc4f576740 1 -- 172.21.15.45:0/7265939 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c7e64a2f0 con 0x558c7e6301c0 2024-04-25T19:00:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.575+0000 7fbc49400640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc2c0a3ec0 0x7fbc2c0a6380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009da0 tx=0x7fbc34005850 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.575+0000 7fbc27400640 1 -- 172.21.15.45:0/7265939 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc34003aa0 con 0x558c7e6301c0 2024-04-25T19:00:41.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.575+0000 7fbc27400640 1 -- 172.21.15.45:0/7265939 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbc40003b90 con 0x558c7e6301c0 2024-04-25T19:00:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.576+0000 7fbc4f576740 10 client.15237 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.576+0000 7fbc4f576740 10 client.15237 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:00:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.576+0000 7fbc4f576740 20 client.15237 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.576+0000 7fbc4f576740 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x558c7e64b7b0 0x558c7e64dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.576+0000 7fbc4f576740 4 client.15237 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:00:41.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.576+0000 7fbc4f576740 1 -- 172.21.15.45:0/7265939 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c7e5a18b0 con 0x558c7e64b7b0 2024-04-25T19:00:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.576+0000 7fbc49e00640 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x558c7e64b7b0 0x558c7e64dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.577+0000 7fbc49e00640 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x558c7e64b7b0 0x558c7e64dba0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5241cd5573563ce server_cookie=e40554132393c285 in_seq=0 out_seq=0 2024-04-25T19:00:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.577+0000 7fbc27400640 10 client.15237 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc27400640 1 -- 172.21.15.45:0/7265939 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558c7e5a18b0 con 0x558c7e64b7b0 2024-04-25T19:00:41.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc27400640 10 client.15237 handle_command_reply: tid=0 2024-04-25T19:00:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc4f576740 1 client.15237 shutdown 2024-04-25T19:00:41.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc4f576740 1 -- 172.21.15.45:0/7265939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e6301c0 msgr2=0x558c7e6383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc4f576740 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e6301c0 0x558c7e6383c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbc40005c40 tx=0x7fbc40005bd0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc4f576740 1 -- 172.21.15.45:0/7265939 shutdown_connections 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc4f576740 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x558c7e64b7b0 0x558c7e64dba0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc4f576740 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e63b3a0 0x558c7e638e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc4f576740 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e630ae0 0x558c7e638900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc4f576740 1 --2- 172.21.15.45:0/7265939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e6301c0 0x558c7e6383c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc4f576740 1 -- 172.21.15.45:0/7265939 >> 172.21.15.45:0/7265939 conn(0x558c7e62c6d0 msgr2=0x558c7e632450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.578+0000 7fbc27400640 0 client.15237 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.579+0000 7fbc4f576740 1 -- 172.21.15.45:0/7265939 shutdown_connections 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.579+0000 7fbc4f576740 1 -- 172.21.15.45:0/7265939 wait complete. 2024-04-25T19:00:41.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.579+0000 7fbc4f576740 20 client.15237 trim_cache size 0 max 16384 2024-04-25T19:00:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.597+0000 7fbc4f576740 1 Processor -- start 2024-04-25T19:00:41.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.597+0000 7fbc4f576740 1 -- start start 2024-04-25T19:00:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 0x558c7e646e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e630ae0 0x558c7e632bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e653460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc4f576740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441ae100 con 0x558c7e630ae0 2024-04-25T19:00:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc4f576740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441ab9d0 con 0x558c7e6301c0 2024-04-25T19:00:41.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc4f576740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441a9840 con 0x558c7e63b3a0 2024-04-25T19:00:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 0x558c7e646e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 0x558c7e646e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:54450/0 (socket says 172.21.15.45:54450) 2024-04-25T19:00:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc49e00640 1 -- 172.21.15.45:0/3008877674 learned_addr learned my addr 172.21.15.45:0/3008877674 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc4a800640 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e653460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc49400640 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e630ae0 0x558c7e632bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc49e00640 1 -- 172.21.15.45:0/3008877674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 msgr2=0x558c7e653460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc49e00640 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e653460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc49e00640 1 -- 172.21.15.45:0/3008877674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e630ae0 msgr2=0x558c7e632bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.598+0000 7fbc49e00640 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e630ae0 0x558c7e632bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.599+0000 7fbc49e00640 1 -- 172.21.15.45:0/3008877674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c7e63db00 con 0x558c7e6301c0 2024-04-25T19:00:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.599+0000 7fbc4a800640 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e653460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.599+0000 7fbc49400640 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e630ae0 0x558c7e632bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:00:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.599+0000 7fbc49e00640 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 0x558c7e646e90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbc380bcda0 tx=0x7fbc380dd040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b9810194a3211d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.599+0000 7fbc32600640 1 -- 172.21.15.45:0/3008877674 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc380bce10 con 0x558c7e6301c0 2024-04-25T19:00:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.599+0000 7fbc32600640 1 -- 172.21.15.45:0/3008877674 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc380c11c0 con 0x558c7e6301c0 2024-04-25T19:00:41.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.599+0000 7fbc32600640 1 -- 172.21.15.45:0/3008877674 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc380b9480 con 0x558c7e6301c0 2024-04-25T19:00:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.600+0000 7fbc4f576740 1 -- 172.21.15.45:0/3008877674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 msgr2=0x558c7e646e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.600+0000 7fbc4f576740 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 0x558c7e646e90 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbc380bcda0 tx=0x7fbc380dd040 comp rx=0 tx=0).stop 2024-04-25T19:00:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.600+0000 7fbc4f576740 1 -- 172.21.15.45:0/3008877674 shutdown_connections 2024-04-25T19:00:41.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.600+0000 7fbc4f576740 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e63b3a0 0x558c7e653460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.600+0000 7fbc4f576740 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e630ae0 0x558c7e632bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.600+0000 7fbc4f576740 1 --2- 172.21.15.45:0/3008877674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e6301c0 0x558c7e646e90 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.600+0000 7fbc4f576740 1 -- 172.21.15.45:0/3008877674 >> 172.21.15.45:0/3008877674 conn(0x558c7e62c6d0 msgr2=0x558c7e6336c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.600+0000 7fbc4f576740 1 -- 172.21.15.45:0/3008877674 shutdown_connections 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.600+0000 7fbc4f576740 1 -- 172.21.15.45:0/3008877674 wait complete. 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.601+0000 7fbc4f576740 1 -- start start 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.601+0000 7fbc4f576740 1 Processor -- start 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.601+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e6301c0 0x558c7e643410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.601+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e630ae0 0x558c7e641f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.601+0000 7fbc4f576740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e63b3a0 0x558c7e6424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.601+0000 7fbc4f576740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441a9e90 con 0x558c7e63b3a0 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.601+0000 7fbc4f576740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441ab300 con 0x558c7e630ae0 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.601+0000 7fbc4f576740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbc441aa560 con 0x558c7e6301c0 2024-04-25T19:00:41.604 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc4f576740 10 client.0 _ll_register_callbacks cb 0x7fbc4000f3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:41.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc49400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e630ae0 0x558c7e641f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e6301c0 0x558c7e643410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc4a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e63b3a0 0x558c7e6424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc49400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e630ae0 0x558c7e641f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:54458/0 (socket says 172.21.15.45:54458) 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc49e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e6301c0 0x558c7e643410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:37642/0 (socket says 172.21.15.45:37642) 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc49400640 1 -- 172.21.15.45:0/2140082584 learned_addr learned my addr 172.21.15.45:0/2140082584 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc49400640 1 -- 172.21.15.45:0/2140082584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e6301c0 msgr2=0x558c7e643410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc49400640 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e6301c0 0x558c7e643410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc49400640 1 -- 172.21.15.45:0/2140082584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e63b3a0 msgr2=0x558c7e6424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc49400640 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e63b3a0 0x558c7e6424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc49400640 1 -- 172.21.15.45:0/2140082584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c7e6429f0 con 0x558c7e630ae0 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.602+0000 7fbc4a800640 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e63b3a0 0x558c7e6424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:41.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.603+0000 7fbc49400640 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e630ae0 0x558c7e641f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400f6b0 tx=0x7fbc3400f780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c602ac619aff1448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.603+0000 7fbc27400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:00:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.603+0000 7fbc27400640 1 -- 172.21.15.45:0/2140082584 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc34034020 con 0x558c7e630ae0 2024-04-25T19:00:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.603+0000 7fbc4f576740 1 -- 172.21.15.45:0/2140082584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc380c11c0 con 0x558c7e630ae0 2024-04-25T19:00:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.603+0000 7fbc27400640 1 -- 172.21.15.45:0/2140082584 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fbc34004bd0 con 0x558c7e630ae0 2024-04-25T19:00:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.604+0000 7fbc27400640 1 -- 172.21.15.45:0/2140082584 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbc3401e040 con 0x558c7e630ae0 2024-04-25T19:00:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.604+0000 7fbc4f576740 10 client.25370 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:41.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.604+0000 7fbc4f576740 1 -- 172.21.15.45:0/2140082584 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558c7e6b19f0 con 0x558c7e630ae0 2024-04-25T19:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.604+0000 7fbc27400640 1 -- 172.21.15.45:0/2140082584 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc380c11c0 con 0x558c7e630ae0 2024-04-25T19:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.604+0000 7fbc27400640 1 -- 172.21.15.45:0/2140082584 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbc34003c60 con 0x558c7e630ae0 2024-04-25T19:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.604+0000 7fbc4f576740 10 client.25370 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.604+0000 7fbc4f576740 10 client.25370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.604+0000 7fbc4f576740 20 client.25370 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.604+0000 7fbc4f576740 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x558c7e6b1ef0 0x558c7e6b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.604+0000 7fbc4f576740 4 client.25370 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.605+0000 7fbc4f576740 1 -- 172.21.15.45:0/2140082584 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558c7e59b320 con 0x558c7e6b1ef0 2024-04-25T19:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.605+0000 7fbc49e00640 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x558c7e6b1ef0 0x558c7e6b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:41.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.605+0000 7fbc49e00640 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x558c7e6b1ef0 0x558c7e6b4310 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b29761ff16ed25a server_cookie=56d3715d4c22fd30 in_seq=0 out_seq=0 2024-04-25T19:00:41.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.605+0000 7fbc27400640 10 client.25370 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.611+0000 7fbc27400640 1 -- 172.21.15.45:0/2140082584 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558c7e59b320 con 0x558c7e6b1ef0 2024-04-25T19:00:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.611+0000 7fbc27400640 10 client.25370 handle_command_reply: tid=0 2024-04-25T19:00:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.611+0000 7fbc4f576740 1 client.25370 shutdown 2024-04-25T19:00:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 1 -- 172.21.15.45:0/2140082584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e630ae0 msgr2=0x558c7e641f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e630ae0 0x558c7e641f70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400f6b0 tx=0x7fbc3400f780 comp rx=0 tx=0).stop 2024-04-25T19:00:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 1 -- 172.21.15.45:0/2140082584 shutdown_connections 2024-04-25T19:00:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x558c7e6b1ef0 0x558c7e6b4310 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x558c7e63b3a0 0x558c7e6424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x558c7e630ae0 0x558c7e641f70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 1 --2- 172.21.15.45:0/2140082584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x558c7e6301c0 0x558c7e643410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 1 -- 172.21.15.45:0/2140082584 >> 172.21.15.45:0/2140082584 conn(0x558c7e62c6d0 msgr2=0x558c7e638380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc27400640 0 client.25370 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 1 -- 172.21.15.45:0/2140082584 shutdown_connections 2024-04-25T19:00:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 1 -- 172.21.15.45:0/2140082584 wait complete. 2024-04-25T19:00:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.612+0000 7fbc4f576740 20 client.25370 trim_cache size 0 max 16384 2024-04-25T19:00:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:00:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:00:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:00:41.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:00:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 -- 172.21.15.45:0/1028973412 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc2c0a3ec0 msgr2=0x7fbc2c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc2c0a3ec0 0x7fbc2c0a6380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009da0 tx=0x7fbc34005850 comp rx=0 tx=0).stop 2024-04-25T19:00:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 -- 172.21.15.45:0/1028973412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc441041f0 msgr2=0x7fbc441a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc441041f0 0x7fbc441a0440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbc38005ab0 tx=0x7fbc38003c10 comp rx=0 tx=0).stop 2024-04-25T19:00:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 -- 172.21.15.45:0/1028973412 shutdown_connections 2024-04-25T19:00:41.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc2c0a3ec0 0x7fbc2c0a6380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc4419a5f0 0x7fbc4419aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc441092c0 0x7fbc441a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 --2- 172.21.15.45:0/1028973412 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc441041f0 0x7fbc441a0440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 -- 172.21.15.45:0/1028973412 >> 172.21.15.45:0/1028973412 conn(0x7fbc440ffa50 msgr2=0x7fbc44100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 -- 172.21.15.45:0/1028973412 shutdown_connections 2024-04-25T19:00:41.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:41.613+0000 7fbc4b200640 1 -- 172.21.15.45:0/1028973412 wait complete. 2024-04-25T19:00:42.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:41 smithi062 ceph-mon[32844]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T19:00:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:41 smithi079 ceph-mon[33002]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T19:00:42.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:41 smithi045 ceph-mon[27776]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T19:00:44.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:43 smithi062 ceph-mon[32844]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T19:00:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:43 smithi079 ceph-mon[33002]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T19:00:44.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:43 smithi045 ceph-mon[27776]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T19:00:46.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:45 smithi062 ceph-mon[32844]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T19:00:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:45 smithi079 ceph-mon[33002]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T19:00:46.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:45 smithi045 ceph-mon[27776]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T19:00:48.266 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:47 smithi045 ceph-mon[27776]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T19:00:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:47 smithi062 ceph-mon[32844]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T19:00:48.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:47 smithi079 ceph-mon[33002]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T19:00:50.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:49 smithi045 ceph-mon[27776]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T19:00:50.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:49 smithi062 ceph-mon[32844]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T19:00:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:49 smithi079 ceph-mon[33002]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T19:00:50.567 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 63% (7876/12492) Updating files: 64% (7995/12492) Updating files: 65% (8120/12492) Updating files: 66% (8245/12492) Updating files: 67% (8370/12492) Updating files: 68% (8495/12492) Updating files: 69% (8620/12492) Updating files: 70% (8745/12492) Updating files: 71% (8870/12492) Updating files: 72% (8995/12492) Updating files: 73% (9120/12492) Updating files: 74% (9245/12492) Updating files: 75% (9369/12492) Updating files: 76% (9494/12492) Updating files: 77% (9619/12492) Updating files: 78% (9744/12492) Updating files: 79% (9869/12492) Updating files: 80% (9994/12492) Updating files: 81% (10119/12492) Updating files: 82% (10244/12492) Updating files: 83% (10369/12492) Updating files: 84% (10494/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T19:00:50.975 INFO:tasks.workunit.client.0.smithi045.stderr:Note: switching to '7797745e6d8b3ac036504dc61491d368615024da'. 2024-04-25T19:00:50.975 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T19:00:50.975 INFO:tasks.workunit.client.0.smithi045.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr: git switch -c 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr:Or undo this operation with: 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr: git switch - 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr: 2024-04-25T19:00:50.976 INFO:tasks.workunit.client.0.smithi045.stderr:HEAD is now at 7797745e6d8 qa/suites/fs: add quiescer to the fs suite 2024-04-25T19:00:50.979 DEBUG:teuthology.orchestra.run.smithi045:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T19:00:51.042 INFO:tasks.workunit.client.0.smithi045.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T19:00:51.049 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T19:00:51.049 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T19:00:51.980 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T19:00:52.090 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T19:00:52.179 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T19:00:52.185 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T19:00:52.185 INFO:tasks.workunit.client.0.smithi045.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T19:00:52.275 INFO:tasks.workunit.client.0.smithi045.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T19:00:52.286 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T19:00:52.286 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T19:00:52.339 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-25T19:00:52.341 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-25T19:00:52.341 DEBUG:teuthology.orchestra.run.smithi045:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7797745e6d8b3ac036504dc61491d368615024da TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-25T19:00:52.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:52 smithi062 ceph-mon[32844]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-25T19:00:52.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:52 smithi062 ceph-mon[32844]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-25T19:00:52.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:52 smithi079 ceph-mon[33002]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-25T19:00:52.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:52 smithi079 ceph-mon[33002]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-25T19:00:52.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:52 smithi045 ceph-mon[27776]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-25T19:00:52.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:52 smithi045 ceph-mon[27776]: mgrmap e27: x(active, since 92s), standbys: z, y 2024-04-25T19:00:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:54 smithi062 ceph-mon[32844]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-25T19:00:54.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:54 smithi079 ceph-mon[33002]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-25T19:00:54.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:54 smithi045 ceph-mon[27776]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1023 B/s wr, 6 op/s 2024-04-25T19:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.761+0000 7f1cfd000640 1 -- 172.21.15.45:0/2531409693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf810c350 msgr2=0x7f1cf810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.761+0000 7f1cfd000640 1 --2- 172.21.15.45:0/2531409693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf810c350 0x7f1cf810e7f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009be0 tx=0x7f1cf002a750 comp rx=0 tx=0).stop 2024-04-25T19:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.762+0000 7f1cfd000640 1 -- 172.21.15.45:0/2531409693 shutdown_connections 2024-04-25T19:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.762+0000 7f1cfd000640 1 --2- 172.21.15.45:0/2531409693 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf810c350 0x7f1cf810e7f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.762+0000 7f1cfd000640 1 --2- 172.21.15.45:0/2531409693 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf8103850 0x7f1cf810bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.762+0000 7f1cfd000640 1 --2- 172.21.15.45:0/2531409693 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8102f30 0x7f1cf8103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.762+0000 7f1cfd000640 1 -- 172.21.15.45:0/2531409693 >> 172.21.15.45:0/2531409693 conn(0x7f1cf80fc730 msgr2=0x7f1cf80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:54.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.762+0000 7f1cfd000640 1 -- 172.21.15.45:0/2531409693 shutdown_connections 2024-04-25T19:00:54.765 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.762+0000 7f1cfd000640 1 -- 172.21.15.45:0/2531409693 wait complete. 2024-04-25T19:00:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.763+0000 7f1cfd000640 1 Processor -- start 2024-04-25T19:00:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.763+0000 7f1cfd000640 1 -- start start 2024-04-25T19:00:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.763+0000 7f1cfd000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8102f30 0x7f1cf81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.763+0000 7f1cfd000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8103850 0x7f1cf8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.763+0000 7f1cfd000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf810c350 0x7f1cf819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.763+0000 7f1cfd000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8111fe0 con 0x7f1cf8103850 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.763+0000 7f1cfd000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8111e60 con 0x7f1cf8102f30 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.763+0000 7f1cfd000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf8112160 con 0x7f1cf810c350 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cefe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8103850 0x7f1cf8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cf7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8102f30 0x7f1cf81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf810c350 0x7f1cf819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cf7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8102f30 0x7f1cf81985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48288/0 (socket says 172.21.15.45:48288) 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cf7400640 1 -- 172.21.15.45:0/232831559 learned_addr learned my addr 172.21.15.45:0/232831559 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cefe00640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8103850 0x7f1cf8198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52094/0 (socket says 172.21.15.45:52094) 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cf7400640 1 -- 172.21.15.45:0/232831559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf810c350 msgr2=0x7f1cf819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cf7400640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf810c350 0x7f1cf819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cf7400640 1 -- 172.21.15.45:0/232831559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8103850 msgr2=0x7f1cf8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cf7400640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8103850 0x7f1cf8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cf7400640 1 -- 172.21.15.45:0/232831559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf819d590 con 0x7f1cf8102f30 2024-04-25T19:00:54.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.764+0000 7f1cf7400640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8102f30 0x7f1cf81985a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800e9c0 tx=0x7f1ce800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.765+0000 7f1cf4c00640 1 -- 172.21.15.45:0/232831559 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce800cde0 con 0x7f1cf8102f30 2024-04-25T19:00:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.765+0000 7f1cfd000640 1 -- 172.21.15.45:0/232831559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf819d880 con 0x7f1cf8102f30 2024-04-25T19:00:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.765+0000 7f1cf4c00640 1 -- 172.21.15.45:0/232831559 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce80035f0 con 0x7f1cf8102f30 2024-04-25T19:00:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.765+0000 7f1cf4c00640 1 -- 172.21.15.45:0/232831559 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8010690 con 0x7f1cf8102f30 2024-04-25T19:00:54.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.765+0000 7f1cfd000640 1 -- 172.21.15.45:0/232831559 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf81a5110 con 0x7f1cf8102f30 2024-04-25T19:00:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.768+0000 7f1d01407740 1 Processor -- start 2024-04-25T19:00:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.768+0000 7f1cf4c00640 1 -- 172.21.15.45:0/232831559 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1ce8004000 con 0x7f1cf8102f30 2024-04-25T19:00:54.770 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.768+0000 7f1d01407740 1 -- start start 2024-04-25T19:00:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1d01407740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498ee7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1d01407740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ee82d0 0x55f498ef2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.771 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1d01407740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ef2b90 0x55f498ef4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1d01407740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f498e4d900 con 0x55f498ee79b0 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1d01407740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f498df3ef0 con 0x55f498ee82d0 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1d01407740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f498e498c0 con 0x55f498ef2b90 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498ee7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498ee7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52102/0 (socket says 172.21.15.45:52102) 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1cf7e00640 1 -- 172.21.15.45:0/226156118 learned_addr learned my addr 172.21.15.45:0/226156118 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1cf7400640 1 --2- 172.21.15.45:0/226156118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ee82d0 0x55f498ef2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1cefe00640 1 --2- 172.21.15.45:0/226156118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ef2b90 0x55f498ef4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1cf7e00640 1 -- 172.21.15.45:0/226156118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ef2b90 msgr2=0x55f498ef4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.769+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/226156118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ef2b90 0x55f498ef4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cf7e00640 1 -- 172.21.15.45:0/226156118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ee82d0 msgr2=0x55f498ef2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/226156118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ee82d0 0x55f498ef2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cf7e00640 1 -- 172.21.15.45:0/226156118 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f498b60210 con 0x55f498ee79b0 2024-04-25T19:00:54.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cefe00640 1 --2- 172.21.15.45:0/226156118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ef2b90 0x55f498ef4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:00:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/226156118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498ee7d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009be0 tx=0x7f1cf0005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ffae9a2a43f9e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cf4c00640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1cdc0a3d00 0x7f1cdc0a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cecc00640 1 -- 172.21.15.45:0/226156118 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0041030 con 0x55f498ee79b0 2024-04-25T19:00:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cecc00640 1 -- 172.21.15.45:0/226156118 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf0003e20 con 0x55f498ee79b0 2024-04-25T19:00:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cecc00640 1 -- 172.21.15.45:0/226156118 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0036040 con 0x55f498ee79b0 2024-04-25T19:00:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cf4c00640 1 -- 172.21.15.45:0/232831559 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f1ce80d0b60 con 0x7f1cf8102f30 2024-04-25T19:00:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.770+0000 7f1cf7400640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1cdc0a3d00 0x7f1cdc0a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.771+0000 7f1d01407740 1 -- 172.21.15.45:0/226156118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 msgr2=0x55f498ee7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:54.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.771+0000 7f1d01407740 1 --2- 172.21.15.45:0/226156118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498ee7d90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009be0 tx=0x7f1cf0005eb0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.771+0000 7f1cf7400640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1cdc0a3d00 0x7f1cdc0a61c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55f498ee8f50 tx=0x7f1ce80d4610 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.771+0000 7f1d01407740 1 -- 172.21.15.45:0/226156118 shutdown_connections 2024-04-25T19:00:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.771+0000 7f1d01407740 1 --2- 172.21.15.45:0/226156118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ef2b90 0x55f498ef4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.771+0000 7f1d01407740 1 --2- 172.21.15.45:0/226156118 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ee82d0 0x55f498ef2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.771+0000 7f1d01407740 1 --2- 172.21.15.45:0/226156118 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498ee7d90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.771+0000 7f1d01407740 1 -- 172.21.15.45:0/226156118 >> 172.21.15.45:0/226156118 conn(0x55f498ee3ec0 msgr2=0x55f498ee42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.771+0000 7f1d01407740 1 -- 172.21.15.45:0/226156118 shutdown_connections 2024-04-25T19:00:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.771+0000 7f1d01407740 1 -- 172.21.15.45:0/226156118 wait complete. 2024-04-25T19:00:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.772+0000 7f1d01407740 1 -- start start 2024-04-25T19:00:54.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.772+0000 7f1d01407740 1 Processor -- start 2024-04-25T19:00:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.772+0000 7f1d01407740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee79b0 0x55f498eefb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.772+0000 7f1d01407740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 0x55f498ef0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1d01407740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1d01407740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f498ef5760 con 0x55f498ee82d0 2024-04-25T19:00:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1d01407740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55f498e07310 con 0x55f498ef2b90 2024-04-25T19:00:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1d01407740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f498ef58e0 con 0x55f498ee79b0 2024-04-25T19:00:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1d01407740 10 client.0 _ll_register_callbacks cb 0x55f498dd5cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cefe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 0x55f498ef0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cefe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 0x55f498ef0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52114/0 (socket says 172.21.15.45:52114) 2024-04-25T19:00:54.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cefe00640 1 -- 172.21.15.45:0/30316316 learned_addr learned my addr 172.21.15.45:0/30316316 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee79b0 0x55f498eefb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cefe00640 1 -- 172.21.15.45:0/30316316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee79b0 msgr2=0x55f498eefb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cefe00640 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee79b0 0x55f498eefb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cefe00640 1 -- 172.21.15.45:0/30316316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 msgr2=0x55f498ef05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cefe00640 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cefe00640 1 -- 172.21.15.45:0/30316316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f498c2c620 con 0x55f498ee82d0 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.773+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.774+0000 7f1cefe00640 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 0x55f498ef0060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0012730 tx=0x7f1ce0012bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388fe9e37e49c394 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.774+0000 7f1ce5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.774+0000 7f1ce5600640 1 -- 172.21.15.45:0/30316316 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000f8a0 con 0x55f498ee82d0 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.774+0000 7f1ce5600640 1 -- 172.21.15.45:0/30316316 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f1ce000fa40 con 0x55f498ee82d0 2024-04-25T19:00:54.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.774+0000 7f1d01407740 1 -- 172.21.15.45:0/30316316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cf0003e20 con 0x55f498ee82d0 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.774+0000 7f1ce5600640 1 -- 172.21.15.45:0/30316316 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ce000fc40 con 0x55f498ee82d0 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.774+0000 7f1d01407740 10 client.15255 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.774+0000 7f1d01407740 1 -- 172.21.15.45:0/30316316 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f498f01a20 con 0x55f498ee82d0 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.775+0000 7f1ce5600640 1 -- 172.21.15.45:0/30316316 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf0003e20 con 0x55f498ee82d0 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.775+0000 7f1ce5600640 1 -- 172.21.15.45:0/30316316 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1ce001c420 con 0x55f498ee82d0 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.775+0000 7f1d01407740 10 client.15255 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.775+0000 7f1d01407740 10 client.15255 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.775+0000 7f1d01407740 20 client.15255 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.775+0000 7f1d01407740 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f498f02f80 0x55f498f05370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.775+0000 7f1d01407740 4 client.15255 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.775+0000 7f1d01407740 1 -- 172.21.15.45:0/30316316 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f498e618b0 con 0x55f498f02f80 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.775+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f498f02f80 0x55f498f05370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.776+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f498f02f80 0x55f498f05370 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f92831819b27e server_cookie=c646f0809d96a284 in_seq=0 out_seq=0 2024-04-25T19:00:54.780 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.776+0000 7f1ce5600640 10 client.15255 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.776+0000 7f1ce5600640 1 -- 172.21.15.45:0/30316316 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f498e618b0 con 0x55f498f02f80 2024-04-25T19:00:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.776+0000 7f1ce5600640 10 client.15255 handle_command_reply: tid=0 2024-04-25T19:00:54.781 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 client.15255 shutdown 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 -- 172.21.15.45:0/30316316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 msgr2=0x55f498ef0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 0x55f498ef0060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0012730 tx=0x7f1ce0012bf0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 -- 172.21.15.45:0/30316316 shutdown_connections 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f498f02f80 0x55f498f05370 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 0x55f498ef0060 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 --2- 172.21.15.45:0/30316316 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee79b0 0x55f498eefb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1ce5600640 0 client.15255 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 -- 172.21.15.45:0/30316316 >> 172.21.15.45:0/30316316 conn(0x55f498ee3ec0 msgr2=0x55f498ee9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 -- 172.21.15.45:0/30316316 shutdown_connections 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 1 -- 172.21.15.45:0/30316316 wait complete. 2024-04-25T19:00:54.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.777+0000 7f1d01407740 20 client.15255 trim_cache size 0 max 16384 2024-04-25T19:00:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.794+0000 7f1d01407740 1 Processor -- start 2024-04-25T19:00:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.794+0000 7f1d01407740 1 -- start start 2024-04-25T19:00:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.794+0000 7f1d01407740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee79b0 0x55f498eeb370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.794+0000 7f1d01407740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 0x55f498efe620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1d01407740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1d01407740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf81b0320 con 0x55f498ee82d0 2024-04-25T19:00:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1d01407740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf81adbf0 con 0x55f498ef2b90 2024-04-25T19:00:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1d01407740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf81aba40 con 0x55f498ee79b0 2024-04-25T19:00:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1cefe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 0x55f498efe620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48308/0 (socket says 172.21.15.45:48308) 2024-04-25T19:00:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1cf7e00640 1 -- 172.21.15.45:0/2020609958 learned_addr learned my addr 172.21.15.45:0/2020609958 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/2020609958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee79b0 0x55f498eeb370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1cf7e00640 1 -- 172.21.15.45:0/2020609958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee79b0 msgr2=0x55f498eeb370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/2020609958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee79b0 0x55f498eeb370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1cf7e00640 1 -- 172.21.15.45:0/2020609958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 msgr2=0x55f498efe620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/2020609958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 0x55f498efe620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.795+0000 7f1cf7e00640 1 -- 172.21.15.45:0/2020609958 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f498ee80c0 con 0x55f498ef2b90 2024-04-25T19:00:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.796+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/2020609958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef03c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009be0 tx=0x7f1cf0030c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a8ec25bfad21cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.796+0000 7f1ce7e00640 1 -- 172.21.15.45:0/2020609958 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0036040 con 0x55f498ef2b90 2024-04-25T19:00:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.796+0000 7f1ce7e00640 1 -- 172.21.15.45:0/2020609958 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf0003e20 con 0x55f498ef2b90 2024-04-25T19:00:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.796+0000 7f1ce7e00640 1 -- 172.21.15.45:0/2020609958 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0041040 con 0x55f498ef2b90 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.797+0000 7f1d01407740 1 -- 172.21.15.45:0/2020609958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 msgr2=0x55f498ef03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.797+0000 7f1d01407740 1 --2- 172.21.15.45:0/2020609958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef03c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009be0 tx=0x7f1cf0030c70 comp rx=0 tx=0).stop 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.797+0000 7f1d01407740 1 -- 172.21.15.45:0/2020609958 shutdown_connections 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.797+0000 7f1d01407740 1 --2- 172.21.15.45:0/2020609958 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef03c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.797+0000 7f1d01407740 1 --2- 172.21.15.45:0/2020609958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee82d0 0x55f498efe620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.797+0000 7f1d01407740 1 --2- 172.21.15.45:0/2020609958 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee79b0 0x55f498eeb370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.797+0000 7f1d01407740 1 -- 172.21.15.45:0/2020609958 >> 172.21.15.45:0/2020609958 conn(0x55f498ee3ec0 msgr2=0x55f498ee9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.797+0000 7f1d01407740 1 -- 172.21.15.45:0/2020609958 shutdown_connections 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.797+0000 7f1d01407740 1 -- 172.21.15.45:0/2020609958 wait complete. 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.798+0000 7f1d01407740 1 -- start start 2024-04-25T19:00:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.798+0000 7f1d01407740 1 Processor -- start 2024-04-25T19:00:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.798+0000 7f1d01407740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498efae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.798+0000 7f1d01407740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee82d0 0x55f498ef8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.798+0000 7f1d01407740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.798+0000 7f1d01407740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf81ac0b0 con 0x55f498ee79b0 2024-04-25T19:00:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1d01407740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf81ad520 con 0x55f498ef2b90 2024-04-25T19:00:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1d01407740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf81ac780 con 0x55f498ee82d0 2024-04-25T19:00:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1d01407740 10 client.0 _ll_register_callbacks cb 0x7f1ce0009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:00:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498efae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cefe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee82d0 0x55f498ef8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cf7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498efae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:52126/0 (socket says 172.21.15.45:52126) 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cf7e00640 1 -- 172.21.15.45:0/3841911257 learned_addr learned my addr 172.21.15.45:0/3841911257 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cefe00640 1 -- 172.21.15.45:0/3841911257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 msgr2=0x55f498ef8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cefe00640 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cefe00640 1 -- 172.21.15.45:0/3841911257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 msgr2=0x55f498efae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cefe00640 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498efae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cefe00640 1 -- 172.21.15.45:0/3841911257 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f498ef8f30 con 0x55f498ee82d0 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1cefe00640 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee82d0 0x55f498ef8220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ce001c910 tx=0x7f1ce000fbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66ecd1886faf1cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.799+0000 7f1ce5600640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.800+0000 7f1ce5600640 1 -- 172.21.15.45:0/3841911257 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce001c630 con 0x55f498ee82d0 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.800+0000 7f1ce5600640 1 -- 172.21.15.45:0/3841911257 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f1ce001cb00 con 0x55f498ee82d0 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.800+0000 7f1d01407740 1 -- 172.21.15.45:0/3841911257 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cf0003e20 con 0x55f498ee82d0 2024-04-25T19:00:54.804 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.800+0000 7f1ce5600640 1 -- 172.21.15.45:0/3841911257 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ce0033050 con 0x55f498ee82d0 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.800+0000 7f1d01407740 10 client.25405 fetch_fsmap learned FSMap version 26 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.800+0000 7f1d01407740 1 -- 172.21.15.45:0/3841911257 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f498f6d8a0 con 0x55f498ee82d0 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.800+0000 7f1ce5600640 1 -- 172.21.15.45:0/3841911257 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cf0003e20 con 0x55f498ee82d0 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.801+0000 7f1ce5600640 1 -- 172.21.15.45:0/3841911257 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f1ce002b070 con 0x55f498ee82d0 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.801+0000 7f1d01407740 10 client.25405 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.801+0000 7f1d01407740 10 client.25405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.801+0000 7f1d01407740 20 client.25405 populate_metadata read hostname 'smithi045' 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.801+0000 7f1d01407740 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f498f6e020 0x55f498f70440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.801+0000 7f1d01407740 4 client.25405 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.801+0000 7f1d01407740 1 -- 172.21.15.45:0/3841911257 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f498e5b320 con 0x55f498f6e020 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.801+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f498f6e020 0x55f498f70440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:00:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.801+0000 7f1cf7e00640 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f498f6e020 0x55f498f70440 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d4d925e6a7fca3 server_cookie=f25c57d34fa5e52 in_seq=0 out_seq=0 2024-04-25T19:00:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.802+0000 7f1ce5600640 10 client.25405 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1ce5600640 1 -- 172.21.15.45:0/3841911257 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f498e5b320 con 0x55f498f6e020 2024-04-25T19:00:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1ce5600640 10 client.25405 handle_command_reply: tid=0 2024-04-25T19:00:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 client.25405 shutdown 2024-04-25T19:00:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 -- 172.21.15.45:0/3841911257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee82d0 msgr2=0x55f498ef8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee82d0 0x55f498ef8220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1ce001c910 tx=0x7f1ce000fbb0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 -- 172.21.15.45:0/3841911257 shutdown_connections 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55f498f6e020 0x55f498f70440 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55f498ef2b90 0x55f498ef8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f498ee82d0 0x55f498ef8220 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 --2- 172.21.15.45:0/3841911257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f498ee79b0 0x55f498efae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1ce5600640 0 client.25405 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 -- 172.21.15.45:0/3841911257 >> 172.21.15.45:0/3841911257 conn(0x55f498ee3ec0 msgr2=0x55f498ee2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 -- 172.21.15.45:0/3841911257 shutdown_connections 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 1 -- 172.21.15.45:0/3841911257 wait complete. 2024-04-25T19:00:54.814 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.808+0000 7f1d01407740 20 client.25405 trim_cache size 0 max 16384 2024-04-25T19:00:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:00:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:00:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:00:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:00:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.809+0000 7f1cfd000640 1 -- 172.21.15.45:0/232831559 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1cdc0a3d00 msgr2=0x7f1cdc0a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.809+0000 7f1cfd000640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1cdc0a3d00 0x7f1cdc0a61c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x55f498ee8f50 tx=0x7f1ce80d4610 comp rx=0 tx=0).stop 2024-04-25T19:00:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.809+0000 7f1cfd000640 1 -- 172.21.15.45:0/232831559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8102f30 msgr2=0x7f1cf81985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:00:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.809+0000 7f1cfd000640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8102f30 0x7f1cf81985a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1ce800e9c0 tx=0x7f1ce800ee80 comp rx=0 tx=0).stop 2024-04-25T19:00:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.809+0000 7f1cfd000640 1 -- 172.21.15.45:0/232831559 shutdown_connections 2024-04-25T19:00:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.809+0000 7f1cfd000640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1cdc0a3d00 0x7f1cdc0a61c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.809+0000 7f1cfd000640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf810c350 0x7f1cf819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.809+0000 7f1cfd000640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1cf8103850 0x7f1cf8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.809+0000 7f1cfd000640 1 --2- 172.21.15.45:0/232831559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1cf8102f30 0x7f1cf81985a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:00:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.809+0000 7f1cfd000640 1 -- 172.21.15.45:0/232831559 >> 172.21.15.45:0/232831559 conn(0x7f1cf80fc730 msgr2=0x7f1cf80fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:00:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.810+0000 7f1cfd000640 1 -- 172.21.15.45:0/232831559 shutdown_connections 2024-04-25T19:00:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:00:54.810+0000 7f1cfd000640 1 -- 172.21.15.45:0/232831559 wait complete. 2024-04-25T19:00:56.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:56 smithi062 ceph-mon[32844]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T19:00:56.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:56 smithi079 ceph-mon[33002]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T19:00:56.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:56 smithi045 ceph-mon[27776]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T19:00:57.928 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:00:58.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:00:58 smithi062 ceph-mon[32844]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-25T19:00:58.388 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:00:58.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:00:58 smithi079 ceph-mon[33002]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-25T19:00:58.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:00:58 smithi045 ceph-mon[27776]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.8 KiB/s wr, 8 op/s 2024-04-25T19:01:00.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:00 smithi062 ceph-mon[32844]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T19:01:00.384 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:00 smithi045 ceph-mon[27776]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T19:01:00.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:00 smithi079 ceph-mon[33002]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T19:01:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.714+0000 7fd22ea00700 1 -- 172.21.15.45:0/1450041934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd228105840 msgr2=0x7fd228109890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.714+0000 7fd22ea00700 1 --2- 172.21.15.45:0/1450041934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd228105840 0x7fd228109890 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2100098f0 tx=0x7fd210009c00 comp rx=0 tx=0).stop 2024-04-25T19:01:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.714+0000 7fd22ea00700 1 -- 172.21.15.45:0/1450041934 shutdown_connections 2024-04-25T19:01:00.717 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.714+0000 7fd22ea00700 1 --2- 172.21.15.45:0/1450041934 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd228109f70 0x7fd228111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.714+0000 7fd22ea00700 1 --2- 172.21.15.45:0/1450041934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd228104e90 0x7fd228105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.714+0000 7fd22ea00700 1 --2- 172.21.15.45:0/1450041934 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd228105840 0x7fd228109890 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.714+0000 7fd22ea00700 1 -- 172.21.15.45:0/1450041934 >> 172.21.15.45:0/1450041934 conn(0x7fd228100720 msgr2=0x7fd228102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.714+0000 7fd22ea00700 1 -- 172.21.15.45:0/1450041934 shutdown_connections 2024-04-25T19:01:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.714+0000 7fd22ea00700 1 -- 172.21.15.45:0/1450041934 wait complete. 2024-04-25T19:01:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.715+0000 7fd22ea00700 1 Processor -- start 2024-04-25T19:01:00.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.715+0000 7fd22ea00700 1 -- start start 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd22ea00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd228104e90 0x7fd22819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd22ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd228105840 0x7fd22819b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd22ea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd228109f70 0x7fd22819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd22ea00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd228114de0 con 0x7fd228109f70 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd22ea00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd228114c60 con 0x7fd228105840 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd22ea00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd228114f60 con 0x7fd228104e90 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd21f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd228105840 0x7fd22819b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd22ca00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd228109f70 0x7fd22819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd21f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd228105840 0x7fd22819b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48328/0 (socket says 172.21.15.45:48328) 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd21f400700 1 -- 172.21.15.45:0/483545594 learned_addr learned my addr 172.21.15.45:0/483545594 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:00.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.716+0000 7fd227e00700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd228104e90 0x7fd22819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.717+0000 7fd21f400700 1 -- 172.21.15.45:0/483545594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd228104e90 msgr2=0x7fd22819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.717+0000 7fd21f400700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd228104e90 0x7fd22819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.717+0000 7fd21f400700 1 -- 172.21.15.45:0/483545594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd228109f70 msgr2=0x7fd22819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.717+0000 7fd21f400700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd228109f70 0x7fd22819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.717+0000 7fd21f400700 1 -- 172.21.15.45:0/483545594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2281a0370 con 0x7fd228105840 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.717+0000 7fd21f400700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd228105840 0x7fd22819b8f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd210009fd0 tx=0x7fd21000fa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.717+0000 7fd225600700 1 -- 172.21.15.45:0/483545594 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd210021df0 con 0x7fd228105840 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.717+0000 7fd225600700 1 -- 172.21.15.45:0/483545594 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2100054f0 con 0x7fd228105840 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.717+0000 7fd225600700 1 -- 172.21.15.45:0/483545594 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2100038e0 con 0x7fd228105840 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.717+0000 7fd22ea00700 1 -- 172.21.15.45:0/483545594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2281a0600 con 0x7fd228105840 2024-04-25T19:01:00.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.718+0000 7fd22ea00700 1 -- 172.21.15.45:0/483545594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd228076e80 con 0x7fd228105840 2024-04-25T19:01:00.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.719+0000 7fd22ea00700 1 -- 172.21.15.45:0/483545594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd228069730 con 0x7fd228105840 2024-04-25T19:01:00.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.722+0000 7fd225600700 1 -- 172.21.15.45:0/483545594 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd21000b560 con 0x7fd228105840 2024-04-25T19:01:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.724+0000 7fd225600700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd20c0a3e30 0x7fd20c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.724+0000 7fd225600700 1 -- 172.21.15.45:0/483545594 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fd2100da630 con 0x7fd228105840 2024-04-25T19:01:00.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.724+0000 7fd227e00700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd20c0a3e30 0x7fd20c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:00.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.725+0000 7fd225600700 1 -- 172.21.15.45:0/483545594 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd21003a020 con 0x7fd228105840 2024-04-25T19:01:00.727 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:00.725+0000 7fd227e00700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd20c0a3e30 0x7fd20c0a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd21800f540 tx=0x7fd21800b040 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:01.019 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.016+0000 7fd22ea00700 1 -- 172.21.15.45:0/483545594 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd22819c6c0 con 0x7fd228105840 2024-04-25T19:01:01.020 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.018+0000 7fd225600700 1 -- 172.21.15.45:0/483545594 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11852 (secure 0 0 0) 0x7fd210013030 con 0x7fd228105840 2024-04-25T19:01:01.020 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:01:01.021 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T18:59:18.321335+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24512,24505]},"id":1}]} 2024-04-25T19:01:01.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.021+0000 7fd22ea00700 1 -- 172.21.15.45:0/483545594 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd20c0a3e30 msgr2=0x7fd20c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.021+0000 7fd22ea00700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd20c0a3e30 0x7fd20c0a62f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd21800f540 tx=0x7fd21800b040 comp rx=0 tx=0).stop 2024-04-25T19:01:01.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.021+0000 7fd22ea00700 1 -- 172.21.15.45:0/483545594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd228105840 msgr2=0x7fd22819b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:01.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.021+0000 7fd22ea00700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd228105840 0x7fd22819b8f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd210009fd0 tx=0x7fd21000fa30 comp rx=0 tx=0).stop 2024-04-25T19:01:01.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.021+0000 7fd22ea00700 1 -- 172.21.15.45:0/483545594 shutdown_connections 2024-04-25T19:01:01.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.021+0000 7fd22ea00700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd20c0a3e30 0x7fd20c0a62f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.021+0000 7fd22ea00700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd228109f70 0x7fd22819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.021+0000 7fd22ea00700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd228104e90 0x7fd22819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.021+0000 7fd22ea00700 1 --2- 172.21.15.45:0/483545594 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd228105840 0x7fd22819b8f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:01.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.021+0000 7fd22ea00700 1 -- 172.21.15.45:0/483545594 >> 172.21.15.45:0/483545594 conn(0x7fd228100720 msgr2=0x7fd228102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:01.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.022+0000 7fd22ea00700 1 -- 172.21.15.45:0/483545594 shutdown_connections 2024-04-25T19:01:01.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:01.022+0000 7fd22ea00700 1 -- 172.21.15.45:0/483545594 wait complete. 2024-04-25T19:01:01.025 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 26 2024-04-25T19:01:01.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:01 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/483545594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:01.424 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:01 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/483545594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:01.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:01 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/483545594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:01.951 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 213d97e7 --release --await' 2024-04-25T19:01:01.952 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24524 quiesce db --set-id 213d97e7 --release --await 2024-04-25T19:01:02.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:02 smithi045 ceph-mon[27776]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T19:01:02.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:02 smithi062 ceph-mon[32844]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T19:01:02.426 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:02.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:02 smithi079 ceph-mon[33002]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-04-25T19:01:04.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:04 smithi045 ceph-mon[27776]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T19:01:04.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:04 smithi062 ceph-mon[32844]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T19:01:04.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:04 smithi079 ceph-mon[33002]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T19:01:04.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.738+0000 7ff872800700 1 -- 172.21.15.45:0/1016118533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c108a30 msgr2=0x7ff86c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.738+0000 7ff872800700 1 --2- 172.21.15.45:0/1016118533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c108a30 0x7ff86c108e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0098f0 tx=0x7ff85c009c00 comp rx=0 tx=0).stop 2024-04-25T19:01:04.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.738+0000 7ff872800700 1 -- 172.21.15.45:0/1016118533 shutdown_connections 2024-04-25T19:01:04.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.738+0000 7ff872800700 1 --2- 172.21.15.45:0/1016118533 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff86c06b0f0 0x7ff86c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.738+0000 7ff872800700 1 --2- 172.21.15.45:0/1016118533 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c108a30 0x7ff86c108e10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.738+0000 7ff872800700 1 --2- 172.21.15.45:0/1016118533 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff86c06a750 0x7ff86c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.738+0000 7ff872800700 1 -- 172.21.15.45:0/1016118533 >> 172.21.15.45:0/1016118533 conn(0x7ff86c0788f0 msgr2=0x7ff86c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:04.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.738+0000 7ff872800700 1 -- 172.21.15.45:0/1016118533 shutdown_connections 2024-04-25T19:01:04.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.738+0000 7ff872800700 1 -- 172.21.15.45:0/1016118533 wait complete. 2024-04-25T19:01:04.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.740+0000 7ff872800700 1 Processor -- start 2024-04-25T19:01:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.740+0000 7ff872800700 1 -- start start 2024-04-25T19:01:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.740+0000 7ff872800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff86c06a750 0x7ff86c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.740+0000 7ff872800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff86c06b0f0 0x7ff86c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff872800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c108a30 0x7ff86c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff872800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c112c00 con 0x7ff86c06a750 2024-04-25T19:01:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff872800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c112a80 con 0x7ff86c06b0f0 2024-04-25T19:01:04.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff872800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c112d80 con 0x7ff86c108a30 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff86b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff86c06a750 0x7ff86c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff86b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff86c06a750 0x7ff86c19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55458/0 (socket says 172.21.15.45:55458) 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff86b400700 1 -- 172.21.15.45:0/9875974 learned_addr learned my addr 172.21.15.45:0/9875974 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff86aa00700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff86c06b0f0 0x7ff86c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff86be00700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c108a30 0x7ff86c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff86b400700 1 -- 172.21.15.45:0/9875974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c108a30 msgr2=0x7ff86c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff86b400700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c108a30 0x7ff86c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff86b400700 1 -- 172.21.15.45:0/9875974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff86c06b0f0 msgr2=0x7ff86c19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff86b400700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff86c06b0f0 0x7ff86c19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.741+0000 7ff86b400700 1 -- 172.21.15.45:0/9875974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86c1a03c0 con 0x7ff86c06a750 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.742+0000 7ff86aa00700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff86c06b0f0 0x7ff86c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:04.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.742+0000 7ff86b400700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff86c06a750 0x7ff86c19b3b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009230 tx=0x7ff85c00fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.742+0000 7ff863e00700 1 -- 172.21.15.45:0/9875974 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c0050c0 con 0x7ff86c06a750 2024-04-25T19:01:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.742+0000 7ff872800700 1 -- 172.21.15.45:0/9875974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86c1a0650 con 0x7ff86c06a750 2024-04-25T19:01:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.742+0000 7ff863e00700 1 -- 172.21.15.45:0/9875974 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff85c00b560 con 0x7ff86c06a750 2024-04-25T19:01:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.742+0000 7ff863e00700 1 -- 172.21.15.45:0/9875974 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85c003a00 con 0x7ff86c06a750 2024-04-25T19:01:04.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.742+0000 7ff872800700 1 -- 172.21.15.45:0/9875974 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86c1a7f30 con 0x7ff86c06a750 2024-04-25T19:01:04.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.746+0000 7ff863e00700 1 -- 172.21.15.45:0/9875974 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7ff85c00b710 con 0x7ff86c06a750 2024-04-25T19:01:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.747+0000 7ff87d23db80 1 Processor -- start 2024-04-25T19:01:04.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.747+0000 7ff87d23db80 1 -- start start 2024-04-25T19:01:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952ad40 0x559c3952b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 0x559c39535f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c395388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff87d23db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559c3947df90 con 0x559c3952b660 2024-04-25T19:01:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff87d23db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559c39424830 con 0x559c395364d0 2024-04-25T19:01:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff87d23db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559c39479f50 con 0x559c3952ad40 2024-04-25T19:01:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff863e00700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7ff8500a3db0 0x7ff8500a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff86be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952ad40 0x559c3952b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff86be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952ad40 0x559c3952b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:54664/0 (socket says 172.21.15.45:54664) 2024-04-25T19:01:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff86be00700 1 -- 172.21.15.45:0/3519473891 learned_addr learned my addr 172.21.15.45:0/3519473891 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff86aa00700 1 --2- 172.21.15.45:0/3519473891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c395388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff86be00700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7ff8500a3db0 0x7ff8500a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff863e00700 1 -- 172.21.15.45:0/9875974 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7ff85c0d1730 con 0x7ff86c06a750 2024-04-25T19:01:04.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.748+0000 7ff86b400700 1 --2- 172.21.15.45:0/3519473891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 0x559c39535f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.749+0000 7ff86be00700 1 -- 172.21.15.45:0/3519473891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 msgr2=0x559c395388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:04.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.749+0000 7ff86be00700 1 --2- 172.21.15.45:0/3519473891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c395388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.749+0000 7ff86be00700 1 -- 172.21.15.45:0/3519473891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 msgr2=0x559c39535f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:04.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.749+0000 7ff86be00700 1 --2- 172.21.15.45:0/3519473891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 0x559c39535f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.749+0000 7ff86be00700 1 -- 172.21.15.45:0/3519473891 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c3941a780 con 0x559c3952ad40 2024-04-25T19:01:04.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.749+0000 7ff86be00700 1 --2- 172.21.15.45:0/3519473891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952ad40 0x559c3952b120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff864008ad0 tx=0x7ff864008de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4312c93f006ab49c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.750+0000 7ff86be00700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7ff8500a3db0 0x7ff8500a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff864011000 tx=0x7ff86401d000 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.750+0000 7ff860c00700 1 -- 172.21.15.45:0/3519473891 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86401a650 con 0x559c3952ad40 2024-04-25T19:01:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.750+0000 7ff860c00700 1 -- 172.21.15.45:0/3519473891 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff86401aca0 con 0x559c3952ad40 2024-04-25T19:01:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.750+0000 7ff860c00700 1 -- 172.21.15.45:0/3519473891 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864024c40 con 0x559c3952ad40 2024-04-25T19:01:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.750+0000 7ff87d23db80 1 -- 172.21.15.45:0/3519473891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952ad40 msgr2=0x559c3952b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.750+0000 7ff87d23db80 1 --2- 172.21.15.45:0/3519473891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952ad40 0x559c3952b120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff864008ad0 tx=0x7ff864008de0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.751+0000 7ff87d23db80 1 -- 172.21.15.45:0/3519473891 shutdown_connections 2024-04-25T19:01:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.751+0000 7ff87d23db80 1 --2- 172.21.15.45:0/3519473891 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c395388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.751+0000 7ff87d23db80 1 --2- 172.21.15.45:0/3519473891 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952ad40 0x559c3952b120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.751+0000 7ff87d23db80 1 --2- 172.21.15.45:0/3519473891 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 0x559c39535f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.751+0000 7ff87d23db80 1 -- 172.21.15.45:0/3519473891 >> 172.21.15.45:0/3519473891 conn(0x559c3945a050 msgr2=0x559c39527250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.751+0000 7ff87d23db80 1 -- 172.21.15.45:0/3519473891 shutdown_connections 2024-04-25T19:01:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.751+0000 7ff87d23db80 1 -- 172.21.15.45:0/3519473891 wait complete. 2024-04-25T19:01:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.752+0000 7ff87d23db80 1 -- start start 2024-04-25T19:01:04.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.752+0000 7ff87d23db80 1 Processor -- start 2024-04-25T19:01:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.752+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 0x559c39535c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39532f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c395334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff87d23db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559c39538f60 con 0x559c3952ad40 2024-04-25T19:01:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff87d23db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559c39437960 con 0x559c395364d0 2024-04-25T19:01:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff87d23db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559c395390e0 con 0x559c3952b660 2024-04-25T19:01:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff87d23db80 10 client.0 _ll_register_callbacks cb 0x559c39416820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff86b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 0x559c39535c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff86b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 0x559c39535c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55476/0 (socket says 172.21.15.45:55476) 2024-04-25T19:01:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff86b400700 1 -- 172.21.15.45:0/2789148307 learned_addr learned my addr 172.21.15.45:0/2789148307 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff86be00700 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c395334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff86aa00700 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39532f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff86be00700 1 -- 172.21.15.45:0/2789148307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 msgr2=0x559c39532f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff86be00700 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39532f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff86be00700 1 -- 172.21.15.45:0/2789148307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 msgr2=0x559c39535c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:04.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.753+0000 7ff86be00700 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 0x559c39535c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.754+0000 7ff86be00700 1 -- 172.21.15.45:0/2789148307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c393cb9d0 con 0x559c395364d0 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.754+0000 7ff86aa00700 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39532f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.754+0000 7ff86be00700 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c395334c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff864009540 tx=0x7ff864004d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b20f5f7b56665c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.754+0000 7ff855600700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.754+0000 7ff855600700 1 -- 172.21.15.45:0/2789148307 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864024df0 con 0x559c395364d0 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.754+0000 7ff87d23db80 1 -- 172.21.15.45:0/2789148307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff86401aca0 con 0x559c395364d0 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.754+0000 7ff855600700 1 -- 172.21.15.45:0/2789148307 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7ff86401a650 con 0x559c395364d0 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff855600700 1 -- 172.21.15.45:0/2789148307 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff86403d050 con 0x559c395364d0 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff87d23db80 10 client.25406 fetch_fsmap learned FSMap version 26 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff87d23db80 1 -- 172.21.15.45:0/2789148307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559c39414bf0 con 0x559c395364d0 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff855600700 1 -- 172.21.15.45:0/2789148307 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff86401aca0 con 0x559c395364d0 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff855600700 1 -- 172.21.15.45:0/2789148307 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff86402c8d0 con 0x559c395364d0 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff87d23db80 10 client.25406 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:01:04.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff87d23db80 10 client.25406 resolve_mds: validated gid 24524 aka daemon mds.b 2024-04-25T19:01:04.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff87d23db80 20 client.25406 populate_metadata read hostname 'smithi045' 2024-04-25T19:01:04.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff87d23db80 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559c39545cc0 0x559c395480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff87d23db80 4 client.25406 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:01:04.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.755+0000 7ff87d23db80 1 -- 172.21.15.45:0/2789148307 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c3948b780 con 0x559c39545cc0 2024-04-25T19:01:04.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.756+0000 7ff86b400700 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559c39545cc0 0x559c395480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.756+0000 7ff86b400700 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559c39545cc0 0x559c395480b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c8660c7028a8aa server_cookie=3fdd49f2b2cf7e0d in_seq=0 out_seq=0 2024-04-25T19:01:04.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.756+0000 7ff855600700 10 client.25406 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:04.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.757+0000 7ff855600700 1 -- 172.21.15.45:0/2789148307 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c3948b780 con 0x559c39545cc0 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.757+0000 7ff855600700 10 client.25406 handle_command_reply: tid=0 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.757+0000 7ff87d23db80 1 client.25406 shutdown 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.757+0000 7ff87d23db80 1 -- 172.21.15.45:0/2789148307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 msgr2=0x559c395334c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.757+0000 7ff87d23db80 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c395334c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff864009540 tx=0x7ff864004d40 comp rx=0 tx=0).stop 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.757+0000 7ff87d23db80 1 -- 172.21.15.45:0/2789148307 shutdown_connections 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.757+0000 7ff87d23db80 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559c39545cc0 0x559c395480b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.758+0000 7ff87d23db80 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c395334c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.758+0000 7ff855600700 0 client.25406 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.758+0000 7ff87d23db80 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 0x559c39535c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.758+0000 7ff87d23db80 1 --2- 172.21.15.45:0/2789148307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39532f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.758+0000 7ff87d23db80 1 -- 172.21.15.45:0/2789148307 >> 172.21.15.45:0/2789148307 conn(0x559c3945a050 msgr2=0x559c39526e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.758+0000 7ff87d23db80 1 -- 172.21.15.45:0/2789148307 shutdown_connections 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.758+0000 7ff87d23db80 1 -- 172.21.15.45:0/2789148307 wait complete. 2024-04-25T19:01:04.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.758+0000 7ff87d23db80 20 client.25406 trim_cache size 0 max 16384 2024-04-25T19:01:04.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.780+0000 7ff87d23db80 1 Processor -- start 2024-04-25T19:01:04.782 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.780+0000 7ff87d23db80 1 -- start start 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.780+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 0x559c39541eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.780+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39530200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.780+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c39530740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.780+0000 7ff87d23db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c1ad940 con 0x559c3952ad40 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.780+0000 7ff87d23db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c1ab1d0 con 0x559c395364d0 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.780+0000 7ff87d23db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c10d960 con 0x559c3952b660 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39530200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86aa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39530200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:54676/0 (socket says 172.21.15.45:54676) 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86aa00700 1 -- 172.21.15.45:0/1822801099 learned_addr learned my addr 172.21.15.45:0/1822801099 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86be00700 1 --2- 172.21.15.45:0/1822801099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c39530740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.783 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86b400700 1 --2- 172.21.15.45:0/1822801099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 0x559c39541eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86aa00700 1 -- 172.21.15.45:0/1822801099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 msgr2=0x559c39530740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86aa00700 1 --2- 172.21.15.45:0/1822801099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c39530740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86aa00700 1 -- 172.21.15.45:0/1822801099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 msgr2=0x559c39541eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86aa00700 1 --2- 172.21.15.45:0/1822801099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 0x559c39541eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86aa00700 1 -- 172.21.15.45:0/1822801099 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c39539120 con 0x559c3952b660 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.781+0000 7ff86be00700 1 --2- 172.21.15.45:0/1822801099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c39530740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.782+0000 7ff86aa00700 1 --2- 172.21.15.45:0/1822801099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39530200 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff8580046a0 tx=0x7ff858009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40c178dab8c607ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.782+0000 7ff857e00700 1 -- 172.21.15.45:0/1822801099 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858027030 con 0x559c3952b660 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.782+0000 7ff857e00700 1 -- 172.21.15.45:0/1822801099 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff858004ce0 con 0x559c3952b660 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.782+0000 7ff857e00700 1 -- 172.21.15.45:0/1822801099 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858023400 con 0x559c3952b660 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.783+0000 7ff87d23db80 1 -- 172.21.15.45:0/1822801099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 msgr2=0x559c39530200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.783+0000 7ff87d23db80 1 --2- 172.21.15.45:0/1822801099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39530200 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff8580046a0 tx=0x7ff858009260 comp rx=0 tx=0).stop 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.783+0000 7ff87d23db80 1 -- 172.21.15.45:0/1822801099 shutdown_connections 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.783+0000 7ff87d23db80 1 --2- 172.21.15.45:0/1822801099 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c395364d0 0x559c39530740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.783+0000 7ff87d23db80 1 --2- 172.21.15.45:0/1822801099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952ad40 0x559c39541eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.783+0000 7ff87d23db80 1 --2- 172.21.15.45:0/1822801099 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c3952b660 0x559c39530200 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.783+0000 7ff87d23db80 1 -- 172.21.15.45:0/1822801099 >> 172.21.15.45:0/1822801099 conn(0x559c3945a050 msgr2=0x559c39526c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.783+0000 7ff87d23db80 1 -- 172.21.15.45:0/1822801099 shutdown_connections 2024-04-25T19:01:04.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.783+0000 7ff87d23db80 1 -- 172.21.15.45:0/1822801099 wait complete. 2024-04-25T19:01:04.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.783+0000 7ff87d23db80 1 -- start start 2024-04-25T19:01:04.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.784+0000 7ff87d23db80 1 Processor -- start 2024-04-25T19:01:04.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.784+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c3952ad40 0x559c3953b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.784+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 0x559c3953ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.784+0000 7ff87d23db80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c395364d0 0x559c395abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.784+0000 7ff87d23db80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c1a9690 con 0x559c3952b660 2024-04-25T19:01:04.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.784+0000 7ff87d23db80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c1aab00 con 0x559c3952ad40 2024-04-25T19:01:04.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.784+0000 7ff87d23db80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff86c1a9d60 con 0x559c395364d0 2024-04-25T19:01:04.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.784+0000 7ff87d23db80 10 client.0 _ll_register_callbacks cb 0x559c39416820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 0x559c3953ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86b400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c3952ad40 0x559c3953b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 0x559c3953ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55496/0 (socket says 172.21.15.45:55496) 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c395364d0 0x559c395abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86aa00700 1 -- 172.21.15.45:0/946051787 learned_addr learned my addr 172.21.15.45:0/946051787 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86be00700 1 -- 172.21.15.45:0/946051787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c3952ad40 msgr2=0x559c3953b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86be00700 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c3952ad40 0x559c3953b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86be00700 1 -- 172.21.15.45:0/946051787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 msgr2=0x559c3953ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86be00700 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 0x559c3953ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86be00700 1 -- 172.21.15.45:0/946051787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3952cbb0 con 0x559c395364d0 2024-04-25T19:01:04.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86aa00700 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 0x559c3953ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff86be00700 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c395364d0 0x559c395abf60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff8640089a0 tx=0x7ff864024730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cfe8de8243464b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.785+0000 7ff855600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.786+0000 7ff855600700 1 -- 172.21.15.45:0/946051787 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff864045020 con 0x559c395364d0 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.786+0000 7ff87d23db80 1 -- 172.21.15.45:0/946051787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff858004ce0 con 0x559c395364d0 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.786+0000 7ff855600700 1 -- 172.21.15.45:0/946051787 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7ff864024bf0 con 0x559c395364d0 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.786+0000 7ff855600700 1 -- 172.21.15.45:0/946051787 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff864050050 con 0x559c395364d0 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.786+0000 7ff87d23db80 10 client.25429 fetch_fsmap learned FSMap version 26 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.786+0000 7ff87d23db80 1 -- 172.21.15.45:0/946051787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559c395ad5e0 con 0x559c395364d0 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.786+0000 7ff855600700 1 -- 172.21.15.45:0/946051787 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff858004ce0 con 0x559c395364d0 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.787+0000 7ff855600700 1 -- 172.21.15.45:0/946051787 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7ff86401aaa0 con 0x559c395364d0 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.787+0000 7ff87d23db80 10 client.25429 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T19:01:04.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.787+0000 7ff87d23db80 10 client.25429 resolve_mds: validated gid 24524 aka daemon mds.b 2024-04-25T19:01:04.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.787+0000 7ff87d23db80 20 client.25429 populate_metadata read hostname 'smithi045' 2024-04-25T19:01:04.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.787+0000 7ff87d23db80 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559c395adc20 0x559c395b0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:04.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.787+0000 7ff87d23db80 4 client.25429 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "213d97e7", "release": true, "await": true}] 2024-04-25T19:01:04.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.787+0000 7ff87d23db80 1 -- 172.21.15.45:0/946051787 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "quiesce db", "set_id": "213d97e7", "release": true, "await": true}) -- 0x559c39485820 con 0x559c395adc20 2024-04-25T19:01:04.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.787+0000 7ff86b400700 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559c395adc20 0x559c395b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:04.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.788+0000 7ff86b400700 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559c395adc20 0x559c395b0010 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40df1bd86bbedd93 server_cookie=9df13be399636976 in_seq=0 out_seq=0 2024-04-25T19:01:04.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.788+0000 7ff855600700 10 client.25429 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:04.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff855600700 1 -- 172.21.15.45:0/946051787 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x559c39485820 con 0x559c395adc20 2024-04-25T19:01:04.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff855600700 10 client.25429 handle_command_reply: tid=0 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff87d23db80 1 client.25429 shutdown 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff87d23db80 1 -- 172.21.15.45:0/946051787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c395364d0 msgr2=0x559c395abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff87d23db80 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c395364d0 0x559c395abf60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff8640089a0 tx=0x7ff864024730 comp rx=0 tx=0).stop 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff87d23db80 1 -- 172.21.15.45:0/946051787 shutdown_connections 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff87d23db80 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559c395adc20 0x559c395b0010 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff87d23db80 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559c3952ad40 0x559c3953b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff855600700 0 client.25429 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff87d23db80 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559c395364d0 0x559c395abf60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff87d23db80 1 --2- 172.21.15.45:0/946051787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559c3952b660 0x559c3953ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.791+0000 7ff87d23db80 1 -- 172.21.15.45:0/946051787 >> 172.21.15.45:0/946051787 conn(0x559c3945a050 msgr2=0x559c3954c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.792+0000 7ff87d23db80 1 -- 172.21.15.45:0/946051787 shutdown_connections 2024-04-25T19:01:04.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.792+0000 7ff87d23db80 1 -- 172.21.15.45:0/946051787 wait complete. 2024-04-25T19:01:04.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.792+0000 7ff87d23db80 20 client.25429 trim_cache size 0 max 16384 2024-04-25T19:01:04.795 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:01:04.795 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 26, 2024-04-25T19:01:04.795 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24524, 2024-04-25T19:01:04.795 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 4, 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "213d97e7": { 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 4, 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 60.0, 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 89.4, 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-04-25T19:01:04.796 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-04-25T19:01:04.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RELEASED", 2024-04-25T19:01:04.797 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-04-25T19:01:04.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:01:04.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:01:04.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:01:04.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:01:04.797 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:01:04.797 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 -- 172.21.15.45:0/9875974 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7ff8500a3db0 msgr2=0x7ff8500a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7ff8500a3db0 0x7ff8500a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff864011000 tx=0x7ff86401d000 comp rx=0 tx=0).stop 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 -- 172.21.15.45:0/9875974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff86c06a750 msgr2=0x7ff86c19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff86c06a750 0x7ff86c19b3b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff85c009230 tx=0x7ff85c00fa70 comp rx=0 tx=0).stop 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 -- 172.21.15.45:0/9875974 shutdown_connections 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7ff8500a3db0 0x7ff8500a6270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff86c108a30 0x7ff86c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff86c06a750 0x7ff86c19b3b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 --2- 172.21.15.45:0/9875974 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7ff86c06b0f0 0x7ff86c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 -- 172.21.15.45:0/9875974 >> 172.21.15.45:0/9875974 conn(0x7ff86c0788f0 msgr2=0x7ff86c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 -- 172.21.15.45:0/9875974 shutdown_connections 2024-04-25T19:01:04.798 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:04.793+0000 7ff872800700 1 -- 172.21.15.45:0/9875974 wait complete. 2024-04-25T19:01:04.807 INFO:tasks.workunit.client.0.smithi045.stderr:+ set -e 2024-04-25T19:01:04.807 INFO:tasks.workunit.client.0.smithi045.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-25T19:01:04.813 INFO:tasks.workunit.client.0.smithi045.stderr:Cloning into 'xfstests-dev'... 2024-04-25T19:01:05.877 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '213d97e7', seconds elapsed: 45.1 2024-04-25T19:01:05.877 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.4 2024-04-25T19:01:05.877 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:01:06.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:06 smithi045 ceph-mon[27776]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-25T19:01:06.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:06 smithi062 ceph-mon[32844]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-25T19:01:06.371 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:06.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:06 smithi079 ceph-mon[33002]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-25T19:01:07.215 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:07 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:01:07.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:07 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:01:07.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:07 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:01:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.953+0000 7fb6b6c00640 1 -- 172.21.15.45:0/2447852333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6b0101880 msgr2=0x7fb6b01058c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.953+0000 7fb6b6c00640 1 --2- 172.21.15.45:0/2447852333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6b0101880 0x7fb6b01058c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009900 tx=0x7fb69c02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.953+0000 7fb6b6c00640 1 -- 172.21.15.45:0/2447852333 shutdown_connections 2024-04-25T19:01:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.953+0000 7fb6b6c00640 1 --2- 172.21.15.45:0/2447852333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b0105fa0 0x7fb6b010db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.953+0000 7fb6b6c00640 1 --2- 172.21.15.45:0/2447852333 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6b0101880 0x7fb6b01058c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.953+0000 7fb6b6c00640 1 --2- 172.21.15.45:0/2447852333 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6b0100ed0 0x7fb6b01012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.954+0000 7fb6b6c00640 1 -- 172.21.15.45:0/2447852333 >> 172.21.15.45:0/2447852333 conn(0x7fb6b00fc790 msgr2=0x7fb6b00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.954+0000 7fb6b6c00640 1 -- 172.21.15.45:0/2447852333 shutdown_connections 2024-04-25T19:01:07.956 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.954+0000 7fb6b6c00640 1 -- 172.21.15.45:0/2447852333 wait complete. 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.955+0000 7fb6b6c00640 1 Processor -- start 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.955+0000 7fb6b6c00640 1 -- start start 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.955+0000 7fb6b6c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6b0100ed0 0x7fb6b01a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.955+0000 7fb6b6c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6b0105fa0 0x7fb6b01a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.955+0000 7fb6b6c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b019a610 0x7fb6b019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.955+0000 7fb6b6c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b0112050 con 0x7fb6b019a610 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.955+0000 7fb6b6c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b0111ed0 con 0x7fb6b0100ed0 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.955+0000 7fb6b6c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01121d0 con 0x7fb6b0105fa0 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.956+0000 7fb6b6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b019a610 0x7fb6b019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.956+0000 7fb6b6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b019a610 0x7fb6b019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55518/0 (socket says 172.21.15.45:55518) 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.956+0000 7fb6b6200640 1 -- 172.21.15.45:0/4110872898 learned_addr learned my addr 172.21.15.45:0/4110872898 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:07.958 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.956+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6b0105fa0 0x7fb6b01a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.959 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.956+0000 7fb6b5800640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6b0100ed0 0x7fb6b01a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.956+0000 7fb6b6200640 1 -- 172.21.15.45:0/4110872898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6b0105fa0 msgr2=0x7fb6b01a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.956+0000 7fb6b6200640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6b0105fa0 0x7fb6b01a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.956+0000 7fb6b6200640 1 -- 172.21.15.45:0/4110872898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6b0100ed0 msgr2=0x7fb6b01a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.956+0000 7fb6b6200640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6b0100ed0 0x7fb6b01a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.956+0000 7fb6b6200640 1 -- 172.21.15.45:0/4110872898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b019b130 con 0x7fb6b019a610 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.957+0000 7fb6b6200640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b019a610 0x7fb6b019aac0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800efd0 tx=0x7fb6a800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.957+0000 7fb6ae000640 1 -- 172.21.15.45:0/4110872898 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8003b70 con 0x7fb6b019a610 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.957+0000 7fb6ae000640 1 -- 172.21.15.45:0/4110872898 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a8003d10 con 0x7fb6b019a610 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.957+0000 7fb6b6c00640 1 -- 172.21.15.45:0/4110872898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b019b420 con 0x7fb6b019a610 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.957+0000 7fb6ae000640 1 -- 172.21.15.45:0/4110872898 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80108a0 con 0x7fb6b019a610 2024-04-25T19:01:07.960 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.957+0000 7fb6b6c00640 1 -- 172.21.15.45:0/4110872898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b01a7670 con 0x7fb6b019a610 2024-04-25T19:01:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.961+0000 7fb6bb00e740 1 Processor -- start 2024-04-25T19:01:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.961+0000 7fb6ae000640 1 -- 172.21.15.45:0/4110872898 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb6a8010a60 con 0x7fb6b019a610 2024-04-25T19:01:07.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.961+0000 7fb6bb00e740 1 -- start start 2024-04-25T19:01:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8909b0 0x5557dd890d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8912d0 0x5557dd89b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 0x5557dd89df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5557dd7f6900 con 0x5557dd8909b0 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5557dd79cef0 con 0x5557dd89bb90 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5557dd7f28c0 con 0x5557dd8912d0 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6b5800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8912d0 0x5557dd89b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6b4e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8909b0 0x5557dd890d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6b5800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8912d0 0x5557dd89b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:54712/0 (socket says 172.21.15.45:54712) 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6b5800640 1 -- 172.21.15.45:0/600711627 learned_addr learned my addr 172.21.15.45:0/600711627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.962+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/600711627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 0x5557dd89df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6b5800640 1 -- 172.21.15.45:0/600711627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 msgr2=0x5557dd89df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6b5800640 1 --2- 172.21.15.45:0/600711627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 0x5557dd89df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6b5800640 1 -- 172.21.15.45:0/600711627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8909b0 msgr2=0x5557dd890d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6b5800640 1 --2- 172.21.15.45:0/600711627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8909b0 0x5557dd890d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6b5800640 1 -- 172.21.15.45:0/600711627 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557dd509210 con 0x5557dd8912d0 2024-04-25T19:01:07.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6b5800640 1 --2- 172.21.15.45:0/600711627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8912d0 0x5557dd89b650 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009d20 tx=0x7fb6a4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3516ab517238f3d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6ae000640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb6980a3d80 0x7fb6980a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6a2a00640 1 -- 172.21.15.45:0/600711627 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a401e070 con 0x5557dd8912d0 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6ae000640 1 -- 172.21.15.45:0/4110872898 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fb6a80d0b70 con 0x7fb6b019a610 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6a2a00640 1 -- 172.21.15.45:0/600711627 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a4019ae0 con 0x5557dd8912d0 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.963+0000 7fb6a2a00640 1 -- 172.21.15.45:0/600711627 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a400ea50 con 0x5557dd8912d0 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.964+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb6980a3d80 0x7fb6980a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.964+0000 7fb6bb00e740 1 -- 172.21.15.45:0/600711627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8912d0 msgr2=0x5557dd89b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.964+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb6980a3d80 0x7fb6980a6240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb69c031440 tx=0x7fb69c041040 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.964+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/600711627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8912d0 0x5557dd89b650 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4009d20 tx=0x7fb6a4011810 comp rx=0 tx=0).stop 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.965+0000 7fb6bb00e740 1 -- 172.21.15.45:0/600711627 shutdown_connections 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.965+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/600711627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 0x5557dd89df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.965+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/600711627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8912d0 0x5557dd89b650 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.965+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/600711627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8909b0 0x5557dd890d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.965+0000 7fb6bb00e740 1 -- 172.21.15.45:0/600711627 >> 172.21.15.45:0/600711627 conn(0x5557dd88cec0 msgr2=0x5557dd88d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.965+0000 7fb6bb00e740 1 -- 172.21.15.45:0/600711627 shutdown_connections 2024-04-25T19:01:07.968 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.965+0000 7fb6bb00e740 1 -- 172.21.15.45:0/600711627 wait complete. 2024-04-25T19:01:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.965+0000 7fb6bb00e740 1 -- start start 2024-04-25T19:01:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.965+0000 7fb6bb00e740 1 Processor -- start 2024-04-25T19:01:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 0x5557dd898b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd899060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd89bb90 0x5557dd8995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5557dd89e7c0 con 0x5557dd89bb90 2024-04-25T19:01:07.971 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5557dd7b0310 con 0x5557dd8912d0 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5557dd89e940 con 0x5557dd8909b0 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6bb00e740 10 client.0 _ll_register_callbacks cb 0x5557dd77ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6b4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd899060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6b6200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd89bb90 0x5557dd8995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6b5800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 0x5557dd898b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6b4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd899060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:32910/0 (socket says 172.21.15.45:32910) 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.966+0000 7fb6b4e00640 1 -- 172.21.15.45:0/14282668 learned_addr learned my addr 172.21.15.45:0/14282668 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb6b5800640 1 -- 172.21.15.45:0/14282668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 msgr2=0x5557dd899060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb6b5800640 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd899060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb6b5800640 1 -- 172.21.15.45:0/14282668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd89bb90 msgr2=0x5557dd8995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:07.972 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb6b5800640 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd89bb90 0x5557dd8995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb6b5800640 1 -- 172.21.15.45:0/14282668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557dd5d5620 con 0x5557dd8909b0 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd899060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb6b5800640 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 0x5557dd898b20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400e590 tx=0x7fb6a4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d45a9c23401ab713 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb697400640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb697400640 1 -- 172.21.15.45:0/14282668 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a401e070 con 0x5557dd8909b0 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb697400640 1 -- 172.21.15.45:0/14282668 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fb6a4019430 con 0x5557dd8909b0 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.967+0000 7fb6bb00e740 1 -- 172.21.15.45:0/14282668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a4019ae0 con 0x5557dd8909b0 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb697400640 1 -- 172.21.15.45:0/14282668 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb6a4030050 con 0x5557dd8909b0 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb6bb00e740 10 client.25441 fetch_fsmap learned FSMap version 27 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb6bb00e740 1 -- 172.21.15.45:0/14282668 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557dd8aaa20 con 0x5557dd8909b0 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb697400640 1 -- 172.21.15.45:0/14282668 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a4019ae0 con 0x5557dd8909b0 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb697400640 1 -- 172.21.15.45:0/14282668 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 27) ==== 8342+0+0 (secure 0 0 0) 0x7fb6a400e790 con 0x5557dd8909b0 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb6bb00e740 10 client.25441 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T19:01:07.973 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb6bb00e740 10 client.25441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:01:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb6bb00e740 20 client.25441 populate_metadata read hostname 'smithi045' 2024-04-25T19:01:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5557dd8abe20 0x5557dd8ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb6bb00e740 4 client.25441 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:01:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.968+0000 7fb6bb00e740 1 -- 172.21.15.45:0/14282668 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557dd80a8b0 con 0x5557dd8abe20 2024-04-25T19:01:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.969+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5557dd8abe20 0x5557dd8ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.969+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5557dd8abe20 0x5557dd8ae210 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8cf843b7cc59b6 server_cookie=6fc7e8a79bd8b1d2 in_seq=0 out_seq=0 2024-04-25T19:01:07.974 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.969+0000 7fb697400640 10 client.25441 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.970+0000 7fb697400640 1 -- 172.21.15.45:0/14282668 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557dd80a8b0 con 0x5557dd8abe20 2024-04-25T19:01:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.970+0000 7fb697400640 10 client.25441 handle_command_reply: tid=0 2024-04-25T19:01:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.970+0000 7fb6bb00e740 1 client.25441 shutdown 2024-04-25T19:01:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 1 -- 172.21.15.45:0/14282668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 msgr2=0x5557dd898b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 0x5557dd898b20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400e590 tx=0x7fb6a4004270 comp rx=0 tx=0).stop 2024-04-25T19:01:07.975 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 1 -- 172.21.15.45:0/14282668 shutdown_connections 2024-04-25T19:01:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5557dd8abe20 0x5557dd8ae210 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd89bb90 0x5557dd8995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd899060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/14282668 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 0x5557dd898b20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb697400640 0 client.25441 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 1 -- 172.21.15.45:0/14282668 >> 172.21.15.45:0/14282668 conn(0x5557dd88cec0 msgr2=0x5557dd892c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 1 -- 172.21.15.45:0/14282668 shutdown_connections 2024-04-25T19:01:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 1 -- 172.21.15.45:0/14282668 wait complete. 2024-04-25T19:01:07.976 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.971+0000 7fb6bb00e740 20 client.25441 trim_cache size 0 max 16384 2024-04-25T19:01:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.990+0000 7fb6bb00e740 1 Processor -- start 2024-04-25T19:01:07.992 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.990+0000 7fb6bb00e740 1 -- start start 2024-04-25T19:01:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 0x5557dd8a7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8912d0 0x5557dd892a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 0x5557dd8b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01b2470 con 0x5557dd8912d0 2024-04-25T19:01:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01afd40 con 0x5557dd89bb90 2024-04-25T19:01:07.993 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01adb90 con 0x5557dd8909b0 2024-04-25T19:01:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6b6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 0x5557dd8b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6b6200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 0x5557dd8b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:32914/0 (socket says 172.21.15.45:32914) 2024-04-25T19:01:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6b6200640 1 -- 172.21.15.45:0/858760400 learned_addr learned my addr 172.21.15.45:0/858760400 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/858760400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8912d0 0x5557dd892a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.994 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6b5800640 1 --2- 172.21.15.45:0/858760400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 0x5557dd8a7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6b6200640 1 -- 172.21.15.45:0/858760400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 msgr2=0x5557dd8a7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6b6200640 1 --2- 172.21.15.45:0/858760400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 0x5557dd8a7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6b6200640 1 -- 172.21.15.45:0/858760400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8912d0 msgr2=0x5557dd892a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6b6200640 1 --2- 172.21.15.45:0/858760400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8912d0 0x5557dd892a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.991+0000 7fb6b6200640 1 -- 172.21.15.45:0/858760400 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557dd89e340 con 0x5557dd89bb90 2024-04-25T19:01:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.992+0000 7fb6b6200640 1 --2- 172.21.15.45:0/858760400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 0x5557dd8b39a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6a809ca40 tx=0x7fb6a8009bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff4c8665f4e8e37e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.992+0000 7fb6a2000640 1 -- 172.21.15.45:0/858760400 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80b5070 con 0x5557dd89bb90 2024-04-25T19:01:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.992+0000 7fb6a2000640 1 -- 172.21.15.45:0/858760400 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a809cab0 con 0x5557dd89bb90 2024-04-25T19:01:07.995 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.992+0000 7fb6a2000640 1 -- 172.21.15.45:0/858760400 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80ce4e0 con 0x5557dd89bb90 2024-04-25T19:01:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.993+0000 7fb6bb00e740 1 -- 172.21.15.45:0/858760400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 msgr2=0x5557dd8b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.993+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/858760400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 0x5557dd8b39a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb6a809ca40 tx=0x7fb6a8009bc0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.993+0000 7fb6bb00e740 1 -- 172.21.15.45:0/858760400 shutdown_connections 2024-04-25T19:01:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.993+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/858760400 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd89bb90 0x5557dd8b39a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.993+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/858760400 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8912d0 0x5557dd892a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.998 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.993+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/858760400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd8909b0 0x5557dd8a7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.993+0000 7fb6bb00e740 1 -- 172.21.15.45:0/858760400 >> 172.21.15.45:0/858760400 conn(0x5557dd88cec0 msgr2=0x5557dd892360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.993+0000 7fb6bb00e740 1 -- 172.21.15.45:0/858760400 shutdown_connections 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.993+0000 7fb6bb00e740 1 -- 172.21.15.45:0/858760400 wait complete. 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.993+0000 7fb6bb00e740 1 -- start start 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.994+0000 7fb6bb00e740 1 Processor -- start 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.994+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8909b0 0x5557dd8a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.994+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd8a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.994+0000 7fb6bb00e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd89bb90 0x5557dd8a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.994+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01ae200 con 0x5557dd8909b0 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.994+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01af670 con 0x5557dd8912d0 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.994+0000 7fb6bb00e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b01ae8d0 con 0x5557dd89bb90 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.994+0000 7fb6bb00e740 10 client.0 _ll_register_callbacks cb 0x7fb6a4009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.994+0000 7fb6b4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd8a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:07.999 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.994+0000 7fb6b5800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8909b0 0x5557dd8a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.995+0000 7fb6b4e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd8a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:32922/0 (socket says 172.21.15.45:32922) 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.995+0000 7fb6b4e00640 1 -- 172.21.15.45:0/2521752238 learned_addr learned my addr 172.21.15.45:0/2521752238 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.995+0000 7fb6b6200640 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd89bb90 0x5557dd8a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.995+0000 7fb6b4e00640 1 -- 172.21.15.45:0/2521752238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd89bb90 msgr2=0x5557dd8a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.995+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd89bb90 0x5557dd8a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.995+0000 7fb6b4e00640 1 -- 172.21.15.45:0/2521752238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8909b0 msgr2=0x5557dd8a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.995+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8909b0 0x5557dd8a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.995+0000 7fb6b4e00640 1 -- 172.21.15.45:0/2521752238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557dd8a1400 con 0x5557dd8912d0 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.995+0000 7fb6b4e00640 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd8a3d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb69c02a910 tx=0x7fb69c02a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b05c91175903b963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.995+0000 7fb697400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.996+0000 7fb697400640 1 -- 172.21.15.45:0/2521752238 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c05d060 con 0x5557dd8912d0 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.996+0000 7fb6bb00e740 1 -- 172.21.15.45:0/2521752238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6a809cab0 con 0x5557dd8912d0 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.996+0000 7fb697400640 1 -- 172.21.15.45:0/2521752238 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fb69c031ab0 con 0x5557dd8912d0 2024-04-25T19:01:08.000 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.996+0000 7fb697400640 1 -- 172.21.15.45:0/2521752238 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb69c046070 con 0x5557dd8912d0 2024-04-25T19:01:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.996+0000 7fb6bb00e740 10 client.25430 fetch_fsmap learned FSMap version 27 2024-04-25T19:01:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.996+0000 7fb6bb00e740 1 -- 172.21.15.45:0/2521752238 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5557dd911c90 con 0x5557dd8912d0 2024-04-25T19:01:08.001 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.996+0000 7fb697400640 1 -- 172.21.15.45:0/2521752238 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6a809cab0 con 0x5557dd8912d0 2024-04-25T19:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.997+0000 7fb697400640 1 -- 172.21.15.45:0/2521752238 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 27) ==== 8342+0+0 (secure 0 0 0) 0x7fb69c057070 con 0x5557dd8912d0 2024-04-25T19:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.997+0000 7fb6bb00e740 10 client.25430 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T19:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.997+0000 7fb6bb00e740 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.997+0000 7fb6bb00e740 20 client.25430 populate_metadata read hostname 'smithi045' 2024-04-25T19:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.997+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5557dd912420 0x5557dd914840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.997+0000 7fb6bb00e740 4 client.25430 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.997+0000 7fb6bb00e740 1 -- 172.21.15.45:0/2521752238 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5557dd892090 con 0x5557dd912420 2024-04-25T19:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.997+0000 7fb6b5800640 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5557dd912420 0x5557dd914840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.998+0000 7fb6b5800640 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5557dd912420 0x5557dd914840 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfeff08783749582 server_cookie=f6c2c5537f4a93e3 in_seq=0 out_seq=0 2024-04-25T19:01:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:07.998+0000 7fb697400640 10 client.25430 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:08.314 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:08 smithi045 ceph-mon[27776]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T19:01:08.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:08 smithi062 ceph-mon[32844]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T19:01:08.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:08 smithi079 ceph-mon[33002]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-04-25T19:01:08.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.610+0000 7f46cba00700 1 -- 172.21.15.45:0/1009678303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c4109f90 msgr2=0x7f46c4111b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.610+0000 7f46cba00700 1 --2- 172.21.15.45:0/1009678303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c4109f90 0x7f46c4111b60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009f90 tx=0x7f46bc00b330 comp rx=0 tx=0).stop 2024-04-25T19:01:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.610+0000 7f46cba00700 1 -- 172.21.15.45:0/1009678303 shutdown_connections 2024-04-25T19:01:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.610+0000 7f46cba00700 1 --2- 172.21.15.45:0/1009678303 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c4109f90 0x7f46c4111b60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.610+0000 7f46cba00700 1 --2- 172.21.15.45:0/1009678303 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4104eb0 0x7f46c4105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.610+0000 7f46cba00700 1 --2- 172.21.15.45:0/1009678303 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c4105860 0x7f46c41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.610+0000 7f46cba00700 1 -- 172.21.15.45:0/1009678303 >> 172.21.15.45:0/1009678303 conn(0x7f46c4100720 msgr2=0x7f46c4102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.610+0000 7f46cba00700 1 -- 172.21.15.45:0/1009678303 shutdown_connections 2024-04-25T19:01:08.613 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.610+0000 7f46cba00700 1 -- 172.21.15.45:0/1009678303 wait complete. 2024-04-25T19:01:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.611+0000 7f46cba00700 1 Processor -- start 2024-04-25T19:01:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.612+0000 7f46cba00700 1 -- start start 2024-04-25T19:01:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.612+0000 7f46cba00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4104eb0 0x7f46c419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.612+0000 7f46cba00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c4105860 0x7f46c419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:08.614 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.612+0000 7f46cba00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c4109f90 0x7f46c419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.612+0000 7f46cba00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4114e80 con 0x7f46c4104eb0 2024-04-25T19:01:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.612+0000 7f46cba00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4114d00 con 0x7f46c4105860 2024-04-25T19:01:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.612+0000 7f46cba00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46c4115000 con 0x7f46c4109f90 2024-04-25T19:01:08.615 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c9000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4104eb0 0x7f46c419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c3e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c4105860 0x7f46c419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c9000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4104eb0 0x7f46c419b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55570/0 (socket says 172.21.15.45:55570) 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c9000700 1 -- 172.21.15.45:0/1923278114 learned_addr learned my addr 172.21.15.45:0/1923278114 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c9a00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c4109f90 0x7f46c419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c9000700 1 -- 172.21.15.45:0/1923278114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c4109f90 msgr2=0x7f46c419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c9000700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c4109f90 0x7f46c419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c9000700 1 -- 172.21.15.45:0/1923278114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c4105860 msgr2=0x7f46c419b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c9000700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c4105860 0x7f46c419b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c9000700 1 -- 172.21.15.45:0/1923278114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c41a02e0 con 0x7f46c4104eb0 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c9a00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c4109f90 0x7f46c419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.613+0000 7f46c3e00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c4105860 0x7f46c419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:08.616 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.614+0000 7f46c9000700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4104eb0 0x7f46c419b2d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46b800e9c0 tx=0x7f46b800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.614+0000 7f46c1600700 1 -- 172.21.15.45:0/1923278114 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8004a30 con 0x7f46c4104eb0 2024-04-25T19:01:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.614+0000 7f46cba00700 1 -- 172.21.15.45:0/1923278114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c41a05d0 con 0x7f46c4104eb0 2024-04-25T19:01:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.614+0000 7f46c1600700 1 -- 172.21.15.45:0/1923278114 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46b800cc70 con 0x7f46c4104eb0 2024-04-25T19:01:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.614+0000 7f46cba00700 1 -- 172.21.15.45:0/1923278114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c41a7e50 con 0x7f46c4104eb0 2024-04-25T19:01:08.617 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.614+0000 7f46c1600700 1 -- 172.21.15.45:0/1923278114 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b80054a0 con 0x7f46c4104eb0 2024-04-25T19:01:08.618 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.615+0000 7f46cba00700 1 -- 172.21.15.45:0/1923278114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a4005330 con 0x7f46c4104eb0 2024-04-25T19:01:08.620 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.618+0000 7f46c1600700 1 -- 172.21.15.45:0/1923278114 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f46b8004bd0 con 0x7f46c4104eb0 2024-04-25T19:01:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.619+0000 7f46c1600700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46ac0a3db0 0x7f46ac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:08.621 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.619+0000 7f46c1600700 1 -- 172.21.15.45:0/1923278114 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f46b80d0a90 con 0x7f46c4104eb0 2024-04-25T19:01:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.619+0000 7f46c3e00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46ac0a3db0 0x7f46ac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:08.622 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.620+0000 7f46c3e00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46ac0a3db0 0x7f46ac0a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46c410b550 tx=0x7f46b0009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:08.624 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.622+0000 7f46c1600700 1 -- 172.21.15.45:0/1923278114 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46b8053ff0 con 0x7f46c4104eb0 2024-04-25T19:01:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.901+0000 7f46cba00700 1 -- 172.21.15.45:0/1923278114 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46a4005620 con 0x7f46c4104eb0 2024-04-25T19:01:08.904 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.902+0000 7f46c1600700 1 -- 172.21.15.45:0/1923278114 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11853 (secure 0 0 0) 0x7f46b800ce20 con 0x7f46c4104eb0 2024-04-25T19:01:08.905 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:01:08.905 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T19:01:05.422246+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24512]},"id":1}]} 2024-04-25T19:01:08.908 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 -- 172.21.15.45:0/1923278114 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46ac0a3db0 msgr2=0x7f46ac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46ac0a3db0 0x7f46ac0a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f46c410b550 tx=0x7f46b0009260 comp rx=0 tx=0).stop 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 -- 172.21.15.45:0/1923278114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4104eb0 msgr2=0x7f46c419b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4104eb0 0x7f46c419b2d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f46b800e9c0 tx=0x7f46b800ecd0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 -- 172.21.15.45:0/1923278114 shutdown_connections 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46ac0a3db0 0x7f46ac0a6270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46c4109f90 0x7f46c419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46c4104eb0 0x7f46c419b2d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 --2- 172.21.15.45:0/1923278114 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46c4105860 0x7f46c419b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 -- 172.21.15.45:0/1923278114 >> 172.21.15.45:0/1923278114 conn(0x7f46c4100720 msgr2=0x7f46c4101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 -- 172.21.15.45:0/1923278114 shutdown_connections 2024-04-25T19:01:08.909 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:08.906+0000 7f46cba00700 1 -- 172.21.15.45:0/1923278114 wait complete. 2024-04-25T19:01:08.910 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 27 2024-04-25T19:01:09.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:09 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/1923278114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:09.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:09 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/1923278114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:09.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:09 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/1923278114' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:09.822 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db / --timeout 60 --expiration 89.4 --await-for 42.0' 2024-04-25T19:01:09.822 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24524 quiesce db / --timeout 60 --expiration 89.4 --await-for 42.0 2024-04-25T19:01:10.287 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:10.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:10 smithi062 ceph-mon[32844]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T19:01:10.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:10 smithi079 ceph-mon[33002]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T19:01:10.443 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:10 smithi045 ceph-mon[27776]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T19:01:10.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb697400640 1 -- 172.21.15.45:0/2521752238 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5557dd892090 con 0x5557dd912420 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb697400640 10 client.25430 handle_command_reply: tid=0 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 client.25430 shutdown 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 -- 172.21.15.45:0/2521752238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 msgr2=0x5557dd8a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd8a3d00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb69c02a910 tx=0x7fb69c02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 -- 172.21.15.45:0/2521752238 shutdown_connections 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5557dd912420 0x5557dd914840 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5557dd89bb90 0x5557dd8a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5557dd8912d0 0x5557dd8a3d00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 --2- 172.21.15.45:0/2521752238 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5557dd8909b0 0x5557dd8a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb697400640 0 client.25430 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:10.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 -- 172.21.15.45:0/2521752238 >> 172.21.15.45:0/2521752238 conn(0x5557dd88cec0 msgr2=0x5557dd88c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 -- 172.21.15.45:0/2521752238 shutdown_connections 2024-04-25T19:01:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.604+0000 7fb6bb00e740 1 -- 172.21.15.45:0/2521752238 wait complete. 2024-04-25T19:01:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.605+0000 7fb6bb00e740 20 client.25430 trim_cache size 0 max 16384 2024-04-25T19:01:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:01:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:01:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:01:10.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:01:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.605+0000 7fb6b6c00640 1 -- 172.21.15.45:0/4110872898 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb6980a3d80 msgr2=0x7fb6980a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.605+0000 7fb6b6c00640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb6980a3d80 0x7fb6980a6240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb69c031440 tx=0x7fb69c041040 comp rx=0 tx=0).stop 2024-04-25T19:01:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.605+0000 7fb6b6c00640 1 -- 172.21.15.45:0/4110872898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b019a610 msgr2=0x7fb6b019aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.605+0000 7fb6b6c00640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b019a610 0x7fb6b019aac0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800efd0 tx=0x7fb6a800c6a0 comp rx=0 tx=0).stop 2024-04-25T19:01:10.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.605+0000 7fb6b6c00640 1 -- 172.21.15.45:0/4110872898 shutdown_connections 2024-04-25T19:01:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.605+0000 7fb6b6c00640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb6980a3d80 0x7fb6980a6240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.605+0000 7fb6b6c00640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb6b019a610 0x7fb6b019aac0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.606+0000 7fb6b6c00640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb6b0105fa0 0x7fb6b01a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.606+0000 7fb6b6c00640 1 --2- 172.21.15.45:0/4110872898 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb6b0100ed0 0x7fb6b01a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.606+0000 7fb6b6c00640 1 -- 172.21.15.45:0/4110872898 >> 172.21.15.45:0/4110872898 conn(0x7fb6b00fc790 msgr2=0x7fb6b00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.606+0000 7fb6b6c00640 1 -- 172.21.15.45:0/4110872898 shutdown_connections 2024-04-25T19:01:10.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:10.606+0000 7fb6b6c00640 1 -- 172.21.15.45:0/4110872898 wait complete. 2024-04-25T19:01:12.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:12 smithi045 ceph-mon[27776]: pgmap v59: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.1 MiB/s wr, 27 op/s 2024-04-25T19:01:12.205 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:12 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:01:12.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:12 smithi062 ceph-mon[32844]: pgmap v59: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.1 MiB/s wr, 27 op/s 2024-04-25T19:01:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:12 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:01:12.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:12 smithi079 ceph-mon[33002]: pgmap v59: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 1.1 MiB/s wr, 27 op/s 2024-04-25T19:01:12.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:12 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:01:12.550 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.548+0000 7f9ad1e00700 1 -- 172.21.15.45:0/2252043415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc104ec0 msgr2=0x7f9acc1052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.548+0000 7f9ad1e00700 1 --2- 172.21.15.45:0/2252043415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc104ec0 0x7f9acc1052a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0099c0 tx=0x7f9abc009cd0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.548+0000 7f9ad1e00700 1 -- 172.21.15.45:0/2252043415 shutdown_connections 2024-04-25T19:01:12.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.548+0000 7f9ad1e00700 1 --2- 172.21.15.45:0/2252043415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc109fa0 0x7f9acc111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.548+0000 7f9ad1e00700 1 --2- 172.21.15.45:0/2252043415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc104ec0 0x7f9acc1052a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.548+0000 7f9ad1e00700 1 --2- 172.21.15.45:0/2252043415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc105870 0x7f9acc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.548+0000 7f9ad1e00700 1 -- 172.21.15.45:0/2252043415 >> 172.21.15.45:0/2252043415 conn(0x7f9acc100750 msgr2=0x7f9acc102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:12.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.548+0000 7f9ad1e00700 1 -- 172.21.15.45:0/2252043415 shutdown_connections 2024-04-25T19:01:12.551 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.548+0000 7f9ad1e00700 1 -- 172.21.15.45:0/2252043415 wait complete. 2024-04-25T19:01:12.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.549+0000 7f9ad1e00700 1 Processor -- start 2024-04-25T19:01:12.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.549+0000 7f9ad1e00700 1 -- start start 2024-04-25T19:01:12.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9ad1e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc104ec0 0x7f9acc19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9ad1e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc105870 0x7f9acc19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.552 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9ad1e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc109fa0 0x7f9acc19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9ad1e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc114e30 con 0x7f9acc109fa0 2024-04-25T19:01:12.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9ad1e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc114cb0 con 0x7f9acc105870 2024-04-25T19:01:12.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9ad1e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc114fb0 con 0x7f9acc104ec0 2024-04-25T19:01:12.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9acaa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc104ec0 0x7f9acc19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9acb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc109fa0 0x7f9acc19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9acaa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc104ec0 0x7f9acc19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:54782/0 (socket says 172.21.15.45:54782) 2024-04-25T19:01:12.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9acaa00700 1 -- 172.21.15.45:0/3601474568 learned_addr learned my addr 172.21.15.45:0/3601474568 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:12.553 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.550+0000 7f9aca000700 1 --2- 172.21.15.45:0/3601474568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc105870 0x7f9acc19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.551+0000 7f9acaa00700 1 -- 172.21.15.45:0/3601474568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc105870 msgr2=0x7f9acc19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.551+0000 7f9acaa00700 1 --2- 172.21.15.45:0/3601474568 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f9acc105870 0x7f9acc19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.551+0000 7f9acaa00700 1 -- 172.21.15.45:0/3601474568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc109fa0 msgr2=0x7f9acc19fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.551+0000 7f9acaa00700 1 --2- 172.21.15.45:0/3601474568 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9acc109fa0 0x7f9acc19fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.551+0000 7f9acaa00700 1 -- 172.21.15.45:0/3601474568 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc1a0290 con 0x7f9acc104ec0 2024-04-25T19:01:12.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.551+0000 7f9acaa00700 1 --2- 172.21.15.45:0/3601474568 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc104ec0 0x7f9acc19b280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9abc006010 tx=0x7f9abc005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:12.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.551+0000 7f9ad0a00700 1 -- 172.21.15.45:0/3601474568 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc026070 con 0x7f9acc104ec0 2024-04-25T19:01:12.554 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.551+0000 7f9ad0a00700 1 -- 172.21.15.45:0/3601474568 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9abc003c70 con 0x7f9acc104ec0 2024-04-25T19:01:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.551+0000 7f9ad1e00700 1 -- 172.21.15.45:0/3601474568 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc1a0520 con 0x7f9acc104ec0 2024-04-25T19:01:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.551+0000 7f9ad0a00700 1 -- 172.21.15.45:0/3601474568 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00fbb0 con 0x7f9acc104ec0 2024-04-25T19:01:12.555 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.552+0000 7f9ad1e00700 1 -- 172.21.15.45:0/3601474568 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc1a7e00 con 0x7f9acc104ec0 2024-04-25T19:01:12.557 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.555+0000 7f9ad0a00700 1 -- 172.21.15.45:0/3601474568 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f9abc029070 con 0x7f9acc104ec0 2024-04-25T19:01:12.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.557+0000 7f9adc7cab80 1 Processor -- start 2024-04-25T19:01:12.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.557+0000 7f9adc7cab80 1 -- start start 2024-04-25T19:01:12.559 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.557+0000 7f9ad0a00700 1 --2- 172.21.15.45:0/3601474568 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f9ab00a3ed0 0x7f9ab00a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.557+0000 7f9ad0a00700 1 -- 172.21.15.45:0/3601474568 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f9abc0d0f20 con 0x7f9acc104ec0 2024-04-25T19:01:12.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.557+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395ddd0 0x55bc3395e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.557+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33969020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.557+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc33969560 0x55bc3396b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc338b1020 con 0x55bc33969560 2024-04-25T19:01:12.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bc338578c0 con 0x55bc3395e6f0 2024-04-25T19:01:12.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bc338acfe0 con 0x55bc3395ddd0 2024-04-25T19:01:12.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9acb400700 1 --2- 172.21.15.45:0/3601474568 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f9ab00a3ed0 0x7f9ab00a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9aca000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33969020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9aca000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33969020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:32962/0 (socket says 172.21.15.45:32962) 2024-04-25T19:01:12.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9aca000700 1 -- 172.21.15.45:0/3264454712 learned_addr learned my addr 172.21.15.45:0/3264454712 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:12.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9acaa00700 1 --2- 172.21.15.45:0/3264454712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395ddd0 0x55bc3395e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9aca000700 1 -- 172.21.15.45:0/3264454712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395ddd0 msgr2=0x55bc3395e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:12.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9aca000700 1 --2- 172.21.15.45:0/3264454712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395ddd0 0x55bc3395e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9aca000700 1 -- 172.21.15.45:0/3264454712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc33969560 msgr2=0x55bc3396b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:12.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9aca000700 1 --2- 172.21.15.45:0/3264454712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc33969560 0x55bc3396b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.558+0000 7f9aca000700 1 -- 172.21.15.45:0/3264454712 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc3384d810 con 0x55bc3395e6f0 2024-04-25T19:01:12.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.559+0000 7f9aca000700 1 --2- 172.21.15.45:0/3264454712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33969020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009cf0 tx=0x7f9ab8011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3298aa8fbf69b4d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:12.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.559+0000 7f9ac0c00700 1 -- 172.21.15.45:0/3264454712 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8011cc0 con 0x55bc3395e6f0 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.559+0000 7f9ac0c00700 1 -- 172.21.15.45:0/3264454712 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab8004500 con 0x55bc3395e6f0 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.559+0000 7f9ac0c00700 1 -- 172.21.15.45:0/3264454712 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8019640 con 0x55bc3395e6f0 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.560+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3264454712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 msgr2=0x55bc33969020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.560+0000 7f9acb400700 1 --2- 172.21.15.45:0/3601474568 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f9ab00a3ed0 0x7f9ab00a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55bc3395f7a0 tx=0x7f9ac4009920 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.560+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/3264454712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33969020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009cf0 tx=0x7f9ab8011440 comp rx=0 tx=0).stop 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.560+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3264454712 shutdown_connections 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.560+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/3264454712 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc33969560 0x55bc3396b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.560+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/3264454712 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395ddd0 0x55bc3395e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.560+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/3264454712 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33969020 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.560+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3264454712 >> 172.21.15.45:0/3264454712 conn(0x55bc3388d0e0 msgr2=0x55bc3395a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.560+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3264454712 shutdown_connections 2024-04-25T19:01:12.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.560+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3264454712 wait complete. 2024-04-25T19:01:12.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.561+0000 7f9adc7cab80 1 -- start start 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.561+0000 7f9adc7cab80 1 Processor -- start 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 0x55bc33968c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395e6f0 0x55bc33965f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc33969560 0x55bc33966480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55bc3396bff0 con 0x55bc3395ddd0 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55bc3386a9f0 con 0x55bc33969560 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55bc3396c170 con 0x55bc3395e6f0 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9adc7cab80 10 client.0 _ll_register_callbacks cb 0x55bc338498b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9acb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc33969560 0x55bc33966480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9acb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc33969560 0x55bc33966480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:32966/0 (socket says 172.21.15.45:32966) 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9acb400700 1 -- 172.21.15.45:0/2085371529 learned_addr learned my addr 172.21.15.45:0/2085371529 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9aca000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395e6f0 0x55bc33965f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.562+0000 7f9acaa00700 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 0x55bc33968c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.563+0000 7f9acb400700 1 -- 172.21.15.45:0/2085371529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395e6f0 msgr2=0x55bc33965f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:12.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.563+0000 7f9acb400700 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395e6f0 0x55bc33965f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.563+0000 7f9acb400700 1 -- 172.21.15.45:0/2085371529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 msgr2=0x55bc33968c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:12.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.563+0000 7f9acb400700 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 0x55bc33968c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.563+0000 7f9acb400700 1 -- 172.21.15.45:0/2085371529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc337fea60 con 0x55bc33969560 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.563+0000 7f9acb400700 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc33969560 0x55bc33966480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4017e60 tx=0x7f9ac4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b855d1bd333f4a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.563+0000 7f9ab5600700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.563+0000 7f9ab5600700 1 -- 172.21.15.45:0/2085371529 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4026070 con 0x55bc33969560 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.563+0000 7f9ab5600700 1 -- 172.21.15.45:0/2085371529 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f9ac4009020 con 0x55bc33969560 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.563+0000 7f9adc7cab80 1 -- 172.21.15.45:0/2085371529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab8004500 con 0x55bc33969560 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.564+0000 7f9ab5600700 1 -- 172.21.15.45:0/2085371529 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9ac4031050 con 0x55bc33969560 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.564+0000 7f9adc7cab80 10 client.25448 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.564+0000 7f9adc7cab80 1 -- 172.21.15.45:0/2085371529 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc33847c80 con 0x55bc33969560 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.564+0000 7f9ab5600700 1 -- 172.21.15.45:0/2085371529 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab8004500 con 0x55bc33969560 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.564+0000 7f9ab5600700 1 -- 172.21.15.45:0/2085371529 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f9ac4021070 con 0x55bc33969560 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.564+0000 7f9adc7cab80 10 client.25448 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:12.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.564+0000 7f9adc7cab80 10 client.25448 resolve_mds: validated gid 24524 aka daemon mds.b 2024-04-25T19:01:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.564+0000 7f9adc7cab80 20 client.25448 populate_metadata read hostname 'smithi045' 2024-04-25T19:01:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.564+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bc33978c80 0x55bc3397b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.565+0000 7f9adc7cab80 4 client.25448 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:01:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.565+0000 7f9adc7cab80 1 -- 172.21.15.45:0/2085371529 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc338be810 con 0x55bc33978c80 2024-04-25T19:01:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.565+0000 7f9acaa00700 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bc33978c80 0x55bc3397b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.565+0000 7f9acaa00700 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bc33978c80 0x55bc3397b0e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5051750b667210cf server_cookie=74c4401fad8ca945 in_seq=0 out_seq=0 2024-04-25T19:01:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.565+0000 7f9ab5600700 10 client.25448 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9ab5600700 1 -- 172.21.15.45:0/2085371529 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bc338be810 con 0x55bc33978c80 2024-04-25T19:01:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9ab5600700 10 client.25448 handle_command_reply: tid=0 2024-04-25T19:01:12.569 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9adc7cab80 1 client.25448 shutdown 2024-04-25T19:01:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9adc7cab80 1 -- 172.21.15.45:0/2085371529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc33969560 msgr2=0x55bc33966480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc33969560 0x55bc33966480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4017e60 tx=0x7f9ac4004ad0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9adc7cab80 1 -- 172.21.15.45:0/2085371529 shutdown_connections 2024-04-25T19:01:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bc33978c80 0x55bc3397b0e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc33969560 0x55bc33966480 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 0x55bc33968c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/2085371529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395e6f0 0x55bc33965f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9ab5600700 0 client.25448 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.566+0000 7f9adc7cab80 1 -- 172.21.15.45:0/2085371529 >> 172.21.15.45:0/2085371529 conn(0x55bc3388d0e0 msgr2=0x55bc33959ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.567+0000 7f9adc7cab80 1 -- 172.21.15.45:0/2085371529 shutdown_connections 2024-04-25T19:01:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.567+0000 7f9adc7cab80 1 -- 172.21.15.45:0/2085371529 wait complete. 2024-04-25T19:01:12.571 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.567+0000 7f9adc7cab80 20 client.25448 trim_cache size 0 max 16384 2024-04-25T19:01:12.590 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.588+0000 7f9adc7cab80 1 Processor -- start 2024-04-25T19:01:12.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9adc7cab80 1 -- start start 2024-04-25T19:01:12.591 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 0x55bc33974ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33971400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc33969560 0x55bc33971940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc1b1c60 con 0x55bc3395ddd0 2024-04-25T19:01:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc1af4f0 con 0x55bc3395e6f0 2024-04-25T19:01:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc1ad340 con 0x55bc33969560 2024-04-25T19:01:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9acaa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 0x55bc33974ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9aca000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33971400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9acaa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 0x55bc33974ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:55618/0 (socket says 172.21.15.45:55618) 2024-04-25T19:01:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9acaa00700 1 -- 172.21.15.45:0/1620782229 learned_addr learned my addr 172.21.15.45:0/1620782229 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:12.592 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.589+0000 7f9acb400700 1 --2- 172.21.15.45:0/1620782229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc33969560 0x55bc33971940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.590+0000 7f9aca000700 1 -- 172.21.15.45:0/1620782229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc33969560 msgr2=0x55bc33971940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.590+0000 7f9aca000700 1 --2- 172.21.15.45:0/1620782229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc33969560 0x55bc33971940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.590+0000 7f9aca000700 1 -- 172.21.15.45:0/1620782229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 msgr2=0x55bc33974ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.590+0000 7f9aca000700 1 --2- 172.21.15.45:0/1620782229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 0x55bc33974ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.590+0000 7f9aca000700 1 -- 172.21.15.45:0/1620782229 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc3395ed70 con 0x55bc3395e6f0 2024-04-25T19:01:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.590+0000 7f9acb400700 1 --2- 172.21.15.45:0/1620782229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc33969560 0x55bc33971940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.590+0000 7f9aca000700 1 --2- 172.21.15.45:0/1620782229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33971400 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8005e90 tx=0x7f9ab8009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d233b0c6b66a14fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.590+0000 7f9ab7e00700 1 -- 172.21.15.45:0/1620782229 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab8029070 con 0x55bc3395e6f0 2024-04-25T19:01:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.590+0000 7f9ab7e00700 1 -- 172.21.15.45:0/1620782229 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab8003e30 con 0x55bc3395e6f0 2024-04-25T19:01:12.594 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.590+0000 7f9ab7e00700 1 -- 172.21.15.45:0/1620782229 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab80054b0 con 0x55bc3395e6f0 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.591+0000 7f9adc7cab80 1 -- 172.21.15.45:0/1620782229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 msgr2=0x55bc33971400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.591+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/1620782229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33971400 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8005e90 tx=0x7f9ab8009570 comp rx=0 tx=0).stop 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.591+0000 7f9adc7cab80 1 -- 172.21.15.45:0/1620782229 shutdown_connections 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.591+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/1620782229 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc33969560 0x55bc33971940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.591+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/1620782229 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc3395ddd0 0x55bc33974ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.591+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/1620782229 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395e6f0 0x55bc33971400 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.591+0000 7f9adc7cab80 1 -- 172.21.15.45:0/1620782229 >> 172.21.15.45:0/1620782229 conn(0x55bc3388d0e0 msgr2=0x55bc33959490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.591+0000 7f9adc7cab80 1 -- 172.21.15.45:0/1620782229 shutdown_connections 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.591+0000 7f9adc7cab80 1 -- 172.21.15.45:0/1620782229 wait complete. 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.592+0000 7f9adc7cab80 1 -- start start 2024-04-25T19:01:12.595 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.592+0000 7f9adc7cab80 1 Processor -- start 2024-04-25T19:01:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.592+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395ddd0 0x55bc33970020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.592+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395e6f0 0x55bc33970560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9adc7cab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc33969560 0x55bc33970aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc1ad9b0 con 0x55bc33969560 2024-04-25T19:01:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc1aee20 con 0x55bc3395ddd0 2024-04-25T19:01:12.597 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9adc7cab80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc1ae080 con 0x55bc3395e6f0 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9adc7cab80 10 client.0 _ll_register_callbacks cb 0x55bc338498b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9acaa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395ddd0 0x55bc33970020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9acb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc33969560 0x55bc33970aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9acaa00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395ddd0 0x55bc33970020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:32986/0 (socket says 172.21.15.45:32986) 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9acaa00700 1 -- 172.21.15.45:0/3807605804 learned_addr learned my addr 172.21.15.45:0/3807605804 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9aca000700 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395e6f0 0x55bc33970560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9acaa00700 1 -- 172.21.15.45:0/3807605804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395e6f0 msgr2=0x55bc33970560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9acaa00700 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395e6f0 0x55bc33970560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9acaa00700 1 -- 172.21.15.45:0/3807605804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc33969560 msgr2=0x55bc33970aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9acaa00700 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc33969560 0x55bc33970aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.593+0000 7f9acaa00700 1 -- 172.21.15.45:0/3807605804 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc339df420 con 0x55bc3395ddd0 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.594+0000 7f9acaa00700 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395ddd0 0x55bc33970020 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9abc09ce80 tx=0x7f9abc005ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152eeff9a45e6200 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.594+0000 7f9ab5600700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:01:12.598 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.594+0000 7f9ab5600700 1 -- 172.21.15.45:0/3807605804 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0ba070 con 0x55bc3395ddd0 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.594+0000 7f9ab5600700 1 -- 172.21.15.45:0/3807605804 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f9abc09b1f0 con 0x55bc3395ddd0 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.594+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3807605804 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ab8003e30 con 0x55bc3395ddd0 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.594+0000 7f9ab5600700 1 -- 172.21.15.45:0/3807605804 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9abc09cef0 con 0x55bc3395ddd0 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.594+0000 7f9adc7cab80 10 client.25454 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.594+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3807605804 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bc339e0700 con 0x55bc3395ddd0 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.595+0000 7f9ab5600700 1 -- 172.21.15.45:0/3807605804 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab8003e30 con 0x55bc3395ddd0 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.595+0000 7f9ab5600700 1 -- 172.21.15.45:0/3807605804 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f9abc09d1e0 con 0x55bc3395ddd0 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.595+0000 7f9adc7cab80 10 client.25454 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.595+0000 7f9adc7cab80 10 client.25454 resolve_mds: validated gid 24524 aka daemon mds.b 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.595+0000 7f9adc7cab80 20 client.25454 populate_metadata read hostname 'smithi045' 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.595+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bc339e0e60 0x55bc339e3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:12.599 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.595+0000 7f9adc7cab80 4 client.25454 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.4, "await_for": 42.0}] 2024-04-25T19:01:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.595+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3807605804 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.4, "await_for": 42.0}) -- 0x55bc338b88b0 con 0x55bc339e0e60 2024-04-25T19:01:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.595+0000 7f9aca000700 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bc339e0e60 0x55bc339e3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.596+0000 7f9aca000700 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bc339e0e60 0x55bc339e3280 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=323ced14b2eebc41 server_cookie=1e34514d80ba8989 in_seq=0 out_seq=0 2024-04-25T19:01:12.600 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:12.596+0000 7f9ab5600700 10 client.25454 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:14.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:14 smithi062 ceph-mon[32844]: pgmap v60: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-25T19:01:14.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:14 smithi079 ceph-mon[33002]: pgmap v60: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-25T19:01:14.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:14 smithi045 ceph-mon[27776]: pgmap v60: 97 pgs: 97 active+clean; 15 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 1.1 MiB/s wr, 25 op/s 2024-04-25T19:01:16.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:16 smithi062 ceph-mon[32844]: pgmap v61: 97 pgs: 97 active+clean; 16 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-04-25T19:01:16.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:16 smithi079 ceph-mon[33002]: pgmap v61: 97 pgs: 97 active+clean; 16 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-04-25T19:01:16.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:16 smithi045 ceph-mon[27776]: pgmap v61: 97 pgs: 97 active+clean; 16 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.4 MiB/s wr, 110 op/s 2024-04-25T19:01:18.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:18 smithi062 ceph-mon[32844]: pgmap v62: 97 pgs: 97 active+clean; 20 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-04-25T19:01:18.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:18 smithi079 ceph-mon[33002]: pgmap v62: 97 pgs: 97 active+clean; 20 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-04-25T19:01:18.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:18 smithi045 ceph-mon[27776]: pgmap v62: 97 pgs: 97 active+clean; 20 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-04-25T19:01:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:20 smithi062 ceph-mon[32844]: pgmap v63: 97 pgs: 97 active+clean; 20 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-04-25T19:01:20.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:20 smithi079 ceph-mon[33002]: pgmap v63: 97 pgs: 97 active+clean; 20 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-04-25T19:01:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:20 smithi045 ceph-mon[27776]: pgmap v63: 97 pgs: 97 active+clean; 20 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-04-25T19:01:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:22 smithi079 ceph-mon[33002]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-04-25T19:01:22.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:22 smithi045 ceph-mon[27776]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-04-25T19:01:22.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:22 smithi062 ceph-mon[32844]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.7 MiB/s wr, 156 op/s 2024-04-25T19:01:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.757+0000 7f93dd400640 1 -- 172.21.15.45:0/2618276293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93d81085a0 msgr2=0x7f93d8108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.757+0000 7f93dd400640 1 --2- 172.21.15.45:0/2618276293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93d81085a0 0x7f93d8108a20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f93c0009930 tx=0x7f93c002a440 comp rx=0 tx=0).stop 2024-04-25T19:01:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.757+0000 7f93dd400640 1 -- 172.21.15.45:0/2618276293 shutdown_connections 2024-04-25T19:01:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.757+0000 7f93dd400640 1 --2- 172.21.15.45:0/2618276293 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d8108f60 0x7f93d8115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.757+0000 7f93dd400640 1 --2- 172.21.15.45:0/2618276293 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93d81085a0 0x7f93d8108a20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.757+0000 7f93dd400640 1 --2- 172.21.15.45:0/2618276293 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d810e660 0x7f93d810ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.757+0000 7f93dd400640 1 -- 172.21.15.45:0/2618276293 >> 172.21.15.45:0/2618276293 conn(0x7f93d8076460 msgr2=0x7f93d8078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.757+0000 7f93dd400640 1 -- 172.21.15.45:0/2618276293 shutdown_connections 2024-04-25T19:01:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.757+0000 7f93dd400640 1 -- 172.21.15.45:0/2618276293 wait complete. 2024-04-25T19:01:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.758+0000 7f93dd400640 1 Processor -- start 2024-04-25T19:01:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.758+0000 7f93dd400640 1 -- start start 2024-04-25T19:01:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.758+0000 7f93dd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93d81085a0 0x7f93d819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.758+0000 7f93dd400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d8108f60 0x7f93d819f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.758+0000 7f93dd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d810e660 0x7f93d81a36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.758+0000 7f93dd400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81188a0 con 0x7f93d810e660 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93dd400640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f93d8118720 con 0x7f93d81085a0 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93dd400640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f93d8118a20 con 0x7f93d8108f60 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93d6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93d81085a0 0x7f93d819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93d6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93d81085a0 0x7f93d819ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:51300/0 (socket says 172.21.15.45:51300) 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93d6800640 1 -- 172.21.15.45:0/1505528326 learned_addr learned my addr 172.21.15.45:0/1505528326 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93cfe00640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d8108f60 0x7f93d819f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93d7200640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d810e660 0x7f93d81a36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93d6800640 1 -- 172.21.15.45:0/1505528326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d8108f60 msgr2=0x7f93d819f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93d6800640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d8108f60 0x7f93d819f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93d6800640 1 -- 172.21.15.45:0/1505528326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d810e660 msgr2=0x7f93d81a36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93d6800640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d810e660 0x7f93d81a36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93d6800640 1 -- 172.21.15.45:0/1505528326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d81a3e10 con 0x7f93d81085a0 2024-04-25T19:01:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.759+0000 7f93cfe00640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d8108f60 0x7f93d819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.760+0000 7f93d6800640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93d81085a0 0x7f93d819ee20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93c800e9f0 tx=0x7f93c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.760+0000 7f93cf400640 1 -- 172.21.15.45:0/1505528326 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c800cd80 con 0x7f93d81085a0 2024-04-25T19:01:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.760+0000 7f93cf400640 1 -- 172.21.15.45:0/1505528326 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c80108e0 con 0x7f93d81085a0 2024-04-25T19:01:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.760+0000 7f93dd400640 1 -- 172.21.15.45:0/1505528326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d81a4100 con 0x7f93d81085a0 2024-04-25T19:01:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.760+0000 7f93cf400640 1 -- 172.21.15.45:0/1505528326 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8018980 con 0x7f93d81085a0 2024-04-25T19:01:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.760+0000 7f93dd400640 1 -- 172.21.15.45:0/1505528326 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d81ab990 con 0x7f93d81085a0 2024-04-25T19:01:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.763+0000 7f93e03d5740 1 Processor -- start 2024-04-25T19:01:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93e03d5740 1 -- start start 2024-04-25T19:01:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93cf400640 1 -- 172.21.15.45:0/1505528326 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f93c8010430 con 0x7f93d81085a0 2024-04-25T19:01:23.766 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe99cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93e03d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55eabe902900 con 0x55eabe99c820 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93e03d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55eabe8a8ef0 con 0x55eabe99d140 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93e03d5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55eabe8fe8c0 con 0x55eabe9a7a00 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93d7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe99cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93d6800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93d7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe99cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40170/0 (socket says 172.21.15.45:40170) 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93d7200640 1 -- 172.21.15.45:0/3811410508 learned_addr learned my addr 172.21.15.45:0/3811410508 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.764+0000 7f93cfe00640 1 --2- 172.21.15.45:0/3811410508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93d7200640 1 -- 172.21.15.45:0/3811410508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 msgr2=0x55eabe9a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93d7200640 1 --2- 172.21.15.45:0/3811410508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.767 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93d7200640 1 -- 172.21.15.45:0/3811410508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 msgr2=0x55eabe9a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93d7200640 1 --2- 172.21.15.45:0/3811410508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9a74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93d7200640 1 -- 172.21.15.45:0/3811410508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eabe615210 con 0x55eabe99c820 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93cfe00640 1 --2- 172.21.15.45:0/3811410508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93d6800640 1 --2- 172.21.15.45:0/3811410508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93d7200640 1 --2- 172.21.15.45:0/3811410508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe99cc00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009fd0 tx=0x7f93d00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c368673268ffc361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93c7400640 1 -- 172.21.15.45:0/3811410508 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0004d20 con 0x55eabe99c820 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93cf400640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f93bc0a3d90 0x7f93bc0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.765+0000 7f93c7400640 1 -- 172.21.15.45:0/3811410508 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d0004050 con 0x55eabe99c820 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.766+0000 7f93c7400640 1 -- 172.21.15.45:0/3811410508 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0005280 con 0x55eabe99c820 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.766+0000 7f93cf400640 1 -- 172.21.15.45:0/1505528326 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f93c80d0a30 con 0x7f93d81085a0 2024-04-25T19:01:23.768 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.766+0000 7f93d6800640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f93bc0a3d90 0x7f93bc0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.766+0000 7f93e03d5740 1 -- 172.21.15.45:0/3811410508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 msgr2=0x55eabe99cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.766+0000 7f93e03d5740 1 --2- 172.21.15.45:0/3811410508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe99cc00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009fd0 tx=0x7f93d00118d0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.766+0000 7f93e03d5740 1 -- 172.21.15.45:0/3811410508 shutdown_connections 2024-04-25T19:01:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.766+0000 7f93e03d5740 1 --2- 172.21.15.45:0/3811410508 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.766+0000 7f93e03d5740 1 --2- 172.21.15.45:0/3811410508 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.766+0000 7f93e03d5740 1 --2- 172.21.15.45:0/3811410508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe99cc00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.766+0000 7f93e03d5740 1 -- 172.21.15.45:0/3811410508 >> 172.21.15.45:0/3811410508 conn(0x55eabe998d30 msgr2=0x55eabe999110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.767+0000 7f93d6800640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f93bc0a3d90 0x7f93bc0a6250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55eabe99ddc0 tx=0x7f93c80d44e0 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:23.772 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.767+0000 7f93e03d5740 1 -- 172.21.15.45:0/3811410508 shutdown_connections 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.767+0000 7f93e03d5740 1 -- 172.21.15.45:0/3811410508 wait complete. 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.767+0000 7f93e03d5740 1 -- start start 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.767+0000 7f93e03d5740 1 Processor -- start 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9a4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93e03d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55eabe9aa630 con 0x55eabe99c820 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93e03d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55eabe8bc310 con 0x55eabe99d140 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93e03d5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55eabe9aa7b0 con 0x55eabe9a7a00 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93e03d5740 10 client.0 _ll_register_callbacks cb 0x55eabe88acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93d7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93d7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9a4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40172/0 (socket says 172.21.15.45:40172) 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93d7200640 1 -- 172.21.15.45:0/2190828913 learned_addr learned my addr 172.21.15.45:0/2190828913 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93cfe00640 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.773 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.768+0000 7f93d7200640 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.769+0000 7f93cfe00640 1 -- 172.21.15.45:0/2190828913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 msgr2=0x55eabe9a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.769+0000 7f93cfe00640 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.769+0000 7f93cfe00640 1 -- 172.21.15.45:0/2190828913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 msgr2=0x55eabe9a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.769+0000 7f93cfe00640 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.769+0000 7f93cfe00640 1 -- 172.21.15.45:0/2190828913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eabe6e1620 con 0x55eabe99d140 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.769+0000 7f93cfe00640 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9a4e70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93c002a960 tx=0x7f93c00050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62cdf707f4f42de8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.769+0000 7f93bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.769+0000 7f93bbe00640 1 -- 172.21.15.45:0/2190828913 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0035040 con 0x55eabe99d140 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.770+0000 7f93e03d5740 1 -- 172.21.15.45:0/2190828913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93d0004050 con 0x55eabe99d140 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.770+0000 7f93bbe00640 1 -- 172.21.15.45:0/2190828913 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f93c0004770 con 0x55eabe99d140 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.770+0000 7f93bbe00640 1 -- 172.21.15.45:0/2190828913 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93c0004970 con 0x55eabe99d140 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.770+0000 7f93e03d5740 10 client.25466 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.770+0000 7f93e03d5740 1 -- 172.21.15.45:0/2190828913 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eabe9b6830 con 0x55eabe99d140 2024-04-25T19:01:23.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.770+0000 7f93bbe00640 1 -- 172.21.15.45:0/2190828913 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d0004050 con 0x55eabe99d140 2024-04-25T19:01:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.770+0000 7f93bbe00640 1 -- 172.21.15.45:0/2190828913 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f93c0003c90 con 0x55eabe99d140 2024-04-25T19:01:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.770+0000 7f93e03d5740 10 client.25466 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.770+0000 7f93e03d5740 10 client.25466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:01:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.770+0000 7f93e03d5740 20 client.25466 populate_metadata read hostname 'smithi045' 2024-04-25T19:01:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.771+0000 7f93e03d5740 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55eabe9b7ca0 0x55eabe9ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.771+0000 7f93e03d5740 4 client.25466 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:01:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.771+0000 7f93e03d5740 1 -- 172.21.15.45:0/2190828913 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eabe9168b0 con 0x55eabe9b7ca0 2024-04-25T19:01:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.771+0000 7f93d7200640 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55eabe9b7ca0 0x55eabe9ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.771+0000 7f93d7200640 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55eabe9b7ca0 0x55eabe9ba0f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8738821500e8af51 server_cookie=e745eeaf235e4af3 in_seq=0 out_seq=0 2024-04-25T19:01:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.771+0000 7f93bbe00640 10 client.25466 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.772+0000 7f93bbe00640 1 -- 172.21.15.45:0/2190828913 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55eabe9168b0 con 0x55eabe9b7ca0 2024-04-25T19:01:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.772+0000 7f93bbe00640 10 client.25466 handle_command_reply: tid=0 2024-04-25T19:01:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.772+0000 7f93e03d5740 1 client.25466 shutdown 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 1 -- 172.21.15.45:0/2190828913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 msgr2=0x55eabe9a4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9a4e70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f93c002a960 tx=0x7f93c00050c0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 1 -- 172.21.15.45:0/2190828913 shutdown_connections 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55eabe9b7ca0 0x55eabe9ba0f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9a4e70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 1 --2- 172.21.15.45:0/2190828913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 1 -- 172.21.15.45:0/2190828913 >> 172.21.15.45:0/2190828913 conn(0x55eabe998d30 msgr2=0x55eabe99eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93bbe00640 0 client.25466 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 1 -- 172.21.15.45:0/2190828913 shutdown_connections 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 1 -- 172.21.15.45:0/2190828913 wait complete. 2024-04-25T19:01:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.773+0000 7f93e03d5740 20 client.25466 trim_cache size 0 max 16384 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.789+0000 7f93e03d5740 1 Processor -- start 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93e03d5740 1 -- start start 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe99c820 0x55eabe9b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99d140 0x55eabe997e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe9a7a00 0x55eabe9bfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93e03d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81b6c70 con 0x55eabe99d140 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93e03d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81b4540 con 0x55eabe9a7a00 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93e03d5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81b2390 con 0x55eabe99c820 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99d140 0x55eabe997e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99d140 0x55eabe997e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40176/0 (socket says 172.21.15.45:40176) 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93cfe00640 1 -- 172.21.15.45:0/631453086 learned_addr learned my addr 172.21.15.45:0/631453086 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.790+0000 7f93d7200640 1 --2- 172.21.15.45:0/631453086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe99c820 0x55eabe9b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93d7200640 1 --2- 172.21.15.45:0/631453086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe9a7a00 0x55eabe9bfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93cfe00640 1 -- 172.21.15.45:0/631453086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe99c820 msgr2=0x55eabe9b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93cfe00640 1 --2- 172.21.15.45:0/631453086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe99c820 0x55eabe9b34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93cfe00640 1 -- 172.21.15.45:0/631453086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe9a7a00 msgr2=0x55eabe9bfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93cfe00640 1 --2- 172.21.15.45:0/631453086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe9a7a00 0x55eabe9bfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93cfe00640 1 -- 172.21.15.45:0/631453086 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eabe9a38e0 con 0x55eabe99d140 2024-04-25T19:01:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93d7200640 1 --2- 172.21.15.45:0/631453086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe9a7a00 0x55eabe9bfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:01:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93cfe00640 1 --2- 172.21.15.45:0/631453086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99d140 0x55eabe997e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005c40 tx=0x7f93c002a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2949c07ab04470 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93c6a00640 1 -- 172.21.15.45:0/631453086 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c0035030 con 0x55eabe99d140 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93c6a00640 1 -- 172.21.15.45:0/631453086 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c0041030 con 0x55eabe99d140 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.791+0000 7f93c6a00640 1 -- 172.21.15.45:0/631453086 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c00035f0 con 0x55eabe99d140 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.792+0000 7f93e03d5740 1 -- 172.21.15.45:0/631453086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99d140 msgr2=0x55eabe997e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.792+0000 7f93e03d5740 1 --2- 172.21.15.45:0/631453086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99d140 0x55eabe997e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f93c0005c40 tx=0x7f93c002a440 comp rx=0 tx=0).stop 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.792+0000 7f93e03d5740 1 -- 172.21.15.45:0/631453086 shutdown_connections 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.792+0000 7f93e03d5740 1 --2- 172.21.15.45:0/631453086 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe9a7a00 0x55eabe9bfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.792+0000 7f93e03d5740 1 --2- 172.21.15.45:0/631453086 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99d140 0x55eabe997e90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.792+0000 7f93e03d5740 1 --2- 172.21.15.45:0/631453086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe99c820 0x55eabe9b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.792+0000 7f93e03d5740 1 -- 172.21.15.45:0/631453086 >> 172.21.15.45:0/631453086 conn(0x55eabe998d30 msgr2=0x55eabe999110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.792+0000 7f93e03d5740 1 -- 172.21.15.45:0/631453086 shutdown_connections 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.792+0000 7f93e03d5740 1 -- 172.21.15.45:0/631453086 wait complete. 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.793+0000 7f93e03d5740 1 -- start start 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.793+0000 7f93e03d5740 1 Processor -- start 2024-04-25T19:01:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.793+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.793+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.793+0000 7f93e03d5740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9ae110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.793+0000 7f93e03d5740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81b2a00 con 0x55eabe99c820 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.793+0000 7f93e03d5740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81b3e70 con 0x55eabe99d140 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.793+0000 7f93e03d5740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81b30d0 con 0x55eabe9a7a00 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.793+0000 7f93e03d5740 10 client.0 _ll_register_callbacks cb 0x7f93c00095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93d7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93d7200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9ad690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40188/0 (socket says 172.21.15.45:40188) 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93d7200640 1 -- 172.21.15.45:0/3985593364 learned_addr learned my addr 172.21.15.45:0/3985593364 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93cfe00640 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93d7200640 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9ae110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93cfe00640 1 -- 172.21.15.45:0/3985593364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 msgr2=0x55eabe9ae110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93cfe00640 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9ae110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93cfe00640 1 -- 172.21.15.45:0/3985593364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 msgr2=0x55eabe9ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:01:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93cfe00640 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93cfe00640 1 -- 172.21.15.45:0/3985593364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eabea1d4d0 con 0x55eabe99d140 2024-04-25T19:01:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93d7200640 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:01:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93cfe00640 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9adbd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55eabe9ba3d0 tx=0x7f93c0005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65b195f4c51326bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.794+0000 7f93bbe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:01:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.795+0000 7f93bbe00640 1 -- 172.21.15.45:0/3985593364 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c005d060 con 0x55eabe99d140 2024-04-25T19:01:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.795+0000 7f93bbe00640 1 -- 172.21.15.45:0/3985593364 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f93c0005580 con 0x55eabe99d140 2024-04-25T19:01:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.795+0000 7f93e03d5740 1 -- 172.21.15.45:0/3985593364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93c0041030 con 0x55eabe99d140 2024-04-25T19:01:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.795+0000 7f93bbe00640 1 -- 172.21.15.45:0/3985593364 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f93c0035070 con 0x55eabe99d140 2024-04-25T19:01:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.795+0000 7f93e03d5740 10 client.25469 fetch_fsmap learned FSMap version 28 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.795+0000 7f93e03d5740 1 -- 172.21.15.45:0/3985593364 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55eabea1e750 con 0x55eabe99d140 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.795+0000 7f93bbe00640 1 -- 172.21.15.45:0/3985593364 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93c0041030 con 0x55eabe99d140 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.796+0000 7f93bbe00640 1 -- 172.21.15.45:0/3985593364 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f93c0003ab0 con 0x55eabe99d140 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.796+0000 7f93e03d5740 10 client.25469 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.796+0000 7f93e03d5740 10 client.25469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.796+0000 7f93e03d5740 20 client.25469 populate_metadata read hostname 'smithi045' 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.796+0000 7f93e03d5740 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55eabea1ee30 0x55eabea21250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.796+0000 7f93e03d5740 4 client.25469 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.796+0000 7f93e03d5740 1 -- 172.21.15.45:0/3985593364 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eabe910320 con 0x55eabea1ee30 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.796+0000 7f93d7200640 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55eabea1ee30 0x55eabea21250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.796+0000 7f93d7200640 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55eabea1ee30 0x55eabea21250 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d5a5ddc3a285d98 server_cookie=8706e51f8f42403 in_seq=0 out_seq=0 2024-04-25T19:01:23.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:01:23.796+0000 7f93bbe00640 10 client.25469 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:24 smithi079 ceph-mon[33002]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 567 KiB/s wr, 148 op/s 2024-04-25T19:01:24.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:24 smithi045 ceph-mon[27776]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 567 KiB/s wr, 148 op/s 2024-04-25T19:01:24.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:24 smithi062 ceph-mon[32844]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 567 KiB/s wr, 148 op/s 2024-04-25T19:01:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:26 smithi079 ceph-mon[33002]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 616 KiB/s wr, 235 op/s 2024-04-25T19:01:26.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:26 smithi045 ceph-mon[27776]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 616 KiB/s wr, 235 op/s 2024-04-25T19:01:26.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:26 smithi062 ceph-mon[32844]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 616 KiB/s wr, 235 op/s 2024-04-25T19:01:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:28 smithi079 ceph-mon[33002]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 419 KiB/s wr, 231 op/s 2024-04-25T19:01:28.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:28 smithi045 ceph-mon[27776]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 419 KiB/s wr, 231 op/s 2024-04-25T19:01:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:28 smithi062 ceph-mon[32844]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 419 KiB/s wr, 231 op/s 2024-04-25T19:01:30.386 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:30 smithi045 ceph-mon[27776]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 82 KiB/s wr, 172 op/s 2024-04-25T19:01:30.386 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:30 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:01:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:30 smithi079 ceph-mon[33002]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 82 KiB/s wr, 172 op/s 2024-04-25T19:01:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:30 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:01:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:30 smithi062 ceph-mon[32844]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 82 KiB/s wr, 172 op/s 2024-04-25T19:01:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:30 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:01:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:31 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:01:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:31 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:01:31.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:31 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:01:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:31 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:01:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:31 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:01:31.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:31 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:01:31.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:31 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:01:31.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:31 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:01:31.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:31 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:01:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:32 smithi079 ceph-mon[33002]: pgmap v69: 97 pgs: 97 active+clean; 24 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 531 KiB/s wr, 286 op/s 2024-04-25T19:01:32.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:32 smithi045 ceph-mon[27776]: pgmap v69: 97 pgs: 97 active+clean; 24 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 531 KiB/s wr, 286 op/s 2024-04-25T19:01:32.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:32 smithi062 ceph-mon[32844]: pgmap v69: 97 pgs: 97 active+clean; 24 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 531 KiB/s wr, 286 op/s 2024-04-25T19:01:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:34 smithi079 ceph-mon[33002]: pgmap v70: 97 pgs: 97 active+clean; 24 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 516 KiB/s wr, 282 op/s 2024-04-25T19:01:34.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:34 smithi045 ceph-mon[27776]: pgmap v70: 97 pgs: 97 active+clean; 24 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 516 KiB/s wr, 282 op/s 2024-04-25T19:01:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:34 smithi062 ceph-mon[32844]: pgmap v70: 97 pgs: 97 active+clean; 24 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 516 KiB/s wr, 282 op/s 2024-04-25T19:01:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:36 smithi079 ceph-mon[33002]: pgmap v71: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 698 KiB/s wr, 287 op/s 2024-04-25T19:01:36.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:36 smithi045 ceph-mon[27776]: pgmap v71: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 698 KiB/s wr, 287 op/s 2024-04-25T19:01:36.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:36 smithi062 ceph-mon[32844]: pgmap v71: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 698 KiB/s wr, 287 op/s 2024-04-25T19:01:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:38 smithi079 ceph-mon[33002]: pgmap v72: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 649 KiB/s wr, 200 op/s 2024-04-25T19:01:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:38 smithi045 ceph-mon[27776]: pgmap v72: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 649 KiB/s wr, 200 op/s 2024-04-25T19:01:38.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:38 smithi062 ceph-mon[32844]: pgmap v72: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 649 KiB/s wr, 200 op/s 2024-04-25T19:01:40.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:40 smithi079 ceph-mon[33002]: pgmap v73: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 631 KiB/s wr, 118 op/s 2024-04-25T19:01:40.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:40 smithi045 ceph-mon[27776]: pgmap v73: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 631 KiB/s wr, 118 op/s 2024-04-25T19:01:40.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:40 smithi062 ceph-mon[32844]: pgmap v73: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 631 KiB/s wr, 118 op/s 2024-04-25T19:01:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:42 smithi079 ceph-mon[33002]: pgmap v74: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 633 KiB/s wr, 121 op/s 2024-04-25T19:01:42.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:42 smithi045 ceph-mon[27776]: pgmap v74: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 633 KiB/s wr, 121 op/s 2024-04-25T19:01:42.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:42 smithi062 ceph-mon[32844]: pgmap v74: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 633 KiB/s wr, 121 op/s 2024-04-25T19:01:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:44 smithi079 ceph-mon[33002]: pgmap v75: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 184 KiB/s wr, 7 op/s 2024-04-25T19:01:44.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:44 smithi045 ceph-mon[27776]: pgmap v75: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 184 KiB/s wr, 7 op/s 2024-04-25T19:01:44.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:44 smithi062 ceph-mon[32844]: pgmap v75: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 184 KiB/s wr, 7 op/s 2024-04-25T19:01:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:46 smithi079 ceph-mon[33002]: pgmap v76: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 10 op/s 2024-04-25T19:01:46.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:46 smithi045 ceph-mon[27776]: pgmap v76: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 10 op/s 2024-04-25T19:01:46.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:46 smithi062 ceph-mon[32844]: pgmap v76: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 186 KiB/s wr, 10 op/s 2024-04-25T19:01:48.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:48 smithi045 ceph-mon[27776]: pgmap v77: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-25T19:01:48.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:48 smithi062 ceph-mon[32844]: pgmap v77: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-25T19:01:48.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:48 smithi079 ceph-mon[33002]: pgmap v77: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.7 KiB/s wr, 6 op/s 2024-04-25T19:01:50.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:50 smithi045 ceph-mon[27776]: pgmap v78: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-04-25T19:01:50.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:50 smithi062 ceph-mon[32844]: pgmap v78: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-04-25T19:01:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:50 smithi079 ceph-mon[33002]: pgmap v78: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-04-25T19:01:52.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:52 smithi045 ceph-mon[27776]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T19:01:52.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:52 smithi062 ceph-mon[32844]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T19:01:52.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:52 smithi079 ceph-mon[33002]: pgmap v79: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T19:01:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:54 smithi045 ceph-mon[27776]: pgmap v80: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T19:01:54.602 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.597+0000 7f9ab5600700 1 -- 172.21.15.45:0/3807605804 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: -115 ) ==== 8+0+592 (crc 0 0 0) 0x55bc338b88b0 con 0x55bc339e0e60 2024-04-25T19:01:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.597+0000 7f9ab5600700 10 client.25454 handle_command_reply: tid=0 2024-04-25T19:01:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.597+0000 7f9adc7cab80 1 client.25454 shutdown 2024-04-25T19:01:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.597+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3807605804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395ddd0 msgr2=0x55bc33970020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.597+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395ddd0 0x55bc33970020 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9abc09ce80 tx=0x7f9abc005ab0 comp rx=0 tx=0).stop 2024-04-25T19:01:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.598+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3807605804 shutdown_connections 2024-04-25T19:01:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.598+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55bc339e0e60 0x55bc339e3280 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.598+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55bc3395ddd0 0x55bc33970020 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:54.603 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.598+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55bc33969560 0x55bc33970aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.598+0000 7f9adc7cab80 1 --2- 172.21.15.45:0/3807605804 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55bc3395e6f0 0x55bc33970560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.598+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3807605804 >> 172.21.15.45:0/3807605804 conn(0x55bc3388d0e0 msgr2=0x55bc33959d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:54.604 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.598+0000 7f9ab5600700 0 client.25454 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:01:54.604 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:01:54.604 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 28, 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24524, 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 5, 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "695e6b8e": { 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 5, 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 0.0, 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCING", 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.0 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 60.0, 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 89.4, 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-04-25T19:01:54.605 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCING", 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 42.0 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.598+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3807605804 shutdown_connections 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.598+0000 7f9adc7cab80 1 -- 172.21.15.45:0/3807605804 wait complete. 2024-04-25T19:01:54.606 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:54.598+0000 7f9adc7cab80 20 client.25454 trim_cache size 0 max 16384 2024-04-25T19:01:54.607 INFO:teuthology.orchestra.run.smithi045.stderr:Error EINPROGRESS: 2024-04-25T19:01:54.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:54 smithi062 ceph-mon[32844]: pgmap v80: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T19:01:54.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:54 smithi079 ceph-mon[33002]: pgmap v80: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T19:01:55.517 DEBUG:teuthology.orchestra.run:got remote process result: 115 2024-04-25T19:01:55.518 WARNING:tasks.quiescer.fs.[cephfs]:Set '695e6b8e' hasn't quiesced after 49.6 seconds (timeout: 60). Dumping ops with locks from all ranks. 2024-04-25T19:01:55.518 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:01:55.998 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:01:56.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:56 smithi045 ceph-mon[27776]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T19:01:56.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:56 smithi062 ceph-mon[32844]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T19:01:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:56 smithi079 ceph-mon[33002]: pgmap v81: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.6 KiB/s wr, 8 op/s 2024-04-25T19:01:58.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.340+0000 7f5257400700 1 -- 172.21.15.45:0/2915208999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525010e3b0 msgr2=0x7f5250115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:58.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.340+0000 7f5257400700 1 --2- 172.21.15.45:0/2915208999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525010e3b0 0x7f5250115f80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f524800b0a0 tx=0x7f524800b3b0 comp rx=0 tx=0).stop 2024-04-25T19:01:58.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.340+0000 7f5257400700 1 -- 172.21.15.45:0/2915208999 shutdown_connections 2024-04-25T19:01:58.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.340+0000 7f5257400700 1 --2- 172.21.15.45:0/2915208999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f525010e3b0 0x7f5250115f80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:58.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.340+0000 7f5257400700 1 --2- 172.21.15.45:0/2915208999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f52501092d0 0x7f52501096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:58.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.340+0000 7f5257400700 1 --2- 172.21.15.45:0/2915208999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5250109c80 0x7f525010dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:58.343 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.340+0000 7f5257400700 1 -- 172.21.15.45:0/2915208999 >> 172.21.15.45:0/2915208999 conn(0x7f5250076210 msgr2=0x7f5250107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:58.344 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:58 smithi045 ceph-mon[27776]: pgmap v82: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T19:01:58.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.340+0000 7f5257400700 1 -- 172.21.15.45:0/2915208999 shutdown_connections 2024-04-25T19:01:58.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.341+0000 7f5257400700 1 -- 172.21.15.45:0/2915208999 wait complete. 2024-04-25T19:01:58.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.343+0000 7f5257400700 1 Processor -- start 2024-04-25T19:01:58.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.343+0000 7f5257400700 1 -- start start 2024-04-25T19:01:58.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.343+0000 7f5257400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52501092d0 0x7f525019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:58.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.343+0000 7f5257400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250109c80 0x7f525019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:58.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5257400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f525010e3b0 0x7f52501a3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:58.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5257400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f52501192a0 con 0x7f5250109c80 2024-04-25T19:01:58.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5257400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5250119120 con 0x7f52501092d0 2024-04-25T19:01:58.346 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5257400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5250119420 con 0x7f525010e3b0 2024-04-25T19:01:58.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f524fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250109c80 0x7f525019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:58.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5254a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52501092d0 0x7f525019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:58.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5254a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52501092d0 0x7f525019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41928/0 (socket says 172.21.15.45:41928) 2024-04-25T19:01:58.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5254a00700 1 -- 172.21.15.45:0/3052912938 learned_addr learned my addr 172.21.15.45:0/3052912938 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:01:58.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5255400700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f525010e3b0 0x7f52501a3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:58.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5254a00700 1 -- 172.21.15.45:0/3052912938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f525010e3b0 msgr2=0x7f52501a3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:58.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5254a00700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f525010e3b0 0x7f52501a3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:58.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5254a00700 1 -- 172.21.15.45:0/3052912938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250109c80 msgr2=0x7f525019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:58.347 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5254a00700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250109c80 0x7f525019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:58.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.344+0000 7f5254a00700 1 -- 172.21.15.45:0/3052912938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52501a4700 con 0x7f52501092d0 2024-04-25T19:01:58.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.345+0000 7f5254a00700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52501092d0 0x7f525019f6f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f524400eb30 tx=0x7f524400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:58.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.345+0000 7f524d600700 1 -- 172.21.15.45:0/3052912938 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f524400cbd0 con 0x7f52501092d0 2024-04-25T19:01:58.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.345+0000 7f524d600700 1 -- 172.21.15.45:0/3052912938 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52440044d0 con 0x7f52501092d0 2024-04-25T19:01:58.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.345+0000 7f5257400700 1 -- 172.21.15.45:0/3052912938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52501a49f0 con 0x7f52501092d0 2024-04-25T19:01:58.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.345+0000 7f524d600700 1 -- 172.21.15.45:0/3052912938 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244003e30 con 0x7f52501092d0 2024-04-25T19:01:58.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.345+0000 7f5257400700 1 -- 172.21.15.45:0/3052912938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52501ac270 con 0x7f52501092d0 2024-04-25T19:01:58.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.346+0000 7f5257400700 1 -- 172.21.15.45:0/3052912938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5250069730 con 0x7f52501092d0 2024-04-25T19:01:58.350 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.348+0000 7f524d600700 1 -- 172.21.15.45:0/3052912938 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5244004000 con 0x7f52501092d0 2024-04-25T19:01:58.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.350+0000 7f524d600700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f52380a3db0 0x7f52380a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:01:58.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.350+0000 7f524d600700 1 -- 172.21.15.45:0/3052912938 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f52440d0bc0 con 0x7f52501092d0 2024-04-25T19:01:58.352 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.350+0000 7f524fe00700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f52380a3db0 0x7f52380a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:01:58.353 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.351+0000 7f524fe00700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f52380a3db0 0x7f52380a6270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f523c009680 tx=0x7f523c009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:01:58.356 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.354+0000 7f524d600700 1 -- 172.21.15.45:0/3052912938 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52440042f0 con 0x7f52501092d0 2024-04-25T19:01:58.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:58 smithi062 ceph-mon[32844]: pgmap v82: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T19:01:58.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.635+0000 7f5257400700 1 -- 172.21.15.45:0/3052912938 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f525002df80 con 0x7f52501092d0 2024-04-25T19:01:58.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.636+0000 7f524d600700 1 -- 172.21.15.45:0/3052912938 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11855 (secure 0 0 0) 0x7f524409d200 con 0x7f52501092d0 2024-04-25T19:01:58.639 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:01:58.639 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T19:01:10.424364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24512,24505]},"id":1}]} 2024-04-25T19:01:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.639+0000 7f5257400700 1 -- 172.21.15.45:0/3052912938 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f52380a3db0 msgr2=0x7f52380a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.639+0000 7f5257400700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f52380a3db0 0x7f52380a6270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f523c009680 tx=0x7f523c009260 comp rx=0 tx=0).stop 2024-04-25T19:01:58.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.639+0000 7f5257400700 1 -- 172.21.15.45:0/3052912938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52501092d0 msgr2=0x7f525019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:01:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.639+0000 7f5257400700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52501092d0 0x7f525019f6f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f524400eb30 tx=0x7f524400ee40 comp rx=0 tx=0).stop 2024-04-25T19:01:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.640+0000 7f5257400700 1 -- 172.21.15.45:0/3052912938 shutdown_connections 2024-04-25T19:01:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.640+0000 7f5257400700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f52380a3db0 0x7f52380a6270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.640+0000 7f5257400700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f52501092d0 0x7f525019f6f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.640+0000 7f5257400700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f525010e3b0 0x7f52501a3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:58.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.640+0000 7f5257400700 1 --2- 172.21.15.45:0/3052912938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5250109c80 0x7f525019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:01:58.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.640+0000 7f5257400700 1 -- 172.21.15.45:0/3052912938 >> 172.21.15.45:0/3052912938 conn(0x7f5250076210 msgr2=0x7f5250076750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:01:58.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.640+0000 7f5257400700 1 -- 172.21.15.45:0/3052912938 shutdown_connections 2024-04-25T19:01:58.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:01:58.640+0000 7f5257400700 1 -- 172.21.15.45:0/3052912938 wait complete. 2024-04-25T19:01:58.644 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T19:01:58.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:58 smithi079 ceph-mon[33002]: pgmap v82: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T19:01:59.363 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:01:59 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3052912938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:59.581 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 2 (i) to a remote file /var/run/ceph/1/ops-695e6b8e-1-mds.i.json 2024-04-25T19:01:59.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.1:2 ops --flags=locks --path=/var/run/ceph/ops-695e6b8e-1-mds.i.json 2024-04-25T19:01:59.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:01:59 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3052912938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:01:59.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:01:59 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3052912938' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:02:00.056 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:00.347 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:00 smithi045 ceph-mon[27776]: pgmap v83: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T19:02:00.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:00 smithi062 ceph-mon[32844]: pgmap v83: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T19:02:00.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:00 smithi079 ceph-mon[33002]: pgmap v83: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T19:02:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.091+0000 7f0b02e00700 1 -- 172.21.15.45:0/3672915870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0afc108a30 msgr2=0x7f0afc108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.091+0000 7f0b02e00700 1 --2- 172.21.15.45:0/3672915870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0afc108a30 0x7f0afc108e10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009990 tx=0x7f0aec009ca0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.091+0000 7f0b02e00700 1 -- 172.21.15.45:0/3672915870 shutdown_connections 2024-04-25T19:02:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.091+0000 7f0b02e00700 1 --2- 172.21.15.45:0/3672915870 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0afc06b0f0 0x7f0afc10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.095 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.091+0000 7f0b02e00700 1 --2- 172.21.15.45:0/3672915870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0afc108a30 0x7f0afc108e10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.091+0000 7f0b02e00700 1 --2- 172.21.15.45:0/3672915870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0afc06a750 0x7f0afc06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.091+0000 7f0b02e00700 1 -- 172.21.15.45:0/3672915870 >> 172.21.15.45:0/3672915870 conn(0x7f0afc0788f0 msgr2=0x7f0afc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.091+0000 7f0b02e00700 1 -- 172.21.15.45:0/3672915870 shutdown_connections 2024-04-25T19:02:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.091+0000 7f0b02e00700 1 -- 172.21.15.45:0/3672915870 wait complete. 2024-04-25T19:02:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.092+0000 7f0b02e00700 1 Processor -- start 2024-04-25T19:02:02.096 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.092+0000 7f0b02e00700 1 -- start start 2024-04-25T19:02:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0b02e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0afc06a750 0x7f0afc19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0b02e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0afc06b0f0 0x7f0afc19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0b02e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0afc108a30 0x7f0afc19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0b02e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc112c20 con 0x7f0afc06b0f0 2024-04-25T19:02:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0b02e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc112aa0 con 0x7f0afc06a750 2024-04-25T19:02:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0b02e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc112da0 con 0x7f0afc108a30 2024-04-25T19:02:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0afb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0afc06b0f0 0x7f0afc19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0afb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0afc06b0f0 0x7f0afc19b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:57992/0 (socket says 172.21.15.45:57992) 2024-04-25T19:02:02.106 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0afb400700 1 -- 172.21.15.45:0/2699205743 learned_addr learned my addr 172.21.15.45:0/2699205743 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0afbe00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0afc06a750 0x7f0afc19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.093+0000 7f0b00e00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0afc108a30 0x7f0afc19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.094+0000 7f0afbe00700 1 -- 172.21.15.45:0/2699205743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0afc108a30 msgr2=0x7f0afc19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.094+0000 7f0afbe00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0afc108a30 0x7f0afc19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.094+0000 7f0afbe00700 1 -- 172.21.15.45:0/2699205743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0afc06b0f0 msgr2=0x7f0afc19b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:02.107 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.094+0000 7f0afbe00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0afc06b0f0 0x7f0afc19b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.094+0000 7f0afbe00700 1 -- 172.21.15.45:0/2699205743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0afc1a0380 con 0x7f0afc06a750 2024-04-25T19:02:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.094+0000 7f0afbe00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0afc06a750 0x7f0afc19b370 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0aec000c00 tx=0x7f0aec0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.094+0000 7f0af8c00700 1 -- 172.21.15.45:0/2699205743 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec026070 con 0x7f0afc06a750 2024-04-25T19:02:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.094+0000 7f0b02e00700 1 -- 172.21.15.45:0/2699205743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0afc1a0610 con 0x7f0afc06a750 2024-04-25T19:02:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.094+0000 7f0af8c00700 1 -- 172.21.15.45:0/2699205743 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aec021410 con 0x7f0afc06a750 2024-04-25T19:02:02.108 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.094+0000 7f0af8c00700 1 -- 172.21.15.45:0/2699205743 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec017650 con 0x7f0afc06a750 2024-04-25T19:02:02.109 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.095+0000 7f0b02e00700 1 -- 172.21.15.45:0/2699205743 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0afc1a7ef0 con 0x7f0afc06a750 2024-04-25T19:02:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.098+0000 7f0af8c00700 1 -- 172.21.15.45:0/2699205743 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0aec029070 con 0x7f0afc06a750 2024-04-25T19:02:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.099+0000 7f0b0d80cb80 1 Processor -- start 2024-04-25T19:02:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.099+0000 7f0af8c00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0ae00a3db0 0x7f0ae00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.099+0000 7f0b0d80cb80 1 -- start start 2024-04-25T19:02:02.110 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.099+0000 7f0af8c00700 1 -- 172.21.15.45:0/2699205743 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f0aec0d1510 con 0x7f0afc06a750 2024-04-25T19:02:02.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0b00e00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0ae00a3db0 0x7f0ae00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07425ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 0x559a07430b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.111 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 0x559a074334c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a07378f70 con 0x559a074310d0 2024-04-25T19:02:02.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a0731f810 con 0x559a07426220 2024-04-25T19:02:02.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559a07374f30 con 0x559a07425900 2024-04-25T19:02:02.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0afb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 0x559a07430b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0afbe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07425ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.112 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0afbe00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07425ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:57848/0 (socket says 172.21.15.45:57848) 2024-04-25T19:02:02.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0afbe00700 1 -- 172.21.15.45:0/3117895418 learned_addr learned my addr 172.21.15.45:0/3117895418 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:02.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0afb400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 0x559a07430b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41948/0 (socket says 172.21.15.45:41948) 2024-04-25T19:02:02.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.100+0000 7f0b00e00700 1 --2- 172.21.15.45:0/3117895418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 0x559a074334c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0afbe00700 1 -- 172.21.15.45:0/3117895418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 msgr2=0x559a07430b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:02.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0afbe00700 1 --2- 172.21.15.45:0/3117895418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 0x559a07430b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.113 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0afbe00700 1 -- 172.21.15.45:0/3117895418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 msgr2=0x559a074334c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:02.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0afbe00700 1 --2- 172.21.15.45:0/3117895418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 0x559a074334c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0afbe00700 1 -- 172.21.15.45:0/3117895418 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a07315760 con 0x559a07425900 2024-04-25T19:02:02.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0b00e00700 1 --2- 172.21.15.45:0/3117895418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 0x559a074334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:02.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0b00e00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0ae00a3db0 0x7f0ae00a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0af400f590 tx=0x7f0af4009920 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:02.114 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0afbe00700 1 --2- 172.21.15.45:0/3117895418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07425ce0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0a5b90 tx=0x7f0aec09cec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d5bbc6dadf18409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:02.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0af1600700 1 -- 172.21.15.45:0/3117895418 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0b6070 con 0x559a07425900 2024-04-25T19:02:02.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0af1600700 1 -- 172.21.15.45:0/3117895418 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aec0a1890 con 0x559a07425900 2024-04-25T19:02:02.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.101+0000 7f0af1600700 1 -- 172.21.15.45:0/3117895418 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec099ce0 con 0x559a07425900 2024-04-25T19:02:02.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.102+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/3117895418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 msgr2=0x559a07425ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:02.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.102+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/3117895418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07425ce0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0a5b90 tx=0x7f0aec09cec0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.102+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/3117895418 shutdown_connections 2024-04-25T19:02:02.115 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.102+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/3117895418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 0x559a074334c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.102+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/3117895418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07425ce0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.102+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/3117895418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 0x559a07430b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.102+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/3117895418 >> 172.21.15.45:0/3117895418 conn(0x559a07354680 msgr2=0x559a07421db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:02.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.102+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/3117895418 shutdown_connections 2024-04-25T19:02:02.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.102+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/3117895418 wait complete. 2024-04-25T19:02:02.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.103+0000 7f0b0d80cb80 1 -- start start 2024-04-25T19:02:02.116 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.103+0000 7f0b0d80cb80 1 Processor -- start 2024-04-25T19:02:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07425900 0x559a074307e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07426220 0x559a0742db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 0x559a0742e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a07433a00 con 0x559a074310d0 2024-04-25T19:02:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a07332940 con 0x559a07425900 2024-04-25T19:02:02.122 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559a07433b80 con 0x559a07426220 2024-04-25T19:02:02.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0b0d80cb80 10 client.0 _ll_register_callbacks cb 0x559a07311800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:02.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0afbe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07425900 0x559a074307e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0afbe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07425900 0x559a074307e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41960/0 (socket says 172.21.15.45:41960) 2024-04-25T19:02:02.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0afbe00700 1 -- 172.21.15.45:0/2136131503 learned_addr learned my addr 172.21.15.45:0/2136131503 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:02.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0b00e00700 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 0x559a0742e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.104+0000 7f0afb400700 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07426220 0x559a0742db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0afbe00700 1 -- 172.21.15.45:0/2136131503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07426220 msgr2=0x559a0742db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:02.123 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0afbe00700 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07426220 0x559a0742db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0afbe00700 1 -- 172.21.15.45:0/2136131503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 msgr2=0x559a0742e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:02.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0afbe00700 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 0x559a0742e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0afbe00700 1 -- 172.21.15.45:0/2136131503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a072c69b0 con 0x559a07425900 2024-04-25T19:02:02.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0afbe00700 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07425900 0x559a074307e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00b2e0 tx=0x7f0aec099810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea2b17163d48b0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:02.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0ae6000700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:02:02.124 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0ae6000700 1 -- 172.21.15.45:0/2136131503 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aec0b6070 con 0x559a07425900 2024-04-25T19:02:02.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/2136131503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aec0a1890 con 0x559a07425900 2024-04-25T19:02:02.125 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0ae6000700 1 -- 172.21.15.45:0/2136131503 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f0aec09b7e0 con 0x559a07425900 2024-04-25T19:02:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0ae6000700 1 -- 172.21.15.45:0/2136131503 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0aec0a60c0 con 0x559a07425900 2024-04-25T19:02:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0b0d80cb80 10 client.25487 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.105+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/2136131503 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a0730fbd0 con 0x559a07425900 2024-04-25T19:02:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.106+0000 7f0ae6000700 1 -- 172.21.15.45:0/2136131503 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aec0a1890 con 0x559a07425900 2024-04-25T19:02:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.106+0000 7f0ae6000700 1 -- 172.21.15.45:0/2136131503 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f0aec0b6bd0 con 0x559a07425900 2024-04-25T19:02:02.126 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.106+0000 7f0b0d80cb80 10 client.25487 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.106+0000 7f0b0d80cb80 10 client.25487 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T19:02:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.106+0000 7f0b0d80cb80 20 client.25487 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.106+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559a07440500 0x559a074428f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.106+0000 7f0b0d80cb80 4 client.25487 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.106+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/2136131503 --> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a07386760 con 0x559a07440500 2024-04-25T19:02:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.107+0000 7f0afb400700 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559a07440500 0x559a074428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.127 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.107+0000 7f0afb400700 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559a07440500 0x559a074428f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ec4fd3cf28934e9 server_cookie=6dbc0963f12a9bb6 in_seq=0 out_seq=0 2024-04-25T19:02:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.107+0000 7f0ae6000700 10 client.25487 ms_handle_connect on v2:172.21.15.79:6838/235958219 2024-04-25T19:02:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.108+0000 7f0ae6000700 1 -- 172.21.15.45:0/2136131503 <== mds.2 v2:172.21.15.79:6838/235958219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a07386760 con 0x559a07440500 2024-04-25T19:02:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.108+0000 7f0ae6000700 10 client.25487 handle_command_reply: tid=0 2024-04-25T19:02:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.108+0000 7f0b0d80cb80 1 client.25487 shutdown 2024-04-25T19:02:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/2136131503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07425900 msgr2=0x559a074307e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07425900 0x559a074307e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00b2e0 tx=0x7f0aec099810 comp rx=0 tx=0).stop 2024-04-25T19:02:02.128 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/2136131503 shutdown_connections 2024-04-25T19:02:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559a07440500 0x559a074428f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07425900 0x559a074307e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a074310d0 0x559a0742e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/2136131503 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07426220 0x559a0742db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0ae6000700 0 client.25487 ms_handle_reset on v2:172.21.15.79:6838/235958219 2024-04-25T19:02:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/2136131503 >> 172.21.15.45:0/2136131503 conn(0x559a07354680 msgr2=0x559a074219a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/2136131503 shutdown_connections 2024-04-25T19:02:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/2136131503 wait complete. 2024-04-25T19:02:02.129 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.109+0000 7f0b0d80cb80 20 client.25487 trim_cache size 0 max 16384 2024-04-25T19:02:02.133 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.129+0000 7f0b0d80cb80 1 Processor -- start 2024-04-25T19:02:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.129+0000 7f0b0d80cb80 1 -- start start 2024-04-25T19:02:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07425900 0x559a07421ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 0x559a0743c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a074310d0 0x559a0742cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1b1cd0 con 0x559a07425900 2024-04-25T19:02:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1af560 con 0x559a07426220 2024-04-25T19:02:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1ad3b0 con 0x559a074310d0 2024-04-25T19:02:02.134 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0afbe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07425900 0x559a07421ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0b00e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a074310d0 0x559a0742cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0afbe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07425900 0x559a07421ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58016/0 (socket says 172.21.15.45:58016) 2024-04-25T19:02:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0afbe00700 1 -- 172.21.15.45:0/41279719 learned_addr learned my addr 172.21.15.45:0/41279719 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.130+0000 7f0afb400700 1 --2- 172.21.15.45:0/41279719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 0x559a0743c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.131+0000 7f0b00e00700 1 -- 172.21.15.45:0/41279719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 msgr2=0x559a0743c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.131+0000 7f0b00e00700 1 --2- 172.21.15.45:0/41279719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 0x559a0743c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.131+0000 7f0b00e00700 1 -- 172.21.15.45:0/41279719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07425900 msgr2=0x559a07421ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.131+0000 7f0b00e00700 1 --2- 172.21.15.45:0/41279719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07425900 0x559a07421ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.135 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.131+0000 7f0b00e00700 1 -- 172.21.15.45:0/41279719 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a0742e460 con 0x559a074310d0 2024-04-25T19:02:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.131+0000 7f0b00e00700 1 --2- 172.21.15.45:0/41279719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a074310d0 0x559a0742cbd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009fd0 tx=0x7f0af401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3243d5553df4029 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.131+0000 7f0af0c00700 1 -- 172.21.15.45:0/41279719 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af4027030 con 0x559a074310d0 2024-04-25T19:02:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.131+0000 7f0af0c00700 1 -- 172.21.15.45:0/41279719 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af4004bb0 con 0x559a074310d0 2024-04-25T19:02:02.136 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.131+0000 7f0af0c00700 1 -- 172.21.15.45:0/41279719 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af401d070 con 0x559a074310d0 2024-04-25T19:02:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.132+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/41279719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a074310d0 msgr2=0x559a0742cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.132+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/41279719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a074310d0 0x559a0742cbd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009fd0 tx=0x7f0af401c000 comp rx=0 tx=0).stop 2024-04-25T19:02:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.132+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/41279719 shutdown_connections 2024-04-25T19:02:02.141 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.132+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/41279719 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a074310d0 0x559a0742cbd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.132+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/41279719 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07425900 0x559a07421ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.132+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/41279719 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a07426220 0x559a0743c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.132+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/41279719 >> 172.21.15.45:0/41279719 conn(0x559a07354680 msgr2=0x559a07429180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.132+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/41279719 shutdown_connections 2024-04-25T19:02:02.142 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.132+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/41279719 wait complete. 2024-04-25T19:02:02.147 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.133+0000 7f0b0d80cb80 1 -- start start 2024-04-25T19:02:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.133+0000 7f0b0d80cb80 1 Processor -- start 2024-04-25T19:02:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07435390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07426220 0x559a074358d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0b0d80cb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a074310d0 0x559a0749e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.148 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1ada20 con 0x559a07426220 2024-04-25T19:02:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1aee90 con 0x559a074310d0 2024-04-25T19:02:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0b0d80cb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0afc1ae0f0 con 0x559a07425900 2024-04-25T19:02:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0b0d80cb80 10 client.0 _ll_register_callbacks cb 0x559a07311800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0afb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07426220 0x559a074358d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0afb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07426220 0x559a074358d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:58020/0 (socket says 172.21.15.45:58020) 2024-04-25T19:02:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0afb400700 1 -- 172.21.15.45:0/405755116 learned_addr learned my addr 172.21.15.45:0/405755116 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0b00e00700 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a074310d0 0x559a0749e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0afbe00700 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07435390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.149 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0afb400700 1 -- 172.21.15.45:0/405755116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 msgr2=0x559a07435390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.134+0000 7f0afb400700 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07435390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.135+0000 7f0afb400700 1 -- 172.21.15.45:0/405755116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a074310d0 msgr2=0x559a0749e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.135+0000 7f0afb400700 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a074310d0 0x559a0749e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.135+0000 7f0afb400700 1 -- 172.21.15.45:0/405755116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a0749f0a0 con 0x559a07426220 2024-04-25T19:02:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.135+0000 7f0afbe00700 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07435390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.135+0000 7f0b00e00700 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a074310d0 0x559a0749e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.135+0000 7f0afb400700 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07426220 0x559a074358d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009e00 tx=0x7f0ae80108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f1c4ee87aaa527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.135+0000 7f0ae6000700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:02:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.135+0000 7f0ae6000700 1 -- 172.21.15.45:0/405755116 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8020480 con 0x559a07426220 2024-04-25T19:02:02.150 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.135+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/405755116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0af4004bb0 con 0x559a07426220 2024-04-25T19:02:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.135+0000 7f0ae6000700 1 -- 172.21.15.45:0/405755116 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f0ae8020620 con 0x559a07426220 2024-04-25T19:02:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0ae6000700 1 -- 172.21.15.45:0/405755116 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0ae8030050 con 0x559a07426220 2024-04-25T19:02:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0b0d80cb80 10 client.15366 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/405755116 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559a074a7460 con 0x559a07426220 2024-04-25T19:02:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0ae6000700 1 -- 172.21.15.45:0/405755116 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af4004bb0 con 0x559a07426220 2024-04-25T19:02:02.151 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0ae6000700 1 -- 172.21.15.45:0/405755116 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f0ae80283c0 con 0x559a07426220 2024-04-25T19:02:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0b0d80cb80 10 client.15366 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0b0d80cb80 10 client.15366 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-04-25T19:02:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0b0d80cb80 20 client.15366 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559a074a7bc0 0x559a074a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0b0d80cb80 4 client.15366 mds_command: new command op to 24505 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-695e6b8e-1-mds.i.json"}] 2024-04-25T19:02:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.136+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/405755116 --> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-695e6b8e-1-mds.i.json"}) -- 0x559a07380800 con 0x559a074a7bc0 2024-04-25T19:02:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.137+0000 7f0afbe00700 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559a074a7bc0 0x559a074a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.137+0000 7f0afbe00700 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559a074a7bc0 0x559a074a9fe0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bdb17078a1db3dbf server_cookie=a54a0fd4ad8507a9 in_seq=0 out_seq=0 2024-04-25T19:02:02.152 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.137+0000 7f0ae6000700 10 client.15366 ms_handle_connect on v2:172.21.15.79:6838/235958219 2024-04-25T19:02:02.154 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0ae6000700 1 -- 172.21.15.45:0/405755116 <== mds.2 v2:172.21.15.79:6838/235958219 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x559a07380800 con 0x559a074a7bc0 2024-04-25T19:02:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0ae6000700 10 client.15366 handle_command_reply: tid=0 2024-04-25T19:02:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 client.15366 shutdown 2024-04-25T19:02:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/405755116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07426220 msgr2=0x559a074358d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07426220 0x559a074358d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009e00 tx=0x7f0ae80108b0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.155 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/405755116 shutdown_connections 2024-04-25T19:02:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.79:6838/235958219,v1:172.21.15.79:6839/235958219] conn(0x559a074a7bc0 0x559a074a9fe0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a074310d0 0x559a0749e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a07425900 0x559a07435390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 --2- 172.21.15.45:0/405755116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a07426220 0x559a074358d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/405755116 >> 172.21.15.45:0/405755116 conn(0x559a07354680 msgr2=0x559a0742fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0ae6000700 0 client.15366 ms_handle_reset on v2:172.21.15.79:6838/235958219 2024-04-25T19:02:02.156 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/405755116 shutdown_connections 2024-04-25T19:02:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 1 -- 172.21.15.45:0/405755116 wait complete. 2024-04-25T19:02:02.157 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.140+0000 7f0b0d80cb80 20 client.15366 trim_cache size 0 max 16384 2024-04-25T19:02:02.157 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:02:02.157 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/var/run/ceph/ops-695e6b8e-1-mds.i.json" 2024-04-25T19:02:02.158 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:02:02.158 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 -- 172.21.15.45:0/2699205743 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0ae00a3db0 msgr2=0x7f0ae00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0ae00a3db0 0x7f0ae00a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0af400f590 tx=0x7f0af4009920 comp rx=0 tx=0).stop 2024-04-25T19:02:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 -- 172.21.15.45:0/2699205743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0afc06a750 msgr2=0x7f0afc19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0afc06a750 0x7f0afc19b370 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0aec000c00 tx=0x7f0aec0047b0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 -- 172.21.15.45:0/2699205743 shutdown_connections 2024-04-25T19:02:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f0ae00a3db0 0x7f0ae00a6270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0afc06a750 0x7f0afc19b370 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0afc108a30 0x7f0afc19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 --2- 172.21.15.45:0/2699205743 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0afc06b0f0 0x7f0afc19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 -- 172.21.15.45:0/2699205743 >> 172.21.15.45:0/2699205743 conn(0x7f0afc0788f0 msgr2=0x7f0afc0ff7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 -- 172.21.15.45:0/2699205743 shutdown_connections 2024-04-25T19:02:02.160 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:02.141+0000 7f0b02e00700 1 -- 172.21.15.45:0/2699205743 wait complete. 2024-04-25T19:02:02.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:02 smithi062 ceph-mon[32844]: pgmap v84: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T19:02:02.658 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:02 smithi045 ceph-mon[27776]: pgmap v84: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T19:02:02.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:02 smithi079 ceph-mon[33002]: pgmap v84: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 8 op/s 2024-04-25T19:02:03.145 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 1 (a) to a remote file /var/run/ceph/1/ops-695e6b8e-1-mds.a.json 2024-04-25T19:02:03.145 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.1:1 ops --flags=locks --path=/var/run/ceph/ops-695e6b8e-1-mds.a.json 2024-04-25T19:02:03.628 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:04.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:04 smithi045 ceph-mon[27776]: pgmap v85: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T19:02:04.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:04 smithi062 ceph-mon[32844]: pgmap v85: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T19:02:04.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:04 smithi079 ceph-mon[33002]: pgmap v85: 97 pgs: 97 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T19:02:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.742+0000 7fd942400700 1 -- 172.21.15.45:0/4211705480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd93c1044c0 msgr2=0x7fd93c10c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.742+0000 7fd942400700 1 --2- 172.21.15.45:0/4211705480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd93c1044c0 0x7fd93c10c9b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd9280098f0 tx=0x7fd928009c00 comp rx=0 tx=0).stop 2024-04-25T19:02:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.742+0000 7fd942400700 1 -- 172.21.15.45:0/4211705480 shutdown_connections 2024-04-25T19:02:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.742+0000 7fd942400700 1 --2- 172.21.15.45:0/4211705480 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd93c10cfd0 0x7fd93c10f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.742+0000 7fd942400700 1 --2- 172.21.15.45:0/4211705480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd93c103ba0 0x7fd93c103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.742+0000 7fd942400700 1 --2- 172.21.15.45:0/4211705480 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd93c1044c0 0x7fd93c10c9b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.742+0000 7fd942400700 1 -- 172.21.15.45:0/4211705480 >> 172.21.15.45:0/4211705480 conn(0x7fd93c0fd3f0 msgr2=0x7fd93c0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.743+0000 7fd942400700 1 -- 172.21.15.45:0/4211705480 shutdown_connections 2024-04-25T19:02:05.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.743+0000 7fd942400700 1 -- 172.21.15.45:0/4211705480 wait complete. 2024-04-25T19:02:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.745+0000 7fd942400700 1 Processor -- start 2024-04-25T19:02:05.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.745+0000 7fd942400700 1 -- start start 2024-04-25T19:02:05.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd942400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd93c103ba0 0x7fd93c1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd942400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd93c1044c0 0x7fd93c199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd942400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd93c10cfd0 0x7fd93c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd942400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c112c10 con 0x7fd93c10cfd0 2024-04-25T19:02:05.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd942400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c112a90 con 0x7fd93c1044c0 2024-04-25T19:02:05.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd942400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c112d90 con 0x7fd93c103ba0 2024-04-25T19:02:05.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd93be00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd93c1044c0 0x7fd93c199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd93be00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd93c1044c0 0x7fd93c199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:50624/0 (socket says 172.21.15.45:50624) 2024-04-25T19:02:05.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd93be00700 1 -- 172.21.15.45:0/3970736633 learned_addr learned my addr 172.21.15.45:0/3970736633 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:05.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd941a00700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd93c10cfd0 0x7fd93c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.746+0000 7fd941000700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd93c103ba0 0x7fd93c1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.747+0000 7fd93be00700 1 -- 172.21.15.45:0/3970736633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd93c103ba0 msgr2=0x7fd93c1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:05.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.747+0000 7fd93be00700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd93c103ba0 0x7fd93c1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.747+0000 7fd93be00700 1 -- 172.21.15.45:0/3970736633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd93c10cfd0 msgr2=0x7fd93c19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:05.749 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.747+0000 7fd93be00700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd93c10cfd0 0x7fd93c19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.747+0000 7fd93be00700 1 -- 172.21.15.45:0/3970736633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c19e190 con 0x7fd93c1044c0 2024-04-25T19:02:05.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.747+0000 7fd93be00700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd93c1044c0 0x7fd93c199710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd928009230 tx=0x7fd92800f990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:05.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.747+0000 7fd939600700 1 -- 172.21.15.45:0/3970736633 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928027070 con 0x7fd93c1044c0 2024-04-25T19:02:05.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.747+0000 7fd942400700 1 -- 172.21.15.45:0/3970736633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93c19e420 con 0x7fd93c1044c0 2024-04-25T19:02:05.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.748+0000 7fd939600700 1 -- 172.21.15.45:0/3970736633 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd928005570 con 0x7fd93c1044c0 2024-04-25T19:02:05.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.748+0000 7fd939600700 1 -- 172.21.15.45:0/3970736633 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd928004930 con 0x7fd93c1044c0 2024-04-25T19:02:05.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.748+0000 7fd942400700 1 -- 172.21.15.45:0/3970736633 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd93c1a5d00 con 0x7fd93c1044c0 2024-04-25T19:02:05.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.751+0000 7fd939600700 1 -- 172.21.15.45:0/3970736633 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd92802a070 con 0x7fd93c1044c0 2024-04-25T19:02:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.752+0000 7fd939600700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9240a3db0 0x7fd9240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.753+0000 7fd94e3e8b80 1 Processor -- start 2024-04-25T19:02:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.753+0000 7fd939600700 1 -- 172.21.15.45:0/3970736633 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fd9280d1210 con 0x7fd93c1044c0 2024-04-25T19:02:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.753+0000 7fd941000700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9240a3db0 0x7fd9240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.753+0000 7fd94e3e8b80 1 -- start start 2024-04-25T19:02:05.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.753+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c82ae80 0x55756c82b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.753+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 0x55756c83a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.753+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c83aa60 0x55756c83ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.753+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55756c77df70 con 0x55756c82ae80 2024-04-25T19:02:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.753+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55756c724810 con 0x55756c83aa60 2024-04-25T19:02:05.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.753+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55756c779f30 con 0x55756c82b7a0 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd93be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 0x55756c83a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd941000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c82ae80 0x55756c82b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd93be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 0x55756c83a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:38134/0 (socket says 172.21.15.45:38134) 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd93be00700 1 -- 172.21.15.45:0/1131056430 learned_addr learned my addr 172.21.15.45:0/1131056430 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd941a00700 1 --2- 172.21.15.45:0/1131056430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c83aa60 0x55756c83ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd941000700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9240a3db0 0x7fd9240a6270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd930007be0 tx=0x7fd9300073d0 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd93be00700 1 -- 172.21.15.45:0/1131056430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c83aa60 msgr2=0x55756c83ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd93be00700 1 --2- 172.21.15.45:0/1131056430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c83aa60 0x55756c83ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd93be00700 1 -- 172.21.15.45:0/1131056430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c82ae80 msgr2=0x55756c82b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd93be00700 1 --2- 172.21.15.45:0/1131056430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c82ae80 0x55756c82b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd93be00700 1 -- 172.21.15.45:0/1131056430 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756c71a760 con 0x55756c82b7a0 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.754+0000 7fd941000700 1 --2- 172.21.15.45:0/1131056430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c82ae80 0x55756c82b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.755+0000 7fd93be00700 1 --2- 172.21.15.45:0/1131056430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 0x55756c83a520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd928099640 tx=0x7fd92809cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec3d509faa003b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:05.757 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.755+0000 7fd92e000700 1 -- 172.21.15.45:0/1131056430 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92809d1e0 con 0x55756c82b7a0 2024-04-25T19:02:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.755+0000 7fd92e000700 1 -- 172.21.15.45:0/1131056430 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9280a1590 con 0x55756c82b7a0 2024-04-25T19:02:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.755+0000 7fd92e000700 1 -- 172.21.15.45:0/1131056430 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9280a1120 con 0x55756c82b7a0 2024-04-25T19:02:05.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.756+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/1131056430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 msgr2=0x55756c83a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.756+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/1131056430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 0x55756c83a520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd928099640 tx=0x7fd92809cb40 comp rx=0 tx=0).stop 2024-04-25T19:02:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.756+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/1131056430 shutdown_connections 2024-04-25T19:02:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.756+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/1131056430 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c83aa60 0x55756c83ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.756+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/1131056430 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c82ae80 0x55756c82b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.756+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/1131056430 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 0x55756c83a520 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.756+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/1131056430 >> 172.21.15.45:0/1131056430 conn(0x55756c75a030 msgr2=0x55756c8273d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.756+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/1131056430 shutdown_connections 2024-04-25T19:02:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.756+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/1131056430 wait complete. 2024-04-25T19:02:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.756+0000 7fd94e3e8b80 1 -- start start 2024-04-25T19:02:05.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.757+0000 7fd94e3e8b80 1 Processor -- start 2024-04-25T19:02:05.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.757+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82ae80 0x55756c83a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.757+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 0x55756c837580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.757+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c83aa60 0x55756c837ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.757+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55756c779f30 con 0x55756c83aa60 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.757+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55756c737940 con 0x55756c82ae80 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.757+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55756c83d390 con 0x55756c82b7a0 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.757+0000 7fd94e3e8b80 10 client.0 _ll_register_callbacks cb 0x55756c716800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd941000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82ae80 0x55756c83a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd941a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c83aa60 0x55756c837ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd941000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82ae80 0x55756c83a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:50640/0 (socket says 172.21.15.45:50640) 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd941000700 1 -- 172.21.15.45:0/3949222214 learned_addr learned my addr 172.21.15.45:0/3949222214 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd93be00700 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 0x55756c837580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd941000700 1 -- 172.21.15.45:0/3949222214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 msgr2=0x55756c837580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd941000700 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 0x55756c837580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd941000700 1 -- 172.21.15.45:0/3949222214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c83aa60 msgr2=0x55756c837ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd941000700 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c83aa60 0x55756c837ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd941000700 1 -- 172.21.15.45:0/3949222214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756c6cb9b0 con 0x55756c82ae80 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd941000700 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82ae80 0x55756c83a280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9300116c0 tx=0x7fd9300047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4744dee0d830d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:05.761 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.758+0000 7fd922a00700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.759+0000 7fd922a00700 1 -- 172.21.15.45:0/3949222214 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930004220 con 0x55756c82ae80 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.759+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/3949222214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9280a1590 con 0x55756c82ae80 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.759+0000 7fd922a00700 1 -- 172.21.15.45:0/3949222214 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fd9300043c0 con 0x55756c82ae80 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.759+0000 7fd922a00700 1 -- 172.21.15.45:0/3949222214 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd930003680 con 0x55756c82ae80 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.759+0000 7fd94e3e8b80 10 client.25505 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.759+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/3949222214 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55756c714bd0 con 0x55756c82ae80 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.760+0000 7fd922a00700 1 -- 172.21.15.45:0/3949222214 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9280a1590 con 0x55756c82ae80 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.760+0000 7fd922a00700 1 -- 172.21.15.45:0/3949222214 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fd930026030 con 0x55756c82ae80 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.760+0000 7fd94e3e8b80 10 client.25505 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.760+0000 7fd94e3e8b80 10 client.25505 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.760+0000 7fd94e3e8b80 20 client.25505 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:05.763 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.760+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55756c849f90 0x55756c84c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.760+0000 7fd94e3e8b80 4 client.25505 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:05.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.760+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/3949222214 --> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55756c78b760 con 0x55756c849f90 2024-04-25T19:02:05.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.760+0000 7fd93be00700 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55756c849f90 0x55756c84c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.761+0000 7fd93be00700 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55756c849f90 0x55756c84c380 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73e40235ad0bc315 server_cookie=a11b7d89a33258b1 in_seq=0 out_seq=0 2024-04-25T19:02:05.764 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.761+0000 7fd922a00700 10 client.25505 ms_handle_connect on v2:172.21.15.45:6840/2490735053 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd922a00700 1 -- 172.21.15.45:0/3949222214 <== mds.1 v2:172.21.15.45:6840/2490735053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55756c78b760 con 0x55756c849f90 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd922a00700 10 client.25505 handle_command_reply: tid=0 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 client.25505 shutdown 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/3949222214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82ae80 msgr2=0x55756c83a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82ae80 0x55756c83a280 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd9300116c0 tx=0x7fd9300047c0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/3949222214 shutdown_connections 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55756c849f90 0x55756c84c380 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82ae80 0x55756c83a280 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c83aa60 0x55756c837ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/3949222214 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c82b7a0 0x55756c837580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/3949222214 >> 172.21.15.45:0/3949222214 conn(0x55756c75a030 msgr2=0x55756c826fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd922a00700 0 client.25505 ms_handle_reset on v2:172.21.15.45:6840/2490735053 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/3949222214 shutdown_connections 2024-04-25T19:02:05.765 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.762+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/3949222214 wait complete. 2024-04-25T19:02:05.766 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.763+0000 7fd94e3e8b80 20 client.25505 trim_cache size 0 max 16384 2024-04-25T19:02:05.784 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd94e3e8b80 1 Processor -- start 2024-04-25T19:02:05.785 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd94e3e8b80 1 -- start start 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 0x55756c832b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c83aa60 0x55756c8460d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c851be0 0x55756c852060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1ad930 con 0x55756c851be0 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1ab1c0 con 0x55756c82b7a0 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1a9010 con 0x55756c83aa60 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd941a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c851be0 0x55756c852060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd941a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c851be0 0x55756c852060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:43870/0 (socket says 172.21.15.45:43870) 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd93be00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c83aa60 0x55756c8460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd941a00700 1 -- 172.21.15.45:0/2924848242 learned_addr learned my addr 172.21.15.45:0/2924848242 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:05.786 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.783+0000 7fd941000700 1 --2- 172.21.15.45:0/2924848242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 0x55756c832b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.784+0000 7fd93be00700 1 -- 172.21.15.45:0/2924848242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 msgr2=0x55756c832b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.784+0000 7fd93be00700 1 --2- 172.21.15.45:0/2924848242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 0x55756c832b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.784+0000 7fd93be00700 1 -- 172.21.15.45:0/2924848242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c851be0 msgr2=0x55756c852060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.784+0000 7fd93be00700 1 --2- 172.21.15.45:0/2924848242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c851be0 0x55756c852060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.784+0000 7fd93be00700 1 -- 172.21.15.45:0/2924848242 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55756c84c9b0 con 0x55756c83aa60 2024-04-25T19:02:05.787 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.784+0000 7fd93be00700 1 --2- 172.21.15.45:0/2924848242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c83aa60 0x55756c8460d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd928099640 tx=0x7fd9280d1110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bab580fc597b4b32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.784+0000 7fd92d600700 1 -- 172.21.15.45:0/2924848242 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9280bb070 con 0x55756c83aa60 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.784+0000 7fd92d600700 1 -- 172.21.15.45:0/2924848242 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9280a1590 con 0x55756c83aa60 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.784+0000 7fd92d600700 1 -- 172.21.15.45:0/2924848242 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92800b660 con 0x55756c83aa60 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.785+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2924848242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c83aa60 msgr2=0x55756c8460d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.785+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/2924848242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c83aa60 0x55756c8460d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd928099640 tx=0x7fd9280d1110 comp rx=0 tx=0).stop 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.785+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2924848242 shutdown_connections 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.785+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/2924848242 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 0x55756c832b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.785+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/2924848242 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c851be0 0x55756c852060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.785+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/2924848242 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c83aa60 0x55756c8460d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.785+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2924848242 >> 172.21.15.45:0/2924848242 conn(0x55756c75a030 msgr2=0x55756c836700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.785+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2924848242 shutdown_connections 2024-04-25T19:02:05.788 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.785+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2924848242 wait complete. 2024-04-25T19:02:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.786+0000 7fd94e3e8b80 1 -- start start 2024-04-25T19:02:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.786+0000 7fd94e3e8b80 1 Processor -- start 2024-04-25T19:02:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.786+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 0x55756c842810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.789 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.786+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c83aa60 0x55756c83fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.786+0000 7fd94e3e8b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c851be0 0x55756c8400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.786+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1a9680 con 0x55756c83aa60 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.786+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1aaaf0 con 0x55756c82b7a0 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.786+0000 7fd94e3e8b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c1a9d50 con 0x55756c851be0 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.786+0000 7fd94e3e8b80 10 client.0 _ll_register_callbacks cb 0x55756c827a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 0x55756c842810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c851be0 0x55756c8400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 0x55756c842810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:50664/0 (socket says 172.21.15.45:50664) 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941000700 1 -- 172.21.15.45:0/2170696053 learned_addr learned my addr 172.21.15.45:0/2170696053 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd93be00700 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c83aa60 0x55756c83fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941000700 1 -- 172.21.15.45:0/2170696053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c851be0 msgr2=0x55756c8400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941000700 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c851be0 0x55756c8400f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941000700 1 -- 172.21.15.45:0/2170696053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c83aa60 msgr2=0x55756c83fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941000700 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c83aa60 0x55756c83fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.790 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941000700 1 -- 172.21.15.45:0/2170696053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55756c82b4b0 con 0x55756c82b7a0 2024-04-25T19:02:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941a00700 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c851be0 0x55756c8400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd941000700 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 0x55756c842810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd930004790 tx=0x7fd930011cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf61ae4e4e715fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:05.791 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.787+0000 7fd922a00700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:02:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.788+0000 7fd922a00700 1 -- 172.21.15.45:0/2170696053 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd930036070 con 0x55756c82b7a0 2024-04-25T19:02:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.788+0000 7fd922a00700 1 -- 172.21.15.45:0/2170696053 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fd930030680 con 0x55756c82b7a0 2024-04-25T19:02:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.788+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2170696053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9280a1590 con 0x55756c82b7a0 2024-04-25T19:02:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.788+0000 7fd922a00700 1 -- 172.21.15.45:0/2170696053 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd930047050 con 0x55756c82b7a0 2024-04-25T19:02:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.788+0000 7fd94e3e8b80 10 client.25511 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:05.792 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.788+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2170696053 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55756c827ea0 con 0x55756c82b7a0 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.789+0000 7fd922a00700 1 -- 172.21.15.45:0/2170696053 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9280a1590 con 0x55756c82b7a0 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.789+0000 7fd922a00700 1 -- 172.21.15.45:0/2170696053 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fd930042030 con 0x55756c82b7a0 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.789+0000 7fd94e3e8b80 10 client.25511 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.789+0000 7fd94e3e8b80 10 client.25511 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.a 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.789+0000 7fd94e3e8b80 20 client.25511 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.789+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55756c8b26f0 0x55756c8b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.789+0000 7fd94e3e8b80 4 client.25511 mds_command: new command op to 24512 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-695e6b8e-1-mds.a.json"}] 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.789+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2170696053 --> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-695e6b8e-1-mds.a.json"}) -- 0x55756c785800 con 0x55756c8b26f0 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.789+0000 7fd93be00700 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55756c8b26f0 0x55756c8b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.790+0000 7fd93be00700 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55756c8b26f0 0x55756c8b4b10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a8be93de4ccd4dd server_cookie=128acbf7627ef0b7 in_seq=0 out_seq=0 2024-04-25T19:02:05.793 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.790+0000 7fd922a00700 10 client.25511 ms_handle_connect on v2:172.21.15.45:6840/2490735053 2024-04-25T19:02:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd922a00700 1 -- 172.21.15.45:0/2170696053 <== mds.1 v2:172.21.15.45:6840/2490735053 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x55756c785800 con 0x55756c8b26f0 2024-04-25T19:02:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd922a00700 10 client.25511 handle_command_reply: tid=0 2024-04-25T19:02:05.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd94e3e8b80 1 client.25511 shutdown 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2170696053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 msgr2=0x55756c842810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 0x55756c842810 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd930004790 tx=0x7fd930011cb0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2170696053 shutdown_connections 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.45:6840/2490735053,v1:172.21.15.45:6841/2490735053] conn(0x55756c8b26f0 0x55756c8b4b10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55756c82b7a0 0x55756c842810 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55756c851be0 0x55756c8400f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd94e3e8b80 1 --2- 172.21.15.45:0/2170696053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55756c83aa60 0x55756c83fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2170696053 >> 172.21.15.45:0/2170696053 conn(0x55756c75a030 msgr2=0x55756c836700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.792+0000 7fd922a00700 0 client.25511 ms_handle_reset on v2:172.21.15.45:6840/2490735053 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.793+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2170696053 shutdown_connections 2024-04-25T19:02:05.795 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.793+0000 7fd94e3e8b80 1 -- 172.21.15.45:0/2170696053 wait complete. 2024-04-25T19:02:05.796 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.793+0000 7fd94e3e8b80 20 client.25511 trim_cache size 0 max 16384 2024-04-25T19:02:05.796 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:02:05.796 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/var/run/ceph/ops-695e6b8e-1-mds.a.json" 2024-04-25T19:02:05.796 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.793+0000 7fd942400700 1 -- 172.21.15.45:0/3970736633 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9240a3db0 msgr2=0x7fd9240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.793+0000 7fd942400700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9240a3db0 0x7fd9240a6270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd930007be0 tx=0x7fd9300073d0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.793+0000 7fd942400700 1 -- 172.21.15.45:0/3970736633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd93c1044c0 msgr2=0x7fd93c199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.793+0000 7fd942400700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd93c1044c0 0x7fd93c199710 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd928009230 tx=0x7fd92800f990 comp rx=0 tx=0).stop 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.794+0000 7fd942400700 1 -- 172.21.15.45:0/3970736633 shutdown_connections 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.794+0000 7fd942400700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd9240a3db0 0x7fd9240a6270 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.794+0000 7fd942400700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd93c10cfd0 0x7fd93c19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.794+0000 7fd942400700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd93c103ba0 0x7fd93c1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.794+0000 7fd942400700 1 --2- 172.21.15.45:0/3970736633 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd93c1044c0 0x7fd93c199710 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.794+0000 7fd942400700 1 -- 172.21.15.45:0/3970736633 >> 172.21.15.45:0/3970736633 conn(0x7fd93c0fd3f0 msgr2=0x7fd93c0fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.794+0000 7fd942400700 1 -- 172.21.15.45:0/3970736633 shutdown_connections 2024-04-25T19:02:05.797 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:05.794+0000 7fd942400700 1 -- 172.21.15.45:0/3970736633 wait complete. 2024-04-25T19:02:06.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:06 smithi062 ceph-mon[32844]: pgmap v86: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-25T19:02:06.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:06 smithi079 ceph-mon[33002]: pgmap v86: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-25T19:02:06.703 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:06 smithi045 ceph-mon[27776]: pgmap v86: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-04-25T19:02:06.779 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 0 (b) to a remote file /var/run/ceph/1/ops-695e6b8e-1-mds.b.json 2024-04-25T19:02:06.780 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph tell mds.1:0 ops --flags=locks --path=/var/run/ceph/ops-695e6b8e-1-mds.b.json 2024-04-25T19:02:07.238 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:08.448 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:08 smithi045 ceph-mon[27776]: pgmap v87: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T19:02:08.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:08 smithi062 ceph-mon[32844]: pgmap v87: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T19:02:08.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:08 smithi079 ceph-mon[33002]: pgmap v87: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T19:02:09.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.433+0000 7efc76800700 1 -- 172.21.15.45:0/1115757460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70076f70 msgr2=0x7efc70077350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:09.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.433+0000 7efc76800700 1 --2- 172.21.15.45:0/1115757460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70076f70 0x7efc70077350 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7efc60009990 tx=0x7efc60009ca0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.433+0000 7efc76800700 1 -- 172.21.15.45:0/1115757460 shutdown_connections 2024-04-25T19:02:09.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.433+0000 7efc76800700 1 --2- 172.21.15.45:0/1115757460 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc70113920 0x7efc70115d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.433+0000 7efc76800700 1 --2- 172.21.15.45:0/1115757460 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70076f70 0x7efc70077350 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.433+0000 7efc76800700 1 --2- 172.21.15.45:0/1115757460 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc70077890 0x7efc701133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.433+0000 7efc76800700 1 -- 172.21.15.45:0/1115757460 >> 172.21.15.45:0/1115757460 conn(0x7efc70100770 msgr2=0x7efc70102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:09.436 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.433+0000 7efc76800700 1 -- 172.21.15.45:0/1115757460 shutdown_connections 2024-04-25T19:02:09.437 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.433+0000 7efc76800700 1 -- 172.21.15.45:0/1115757460 wait complete. 2024-04-25T19:02:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.434+0000 7efc76800700 1 Processor -- start 2024-04-25T19:02:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc76800700 1 -- start start 2024-04-25T19:02:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc76800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc70076f70 0x7efc7019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc76800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70077890 0x7efc7019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc76800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc70113920 0x7efc701a4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc76800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc70119290 con 0x7efc70077890 2024-04-25T19:02:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc76800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc70119110 con 0x7efc70076f70 2024-04-25T19:02:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc76800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efc70119410 con 0x7efc70113920 2024-04-25T19:02:09.438 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc6f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc70076f70 0x7efc7019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc67e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70077890 0x7efc7019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc6f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc70076f70 0x7efc7019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:50688/0 (socket says 172.21.15.45:50688) 2024-04-25T19:02:09.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc6f400700 1 -- 172.21.15.45:0/2645576826 learned_addr learned my addr 172.21.15.45:0/2645576826 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:09.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.435+0000 7efc6fe00700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc70113920 0x7efc701a4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.436+0000 7efc6f400700 1 -- 172.21.15.45:0/2645576826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc70113920 msgr2=0x7efc701a4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:09.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.436+0000 7efc6f400700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc70113920 0x7efc701a4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.436+0000 7efc6f400700 1 -- 172.21.15.45:0/2645576826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70077890 msgr2=0x7efc7019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:09.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.436+0000 7efc6f400700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70077890 0x7efc7019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.436+0000 7efc6f400700 1 -- 172.21.15.45:0/2645576826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc701a4740 con 0x7efc70076f70 2024-04-25T19:02:09.439 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.436+0000 7efc6f400700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc70076f70 0x7efc7019f780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efc600057c0 tx=0x7efc60009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:09.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.436+0000 7efc6cc00700 1 -- 172.21.15.45:0/2645576826 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60026070 con 0x7efc70076f70 2024-04-25T19:02:09.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.436+0000 7efc6cc00700 1 -- 172.21.15.45:0/2645576826 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc60021410 con 0x7efc70076f70 2024-04-25T19:02:09.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.436+0000 7efc76800700 1 -- 172.21.15.45:0/2645576826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc701a49d0 con 0x7efc70076f70 2024-04-25T19:02:09.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.436+0000 7efc6cc00700 1 -- 172.21.15.45:0/2645576826 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc60017650 con 0x7efc70076f70 2024-04-25T19:02:09.440 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.437+0000 7efc76800700 1 -- 172.21.15.45:0/2645576826 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc701ac2b0 con 0x7efc70076f70 2024-04-25T19:02:09.442 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.440+0000 7efc6cc00700 1 -- 172.21.15.45:0/2645576826 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7efc60029070 con 0x7efc70076f70 2024-04-25T19:02:09.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.442+0000 7efc8110eb80 1 Processor -- start 2024-04-25T19:02:09.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.442+0000 7efc6cc00700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efc540a3db0 0x7efc540a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.444 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.442+0000 7efc8110eb80 1 -- start start 2024-04-25T19:02:09.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.442+0000 7efc6cc00700 1 -- 172.21.15.45:0/2645576826 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7efc600d1510 con 0x7efc70076f70 2024-04-25T19:02:09.445 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc6fe00700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efc540a3db0 0x7efc540a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068ce6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068cf0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cf38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc8110eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557068c38f70 con 0x557068ce6640 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc8110eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557068bdf810 con 0x557068cf14b0 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc8110eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557068c34f30 con 0x557068ce5d20 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc67e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068cf0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc6f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068ce6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc6f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068ce6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:38178/0 (socket says 172.21.15.45:38178) 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.443+0000 7efc6f400700 1 -- 172.21.15.45:0/3532057619 learned_addr learned my addr 172.21.15.45:0/3532057619 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:09.446 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.444+0000 7efc6fe00700 1 --2- 172.21.15.45:0/3532057619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cf38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.444+0000 7efc67e00700 1 -- 172.21.15.45:0/3532057619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 msgr2=0x557068ce6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.444+0000 7efc67e00700 1 --2- 172.21.15.45:0/3532057619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068ce6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.444+0000 7efc67e00700 1 -- 172.21.15.45:0/3532057619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 msgr2=0x557068cf38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.444+0000 7efc67e00700 1 --2- 172.21.15.45:0/3532057619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cf38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.444+0000 7efc67e00700 1 -- 172.21.15.45:0/3532057619 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557068bd5760 con 0x557068ce6640 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.444+0000 7efc6fe00700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efc540a3db0 0x7efc540a6270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc7006b500 tx=0x7efc6800b040 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.444+0000 7efc67e00700 1 --2- 172.21.15.45:0/3532057619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068cf0f70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efc58007bc0 tx=0x7efc58007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2ca123a55f21b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.445+0000 7efc64c00700 1 -- 172.21.15.45:0/3532057619 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58004310 con 0x557068ce6640 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.445+0000 7efc64c00700 1 -- 172.21.15.45:0/3532057619 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc58015070 con 0x557068ce6640 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.445+0000 7efc64c00700 1 -- 172.21.15.45:0/3532057619 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc58005020 con 0x557068ce6640 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.445+0000 7efc8110eb80 1 -- 172.21.15.45:0/3532057619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 msgr2=0x557068cf0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:09.448 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.445+0000 7efc8110eb80 1 --2- 172.21.15.45:0/3532057619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068cf0f70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efc58007bc0 tx=0x7efc58007ed0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.446+0000 7efc8110eb80 1 -- 172.21.15.45:0/3532057619 shutdown_connections 2024-04-25T19:02:09.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.446+0000 7efc8110eb80 1 --2- 172.21.15.45:0/3532057619 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cf38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.446+0000 7efc8110eb80 1 --2- 172.21.15.45:0/3532057619 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068ce6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.446+0000 7efc8110eb80 1 --2- 172.21.15.45:0/3532057619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068cf0f70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.446+0000 7efc8110eb80 1 -- 172.21.15.45:0/3532057619 >> 172.21.15.45:0/3532057619 conn(0x557068c15030 msgr2=0x557068ce2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:09.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.446+0000 7efc8110eb80 1 -- 172.21.15.45:0/3532057619 shutdown_connections 2024-04-25T19:02:09.449 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.446+0000 7efc8110eb80 1 -- 172.21.15.45:0/3532057619 wait complete. 2024-04-25T19:02:09.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.446+0000 7efc8110eb80 1 -- start start 2024-04-25T19:02:09.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc8110eb80 1 Processor -- start 2024-04-25T19:02:09.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068cf0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068cedee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cee420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc8110eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557068cf3fa0 con 0x557068ce6640 2024-04-25T19:02:09.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc8110eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557068bf2940 con 0x557068cf14b0 2024-04-25T19:02:09.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc8110eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557068cf4120 con 0x557068ce5d20 2024-04-25T19:02:09.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc8110eb80 10 client.0 _ll_register_callbacks cb 0x557068bd1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:09.452 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc6f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068cf0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc6f400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068cf0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:38180/0 (socket says 172.21.15.45:38180) 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc6f400700 1 -- 172.21.15.45:0/3428442224 learned_addr learned my addr 172.21.15.45:0/3428442224 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.447+0000 7efc67e00700 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068cedee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.448+0000 7efc6fe00700 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cee420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.448+0000 7efc6f400700 1 -- 172.21.15.45:0/3428442224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 msgr2=0x557068cee420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.448+0000 7efc6f400700 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cee420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.448+0000 7efc6f400700 1 -- 172.21.15.45:0/3428442224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 msgr2=0x557068cedee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.448+0000 7efc6f400700 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068cedee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.448+0000 7efc6f400700 1 -- 172.21.15.45:0/3428442224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557068b869b0 con 0x557068ce5d20 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.448+0000 7efc6f400700 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068cf0b70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc60028000 tx=0x7efc6009d340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1beedf834f6d44d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.448+0000 7efc5d600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.448+0000 7efc5d600700 1 -- 172.21.15.45:0/3428442224 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc600b6060 con 0x557068ce5d20 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc5d600700 1 -- 172.21.15.45:0/3428442224 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7efc6009d9d0 con 0x557068ce5d20 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc8110eb80 1 -- 172.21.15.45:0/3428442224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc58015070 con 0x557068ce5d20 2024-04-25T19:02:09.453 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc5d600700 1 -- 172.21.15.45:0/3428442224 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efc6009ced0 con 0x557068ce5d20 2024-04-25T19:02:09.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc8110eb80 10 client.25543 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:09.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc8110eb80 1 -- 172.21.15.45:0/3428442224 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557068bcfbd0 con 0x557068ce5d20 2024-04-25T19:02:09.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc5d600700 1 -- 172.21.15.45:0/3428442224 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc58015070 con 0x557068ce5d20 2024-04-25T19:02:09.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc5d600700 1 -- 172.21.15.45:0/3428442224 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7efc600a5cc0 con 0x557068ce5d20 2024-04-25T19:02:09.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc8110eb80 10 client.25543 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:09.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc8110eb80 10 client.25543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T19:02:09.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc8110eb80 20 client.25543 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:09.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc8110eb80 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557068d00c00 0x557068d03030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:09.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc8110eb80 4 client.25543 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:09.454 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.449+0000 7efc8110eb80 1 -- 172.21.15.45:0/3428442224 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557068c46760 con 0x557068d00c00 2024-04-25T19:02:09.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.450+0000 7efc67e00700 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557068d00c00 0x557068d03030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:09.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.450+0000 7efc67e00700 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557068d00c00 0x557068d03030 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adfa24a7cb135036 server_cookie=b68de7a8e2933a4c in_seq=0 out_seq=0 2024-04-25T19:02:09.455 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:09.450+0000 7efc5d600700 10 client.25543 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:10.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:10 smithi062 ceph-mon[32844]: pgmap v88: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T19:02:10.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:10 smithi079 ceph-mon[33002]: pgmap v88: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T19:02:10.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:10 smithi045 ceph-mon[27776]: pgmap v88: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T19:02:12.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:12 smithi062 ceph-mon[32844]: pgmap v89: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T19:02:12.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:12 smithi079 ceph-mon[33002]: pgmap v89: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T19:02:12.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:12 smithi045 ceph-mon[27776]: pgmap v89: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T19:02:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93bbe00640 1 -- 172.21.15.45:0/3985593364 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eabe910320 con 0x55eabea1ee30 2024-04-25T19:02:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93bbe00640 10 client.25469 handle_command_reply: tid=0 2024-04-25T19:02:12.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93e03d5740 1 client.25469 shutdown 2024-04-25T19:02:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93e03d5740 1 -- 172.21.15.45:0/3985593364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 msgr2=0x55eabe9adbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93e03d5740 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9adbd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55eabe9ba3d0 tx=0x7f93c0005970 comp rx=0 tx=0).stop 2024-04-25T19:02:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93e03d5740 1 -- 172.21.15.45:0/3985593364 shutdown_connections 2024-04-25T19:02:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93e03d5740 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55eabea1ee30 0x55eabea21250 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93e03d5740 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55eabe9a7a00 0x55eabe9ae110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93e03d5740 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55eabe99d140 0x55eabe9adbd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93e03d5740 1 --2- 172.21.15.45:0/3985593364 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55eabe99c820 0x55eabe9ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93e03d5740 1 -- 172.21.15.45:0/3985593364 >> 172.21.15.45:0/3985593364 conn(0x55eabe998d30 msgr2=0x55eabe9a5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:12.898 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.895+0000 7efc5d600700 1 -- 172.21.15.45:0/3428442224 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557068c46760 con 0x557068d00c00 2024-04-25T19:02:12.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.895+0000 7efc5d600700 10 client.25543 handle_command_reply: tid=0 2024-04-25T19:02:12.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.895+0000 7efc8110eb80 1 client.25543 shutdown 2024-04-25T19:02:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.895+0000 7f93bbe00640 0 client.25469 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:12.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.896+0000 7f93e03d5740 1 -- 172.21.15.45:0/3985593364 shutdown_connections 2024-04-25T19:02:12.899 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc8110eb80 1 -- 172.21.15.45:0/3428442224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 msgr2=0x557068cf0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc8110eb80 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068cf0b70 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc60028000 tx=0x7efc6009d340 comp rx=0 tx=0).stop 2024-04-25T19:02:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.896+0000 7f93e03d5740 1 -- 172.21.15.45:0/3985593364 wait complete. 2024-04-25T19:02:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc8110eb80 1 -- 172.21.15.45:0/3428442224 shutdown_connections 2024-04-25T19:02:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc8110eb80 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557068d00c00 0x557068d03030 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc8110eb80 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cee420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc8110eb80 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068cf0b70 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc8110eb80 1 --2- 172.21.15.45:0/3428442224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068cedee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.900 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc5d600700 0 client.25543 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:12.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc8110eb80 1 -- 172.21.15.45:0/3428442224 >> 172.21.15.45:0/3428442224 conn(0x557068c15030 msgr2=0x557068ce1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.896+0000 7f93e03d5740 20 client.25469 trim_cache size 0 max 16384 2024-04-25T19:02:12.901 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc8110eb80 1 -- 172.21.15.45:0/3428442224 shutdown_connections 2024-04-25T19:02:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:02:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:02:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:02:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:02:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.896+0000 7efc8110eb80 1 -- 172.21.15.45:0/3428442224 wait complete. 2024-04-25T19:02:12.902 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.897+0000 7efc8110eb80 20 client.25543 trim_cache size 0 max 16384 2024-04-25T19:02:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.897+0000 7f93dd400640 1 -- 172.21.15.45:0/1505528326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f93bc0a3d90 msgr2=0x7f93bc0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.897+0000 7f93dd400640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f93bc0a3d90 0x7f93bc0a6250 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x55eabe99ddc0 tx=0x7f93c80d44e0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.897+0000 7f93dd400640 1 -- 172.21.15.45:0/1505528326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93d81085a0 msgr2=0x7f93d819ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.897+0000 7f93dd400640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93d81085a0 0x7f93d819ee20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93c800e9f0 tx=0x7f93c800eeb0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.898+0000 7f93dd400640 1 -- 172.21.15.45:0/1505528326 shutdown_connections 2024-04-25T19:02:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.898+0000 7f93dd400640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f93bc0a3d90 0x7f93bc0a6250 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.898+0000 7f93dd400640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f93d810e660 0x7f93d81a36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.898+0000 7f93dd400640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93d8108f60 0x7f93d819f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.898+0000 7f93dd400640 1 --2- 172.21.15.45:0/1505528326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f93d81085a0 0x7f93d819ee20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.898+0000 7f93dd400640 1 -- 172.21.15.45:0/1505528326 >> 172.21.15.45:0/1505528326 conn(0x7f93d8076460 msgr2=0x7f93d8077ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.898+0000 7f93dd400640 1 -- 172.21.15.45:0/1505528326 shutdown_connections 2024-04-25T19:02:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:12.898+0000 7f93dd400640 1 -- 172.21.15.45:0/1505528326 wait complete. 2024-04-25T19:02:12.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.924+0000 7efc8110eb80 1 Processor -- start 2024-04-25T19:02:12.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.924+0000 7efc8110eb80 1 -- start start 2024-04-25T19:02:12.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068d08940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:12.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068d08e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:12.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cf93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:12.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc8110eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc701b6210 con 0x557068ce6640 2024-04-25T19:02:12.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc8110eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc701b3aa0 con 0x557068cf14b0 2024-04-25T19:02:12.927 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc8110eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efc701b18f0 con 0x557068ce5d20 2024-04-25T19:02:12.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc6fe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cf93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:12.928 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc6fe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cf93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:50720/0 (socket says 172.21.15.45:50720) 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc6fe00700 1 -- 172.21.15.45:0/4253988453 learned_addr learned my addr 172.21.15.45:0/4253988453 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc67e00700 1 --2- 172.21.15.45:0/4253988453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068d08e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.925+0000 7efc6f400700 1 --2- 172.21.15.45:0/4253988453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068d08940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.926+0000 7efc6fe00700 1 -- 172.21.15.45:0/4253988453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 msgr2=0x557068d08940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.926+0000 7efc6fe00700 1 --2- 172.21.15.45:0/4253988453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068d08940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.926+0000 7efc6fe00700 1 -- 172.21.15.45:0/4253988453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 msgr2=0x557068d08e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.926+0000 7efc6fe00700 1 --2- 172.21.15.45:0/4253988453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068d08e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.926+0000 7efc6fe00700 1 -- 172.21.15.45:0/4253988453 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557068ce7b90 con 0x557068cf14b0 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.926+0000 7efc6fe00700 1 --2- 172.21.15.45:0/4253988453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cf93a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efc68009390 tx=0x7efc6801c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d305e66d04a1cb74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.927+0000 7efc5fe00700 1 -- 172.21.15.45:0/4253988453 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc680093c0 con 0x557068cf14b0 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.927+0000 7efc5fe00700 1 -- 172.21.15.45:0/4253988453 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc68026070 con 0x557068cf14b0 2024-04-25T19:02:12.929 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.927+0000 7efc5fe00700 1 -- 172.21.15.45:0/4253988453 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc680178c0 con 0x557068cf14b0 2024-04-25T19:02:12.930 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.928+0000 7efc8110eb80 1 -- 172.21.15.45:0/4253988453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 msgr2=0x557068cf93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.928+0000 7efc8110eb80 1 --2- 172.21.15.45:0/4253988453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cf93a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7efc68009390 tx=0x7efc6801c000 comp rx=0 tx=0).stop 2024-04-25T19:02:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.928+0000 7efc8110eb80 1 -- 172.21.15.45:0/4253988453 shutdown_connections 2024-04-25T19:02:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.928+0000 7efc8110eb80 1 --2- 172.21.15.45:0/4253988453 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068cf14b0 0x557068cf93a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.928+0000 7efc8110eb80 1 --2- 172.21.15.45:0/4253988453 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068d08940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.928+0000 7efc8110eb80 1 --2- 172.21.15.45:0/4253988453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068ce6640 0x557068d08e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.928+0000 7efc8110eb80 1 -- 172.21.15.45:0/4253988453 >> 172.21.15.45:0/4253988453 conn(0x557068c15030 msgr2=0x557068ced060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.928+0000 7efc8110eb80 1 -- 172.21.15.45:0/4253988453 shutdown_connections 2024-04-25T19:02:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.928+0000 7efc8110eb80 1 -- 172.21.15.45:0/4253988453 wait complete. 2024-04-25T19:02:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.929+0000 7efc8110eb80 1 -- start start 2024-04-25T19:02:12.931 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.929+0000 7efc8110eb80 1 Processor -- start 2024-04-25T19:02:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.929+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068cf5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:12.932 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.929+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068ce6640 0x557068cf6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.929+0000 7efc8110eb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068cf14b0 0x557068cf6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.929+0000 7efc8110eb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc701b1f60 con 0x557068cf14b0 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.929+0000 7efc8110eb80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efc701b33d0 con 0x557068ce6640 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.929+0000 7efc8110eb80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efc701b2630 con 0x557068ce5d20 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc8110eb80 10 client.0 _ll_register_callbacks cb 0x557068bd1800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc67e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068ce6640 0x557068cf6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc67e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068ce6640 0x557068cf6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:50728/0 (socket says 172.21.15.45:50728) 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc67e00700 1 -- 172.21.15.45:0/2859068295 learned_addr learned my addr 172.21.15.45:0/2859068295 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc6fe00700 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068cf14b0 0x557068cf6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc6f400700 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068cf5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc67e00700 1 -- 172.21.15.45:0/2859068295 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 msgr2=0x557068cf5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc67e00700 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068cf5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc67e00700 1 -- 172.21.15.45:0/2859068295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068cf14b0 msgr2=0x557068cf6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:12.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc67e00700 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068cf14b0 0x557068cf6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.930+0000 7efc67e00700 1 -- 172.21.15.45:0/2859068295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557068d671c0 con 0x557068ce6640 2024-04-25T19:02:12.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.931+0000 7efc67e00700 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068ce6640 0x557068cf6220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efc58007670 tx=0x7efc58005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966dcd5eabd7604f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:12.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.931+0000 7efc5d600700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:02:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.931+0000 7efc5d600700 1 -- 172.21.15.45:0/2859068295 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5802d020 con 0x557068ce6640 2024-04-25T19:02:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.931+0000 7efc5d600700 1 -- 172.21.15.45:0/2859068295 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7efc58010920 con 0x557068ce6640 2024-04-25T19:02:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.931+0000 7efc8110eb80 1 -- 172.21.15.45:0/2859068295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc68026070 con 0x557068ce6640 2024-04-25T19:02:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.931+0000 7efc5d600700 1 -- 172.21.15.45:0/2859068295 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7efc58038050 con 0x557068ce6640 2024-04-25T19:02:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.931+0000 7efc8110eb80 10 client.25529 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.931+0000 7efc8110eb80 1 -- 172.21.15.45:0/2859068295 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557068d683f0 con 0x557068ce6640 2024-04-25T19:02:12.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.931+0000 7efc5d600700 1 -- 172.21.15.45:0/2859068295 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc68026070 con 0x557068ce6640 2024-04-25T19:02:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.932+0000 7efc5d600700 1 -- 172.21.15.45:0/2859068295 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7efc580296f0 con 0x557068ce6640 2024-04-25T19:02:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.932+0000 7efc8110eb80 10 client.25529 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.932+0000 7efc8110eb80 10 client.25529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T19:02:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.932+0000 7efc8110eb80 20 client.25529 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.932+0000 7efc8110eb80 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557068d68930 0x557068d6ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.932+0000 7efc8110eb80 4 client.25529 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-695e6b8e-1-mds.b.json"}] 2024-04-25T19:02:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.932+0000 7efc8110eb80 1 -- 172.21.15.45:0/2859068295 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-695e6b8e-1-mds.b.json"}) -- 0x557068c40800 con 0x557068d68930 2024-04-25T19:02:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.932+0000 7efc6f400700 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557068d68930 0x557068d6ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.933+0000 7efc6f400700 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557068d68930 0x557068d6ad50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e99764a7fe50422 server_cookie=5c8d00ca682b3f91 in_seq=0 out_seq=0 2024-04-25T19:02:12.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.933+0000 7efc5d600700 10 client.25529 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.944+0000 7efc5d600700 1 -- 172.21.15.45:0/2859068295 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x557068c40800 con 0x557068d68930 2024-04-25T19:02:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.944+0000 7efc5d600700 10 client.25529 handle_command_reply: tid=0 2024-04-25T19:02:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.944+0000 7efc8110eb80 1 client.25529 shutdown 2024-04-25T19:02:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 1 -- 172.21.15.45:0/2859068295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068ce6640 msgr2=0x557068cf6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068ce6640 0x557068cf6220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7efc58007670 tx=0x7efc58005cf0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 1 -- 172.21.15.45:0/2859068295 shutdown_connections 2024-04-25T19:02:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557068d68930 0x557068d6ad50 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.947 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557068cf14b0 0x557068cf6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557068ce5d20 0x557068cf5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 1 --2- 172.21.15.45:0/2859068295 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557068ce6640 0x557068cf6220 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc5d600700 0 client.25529 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 1 -- 172.21.15.45:0/2859068295 >> 172.21.15.45:0/2859068295 conn(0x557068c15030 msgr2=0x557068ce13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 1 -- 172.21.15.45:0/2859068295 shutdown_connections 2024-04-25T19:02:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 1 -- 172.21.15.45:0/2859068295 wait complete. 2024-04-25T19:02:12.948 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.945+0000 7efc8110eb80 20 client.25529 trim_cache size 0 max 16384 2024-04-25T19:02:12.949 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:02:12.949 INFO:teuthology.orchestra.run.smithi045.stdout: "path": "/var/run/ceph/ops-695e6b8e-1-mds.b.json" 2024-04-25T19:02:12.949 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:02:12.949 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.946+0000 7efc76800700 1 -- 172.21.15.45:0/2645576826 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efc540a3db0 msgr2=0x7efc540a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.946+0000 7efc76800700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efc540a3db0 0x7efc540a6270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc7006b500 tx=0x7efc6800b040 comp rx=0 tx=0).stop 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.946+0000 7efc76800700 1 -- 172.21.15.45:0/2645576826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc70076f70 msgr2=0x7efc7019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.946+0000 7efc76800700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc70076f70 0x7efc7019f780 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efc600057c0 tx=0x7efc60009580 comp rx=0 tx=0).stop 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.947+0000 7efc76800700 1 -- 172.21.15.45:0/2645576826 shutdown_connections 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.947+0000 7efc76800700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efc540a3db0 0x7efc540a6270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.947+0000 7efc76800700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efc70076f70 0x7efc7019f780 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.947+0000 7efc76800700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efc70113920 0x7efc701a4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.947+0000 7efc76800700 1 --2- 172.21.15.45:0/2645576826 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc70077890 0x7efc7019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.947+0000 7efc76800700 1 -- 172.21.15.45:0/2645576826 >> 172.21.15.45:0/2645576826 conn(0x7efc70100770 msgr2=0x7efc70102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.947+0000 7efc76800700 1 -- 172.21.15.45:0/2645576826 shutdown_connections 2024-04-25T19:02:12.950 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:12.947+0000 7efc76800700 1 -- 172.21.15.45:0/2645576826 wait complete. 2024-04-25T19:02:13.925 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-04-25T19:02:13.943 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.NFEDdMW3u5 2024-04-25T19:02:13.943 DEBUG:teuthology.orchestra.run.smithi079:> sudo cp /var/run/ceph/1/ops-695e6b8e-1-mds.i.json /tmp/tmp.NFEDdMW3u5 2024-04-25T19:02:14.012 INFO:teuthology.orchestra.run.smithi079.stderr:cp: cannot stat '/var/run/ceph/1/ops-695e6b8e-1-mds.i.json': No such file or directory 2024-04-25T19:02:14.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:02:14.014 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't pull ops dump at '/var/run/ceph/1/ops-695e6b8e-1-mds.i.json' on rank 2 (i), error: Command failed on smithi079 with status 1: 'sudo cp /var/run/ceph/1/ops-695e6b8e-1-mds.i.json /tmp/tmp.NFEDdMW3u5' 2024-04-25T19:02:14.014 DEBUG:teuthology.orchestra.run.smithi045:> mktemp 2024-04-25T19:02:14.075 INFO:teuthology.orchestra.run.smithi045.stdout:/tmp/tmp.MXnJ7bLq1b 2024-04-25T19:02:14.075 DEBUG:teuthology.orchestra.run.smithi045:> sudo cp /var/run/ceph/1/ops-695e6b8e-1-mds.a.json /tmp/tmp.MXnJ7bLq1b 2024-04-25T19:02:14.211 INFO:teuthology.orchestra.run.smithi045.stderr:cp: cannot stat '/var/run/ceph/1/ops-695e6b8e-1-mds.a.json': No such file or directory 2024-04-25T19:02:14.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:02:14.216 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't pull ops dump at '/var/run/ceph/1/ops-695e6b8e-1-mds.a.json' on rank 1 (a), error: Command failed on smithi045 with status 1: 'sudo cp /var/run/ceph/1/ops-695e6b8e-1-mds.a.json /tmp/tmp.MXnJ7bLq1b' 2024-04-25T19:02:14.216 DEBUG:teuthology.orchestra.run.smithi062:> mktemp 2024-04-25T19:02:14.231 INFO:teuthology.orchestra.run.smithi062.stdout:/tmp/tmp.1xwb6YwUqx 2024-04-25T19:02:14.232 DEBUG:teuthology.orchestra.run.smithi062:> sudo cp /var/run/ceph/1/ops-695e6b8e-1-mds.b.json /tmp/tmp.1xwb6YwUqx 2024-04-25T19:02:14.299 INFO:teuthology.orchestra.run.smithi062.stderr:cp: cannot stat '/var/run/ceph/1/ops-695e6b8e-1-mds.b.json': No such file or directory 2024-04-25T19:02:14.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:02:14.300 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't pull ops dump at '/var/run/ceph/1/ops-695e6b8e-1-mds.b.json' on rank 0 (b), error: Command failed on smithi062 with status 1: 'sudo cp /var/run/ceph/1/ops-695e6b8e-1-mds.b.json /tmp/tmp.1xwb6YwUqx' 2024-04-25T19:02:14.300 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:02:14.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:14 smithi045 ceph-mon[27776]: pgmap v90: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-25T19:02:14.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:14 smithi062 ceph-mon[32844]: pgmap v90: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-25T19:02:14.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:14 smithi079 ceph-mon[33002]: pgmap v90: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-04-25T19:02:14.758 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:16.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:16 smithi045 ceph-mon[27776]: pgmap v91: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 250 KiB/s wr, 77 op/s 2024-04-25T19:02:16.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:16 smithi062 ceph-mon[32844]: pgmap v91: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 250 KiB/s wr, 77 op/s 2024-04-25T19:02:16.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:16 smithi079 ceph-mon[33002]: pgmap v91: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 250 KiB/s wr, 77 op/s 2024-04-25T19:02:17.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.020+0000 7f31fc600700 1 -- 172.21.15.45:0/787330162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f41034b0 msgr2=0x7f31f410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.020+0000 7f31fc600700 1 --2- 172.21.15.45:0/787330162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f41034b0 0x7f31f410f920 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009fc0 tx=0x7f31ec00b330 comp rx=0 tx=0).stop 2024-04-25T19:02:17.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.020+0000 7f31fc600700 1 -- 172.21.15.45:0/787330162 shutdown_connections 2024-04-25T19:02:17.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.020+0000 7f31fc600700 1 --2- 172.21.15.45:0/787330162 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f41034b0 0x7f31f410f920 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.021+0000 7f31fc600700 1 --2- 172.21.15.45:0/787330162 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f4107940 0x7f31f4107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.023 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.021+0000 7f31fc600700 1 --2- 172.21.15.45:0/787330162 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f4102af0 0x7f31f4102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.021+0000 7f31fc600700 1 -- 172.21.15.45:0/787330162 >> 172.21.15.45:0/787330162 conn(0x7f31f4079ba0 msgr2=0x7f31f4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.021+0000 7f31fc600700 1 -- 172.21.15.45:0/787330162 shutdown_connections 2024-04-25T19:02:17.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.021+0000 7f31fc600700 1 -- 172.21.15.45:0/787330162 wait complete. 2024-04-25T19:02:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.022+0000 7f31fc600700 1 Processor -- start 2024-04-25T19:02:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.022+0000 7f31fc600700 1 -- start start 2024-04-25T19:02:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.022+0000 7f31fc600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f4102af0 0x7f31f419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.022+0000 7f31fc600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f41034b0 0x7f31f419b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.022+0000 7f31fc600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f4107940 0x7f31f419fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.022+0000 7f31fc600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4112ca0 con 0x7f31f4107940 2024-04-25T19:02:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.022+0000 7f31fc600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4112b20 con 0x7f31f4102af0 2024-04-25T19:02:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.022+0000 7f31fc600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f31f4112e20 con 0x7f31f41034b0 2024-04-25T19:02:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31f9c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f4102af0 0x7f31f419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.025 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31f9c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f4102af0 0x7f31f419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:37564/0 (socket says 172.21.15.45:37564) 2024-04-25T19:02:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31f9c00700 1 -- 172.21.15.45:0/2573213213 learned_addr learned my addr 172.21.15.45:0/2573213213 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31fa600700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f4107940 0x7f31f419fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.026 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31f9200700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f41034b0 0x7f31f419b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31f9c00700 1 -- 172.21.15.45:0/2573213213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f41034b0 msgr2=0x7f31f419b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31f9c00700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f41034b0 0x7f31f419b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31f9c00700 1 -- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f4107940 msgr2=0x7f31f419fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31f9c00700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f4107940 0x7f31f419fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31f9c00700 1 -- 172.21.15.45:0/2573213213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f41a0380 con 0x7f31f4102af0 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.023+0000 7f31fa600700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f4107940 0x7f31f419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.024+0000 7f31f9c00700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f4102af0 0x7f31f419b370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f31e800e9c0 tx=0x7f31e800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.024+0000 7f31f2000700 1 -- 172.21.15.45:0/2573213213 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8004a30 con 0x7f31f4102af0 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.024+0000 7f31f2000700 1 -- 172.21.15.45:0/2573213213 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31e800cc70 con 0x7f31f4102af0 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.024+0000 7f31fc600700 1 -- 172.21.15.45:0/2573213213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f41a0670 con 0x7f31f4102af0 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.024+0000 7f31f2000700 1 -- 172.21.15.45:0/2573213213 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e8005490 con 0x7f31f4102af0 2024-04-25T19:02:17.027 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.024+0000 7f31fc600700 1 -- 172.21.15.45:0/2573213213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f41a7ef0 con 0x7f31f4102af0 2024-04-25T19:02:17.028 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.025+0000 7f31fc600700 1 -- 172.21.15.45:0/2573213213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f4069730 con 0x7f31f4102af0 2024-04-25T19:02:17.029 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.027+0000 7f31f2000700 1 -- 172.21.15.45:0/2573213213 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f31e8004bd0 con 0x7f31f4102af0 2024-04-25T19:02:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.028+0000 7f31f2000700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f31dc0a3e30 0x7f31dc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.029+0000 7f31f2000700 1 -- 172.21.15.45:0/2573213213 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f31e80d0810 con 0x7f31f4102af0 2024-04-25T19:02:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.029+0000 7f31f9200700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f31dc0a3e30 0x7f31dc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:17.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.029+0000 7f31f9200700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f31dc0a3e30 0x7f31dc0a62f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f31f410fcb0 tx=0x7f31e0009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:17.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.033+0000 7f31f2000700 1 -- 172.21.15.45:0/2573213213 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31e809ce50 con 0x7f31f4102af0 2024-04-25T19:02:17.318 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.315+0000 7f31fc600700 1 -- 172.21.15.45:0/2573213213 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31f41a8120 con 0x7f31f4102af0 2024-04-25T19:02:17.319 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.316+0000 7f31f2000700 1 -- 172.21.15.45:0/2573213213 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11855 (secure 0 0 0) 0x7f31e809c750 con 0x7f31f4102af0 2024-04-25T19:02:17.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:02:17.320 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T19:01:10.424364+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24493":{"gid":24493,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/3290580293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":3290580293},{"type":"v1","addr":"172.21.15.79:6837","nonce":3290580293}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24512,24505]},"id":1}]} 2024-04-25T19:02:17.322 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.320+0000 7f31fc600700 1 -- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f31dc0a3e30 msgr2=0x7f31dc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.320+0000 7f31fc600700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f31dc0a3e30 0x7f31dc0a62f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f31f410fcb0 tx=0x7f31e0009260 comp rx=0 tx=0).stop 2024-04-25T19:02:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.320+0000 7f31fc600700 1 -- 172.21.15.45:0/2573213213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f4102af0 msgr2=0x7f31f419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.320+0000 7f31fc600700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f4102af0 0x7f31f419b370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f31e800e9c0 tx=0x7f31e800ecd0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.321+0000 7f31fc600700 1 -- 172.21.15.45:0/2573213213 shutdown_connections 2024-04-25T19:02:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.321+0000 7f31fc600700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f31dc0a3e30 0x7f31dc0a62f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.323 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.321+0000 7f31fc600700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f31f4102af0 0x7f31f419b370 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.321+0000 7f31fc600700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31f4107940 0x7f31f419fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.321+0000 7f31fc600700 1 --2- 172.21.15.45:0/2573213213 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f31f41034b0 0x7f31f419b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.321+0000 7f31fc600700 1 -- 172.21.15.45:0/2573213213 >> 172.21.15.45:0/2573213213 conn(0x7f31f4079ba0 msgr2=0x7f31f4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.321+0000 7f31fc600700 1 -- 172.21.15.45:0/2573213213 shutdown_connections 2024-04-25T19:02:17.324 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:17.321+0000 7f31fc600700 1 -- 172.21.15.45:0/2573213213 wait complete. 2024-04-25T19:02:17.325 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 28 2024-04-25T19:02:18.247 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --set-id 695e6b8e --await-for 42.0' 2024-04-25T19:02:18.247 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24524 quiesce db --set-id 695e6b8e --await-for 42.0 2024-04-25T19:02:18.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:18 smithi045 ceph-mon[27776]: pgmap v92: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 226 KiB/s wr, 59 op/s 2024-04-25T19:02:18.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:18 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2573213213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:02:18.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:18 smithi062 ceph-mon[32844]: pgmap v92: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 226 KiB/s wr, 59 op/s 2024-04-25T19:02:18.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:18 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2573213213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:02:18.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:18 smithi079 ceph-mon[33002]: pgmap v92: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 226 KiB/s wr, 59 op/s 2024-04-25T19:02:18.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:18 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2573213213' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:02:18.726 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:02:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:20 smithi045 ceph-mon[27776]: pgmap v93: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 226 KiB/s wr, 59 op/s 2024-04-25T19:02:20.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:20 smithi062 ceph-mon[32844]: pgmap v93: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 226 KiB/s wr, 59 op/s 2024-04-25T19:02:20.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:20 smithi079 ceph-mon[33002]: pgmap v93: 97 pgs: 97 active+clean; 25 MiB data, 518 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 226 KiB/s wr, 59 op/s 2024-04-25T19:02:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.028+0000 7f2d27c00700 1 -- 172.21.15.45:0/2574098310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d20107940 msgr2=0x7f2d20107d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.028+0000 7f2d27c00700 1 --2- 172.21.15.45:0/2574098310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d20107940 0x7f2d20107d20 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2d140099c0 tx=0x7f2d14009cd0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.028+0000 7f2d27c00700 1 -- 172.21.15.45:0/2574098310 shutdown_connections 2024-04-25T19:02:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.028+0000 7f2d27c00700 1 --2- 172.21.15.45:0/2574098310 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d201034b0 0x7f2d2010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.028+0000 7f2d27c00700 1 --2- 172.21.15.45:0/2574098310 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d20107940 0x7f2d20107d20 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.028+0000 7f2d27c00700 1 --2- 172.21.15.45:0/2574098310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d20102af0 0x7f2d20102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.028+0000 7f2d27c00700 1 -- 172.21.15.45:0/2574098310 >> 172.21.15.45:0/2574098310 conn(0x7f2d20079ba0 msgr2=0x7f2d20079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.028+0000 7f2d27c00700 1 -- 172.21.15.45:0/2574098310 shutdown_connections 2024-04-25T19:02:21.031 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.028+0000 7f2d27c00700 1 -- 172.21.15.45:0/2574098310 wait complete. 2024-04-25T19:02:21.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.030+0000 7f2d27c00700 1 Processor -- start 2024-04-25T19:02:21.032 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.031+0000 7f2d27c00700 1 -- start start 2024-04-25T19:02:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.031+0000 7f2d27c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d20102af0 0x7f2d2019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.034 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.031+0000 7f2d27c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d201034b0 0x7f2d2019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.031+0000 7f2d27c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d20107940 0x7f2d2019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.031+0000 7f2d27c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20112c90 con 0x7f2d20102af0 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.031+0000 7f2d27c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20112b10 con 0x7f2d201034b0 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.031+0000 7f2d27c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2d20112e10 con 0x7f2d20107940 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d25200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d20102af0 0x7f2d2019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d25200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d20102af0 0x7f2d2019b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39794/0 (socket says 172.21.15.45:39794) 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d25200700 1 -- 172.21.15.45:0/1336646565 learned_addr learned my addr 172.21.15.45:0/1336646565 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/1336646565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d201034b0 0x7f2d2019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d25c00700 1 --2- 172.21.15.45:0/1336646565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d20107940 0x7f2d2019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d1fe00700 1 -- 172.21.15.45:0/1336646565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d20107940 msgr2=0x7f2d2019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/1336646565 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2d20107940 0x7f2d2019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.035 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d1fe00700 1 -- 172.21.15.45:0/1336646565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d20102af0 msgr2=0x7f2d2019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:21.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/1336646565 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2d20102af0 0x7f2d2019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d1fe00700 1 -- 172.21.15.45:0/1336646565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d201a0450 con 0x7f2d201034b0 2024-04-25T19:02:21.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.032+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/1336646565 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2d201034b0 0x7f2d2019b980 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00e9f0 tx=0x7f2d0c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:21.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.033+0000 7f2d1d600700 1 -- 172.21.15.45:0/1336646565 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00c9c0 con 0x7f2d201034b0 2024-04-25T19:02:21.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.033+0000 7f2d1d600700 1 -- 172.21.15.45:0/1336646565 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c00cb60 con 0x7f2d201034b0 2024-04-25T19:02:21.036 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.033+0000 7f2d1d600700 1 -- 172.21.15.45:0/1336646565 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c018680 con 0x7f2d201034b0 2024-04-25T19:02:21.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.033+0000 7f2d27c00700 1 -- 172.21.15.45:0/1336646565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d201a0740 con 0x7f2d201034b0 2024-04-25T19:02:21.037 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.033+0000 7f2d27c00700 1 -- 172.21.15.45:0/1336646565 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d201a7fc0 con 0x7f2d201034b0 2024-04-25T19:02:21.038 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.036+0000 7f2d1d600700 1 -- 172.21.15.45:0/1336646565 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2d0c010430 con 0x7f2d201034b0 2024-04-25T19:02:21.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.037+0000 7f2d1d600700 1 --2- 172.21.15.45:0/1336646565 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2d080a3e30 0x7f2d080a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.038+0000 7f2d1d600700 1 -- 172.21.15.45:0/1336646565 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2d0c0d0950 con 0x7f2d201034b0 2024-04-25T19:02:21.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.038+0000 7f2d324d6b80 1 Processor -- start 2024-04-25T19:02:21.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.038+0000 7f2d25200700 1 --2- 172.21.15.45:0/1336646565 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2d080a3e30 0x7f2d080a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.040 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.038+0000 7f2d324d6b80 1 -- start start 2024-04-25T19:02:21.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.038+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 0x5641127ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.041 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.038+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.038+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127b98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5641126fef90 con 0x5641127ac660 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5641126a5830 con 0x5641127abd40 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5641126faf50 con 0x5641127b74d0 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d1fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d25200700 1 --2- 172.21.15.45:0/1336646565 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2d080a3e30 0x7f2d080a62f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d14003760 tx=0x7f2d14005fd0 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d25200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 0x5641127ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d1fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39798/0 (socket says 172.21.15.45:39798) 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d25200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 0x5641127ac120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:37586/0 (socket says 172.21.15.45:37586) 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d1fe00700 1 -- 172.21.15.45:0/1985305146 learned_addr learned my addr 172.21.15.45:0/1985305146 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:21.042 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d25c00700 1 --2- 172.21.15.45:0/1985305146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127b98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d25200700 1 -- 172.21.15.45:0/1985305146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 msgr2=0x5641127b98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:21.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d25200700 1 --2- 172.21.15.45:0/1985305146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127b98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d25200700 1 -- 172.21.15.45:0/1985305146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 msgr2=0x5641127b6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:21.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d25200700 1 --2- 172.21.15.45:0/1985305146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.043 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.039+0000 7f2d25200700 1 -- 172.21.15.45:0/1985305146 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56411269b780 con 0x5641127abd40 2024-04-25T19:02:21.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.040+0000 7f2d25200700 1 --2- 172.21.15.45:0/1985305146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 0x5641127ac120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2d14000c00 tx=0x7f2d1401a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40db9cea20c862c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:21.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.040+0000 7f2d12000700 1 -- 172.21.15.45:0/1985305146 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14004c30 con 0x5641127abd40 2024-04-25T19:02:21.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.040+0000 7f2d12000700 1 -- 172.21.15.45:0/1985305146 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14004dd0 con 0x5641127abd40 2024-04-25T19:02:21.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.040+0000 7f2d12000700 1 -- 172.21.15.45:0/1985305146 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140239e0 con 0x5641127abd40 2024-04-25T19:02:21.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.041+0000 7f2d324d6b80 1 -- 172.21.15.45:0/1985305146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 msgr2=0x5641127ac120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:21.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.041+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/1985305146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 0x5641127ac120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2d14000c00 tx=0x7f2d1401a070 comp rx=0 tx=0).stop 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.041+0000 7f2d324d6b80 1 -- 172.21.15.45:0/1985305146 shutdown_connections 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.041+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/1985305146 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 0x5641127ac120 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.041+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/1985305146 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127b98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.041+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/1985305146 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.041+0000 7f2d324d6b80 1 -- 172.21.15.45:0/1985305146 >> 172.21.15.45:0/1985305146 conn(0x5641126db050 msgr2=0x5641127a8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.041+0000 7f2d324d6b80 1 -- 172.21.15.45:0/1985305146 shutdown_connections 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.041+0000 7f2d324d6b80 1 -- 172.21.15.45:0/1985305146 wait complete. 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.042+0000 7f2d324d6b80 1 -- start start 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.042+0000 7f2d324d6b80 1 Processor -- start 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 0x5641127b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5641127b9ea0 con 0x5641127ac660 2024-04-25T19:02:21.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5641126b8960 con 0x5641127abd40 2024-04-25T19:02:21.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5641127ba020 con 0x5641127b74d0 2024-04-25T19:02:21.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d324d6b80 10 client.0 _ll_register_callbacks cb 0x564112697820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:21.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d1fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d1fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39808/0 (socket says 172.21.15.45:39808) 2024-04-25T19:02:21.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d1fe00700 1 -- 172.21.15.45:0/498155037 learned_addr learned my addr 172.21.15.45:0/498155037 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:21.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d25200700 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 0x5641127b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d25c00700 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d1fe00700 1 -- 172.21.15.45:0/498155037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 msgr2=0x5641127b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d1fe00700 1 -- 172.21.15.45:0/498155037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 msgr2=0x5641127b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 0x5641127b6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.043+0000 7f2d1fe00700 1 -- 172.21.15.45:0/498155037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56411264c9d0 con 0x5641127ac660 2024-04-25T19:02:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.044+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b3f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c098ca0 tx=0x7f2d0c09c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7b07fb7aa86e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.044+0000 7f2d06a00700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:02:21.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.044+0000 7f2d06a00700 1 -- 172.21.15.45:0/498155037 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c09ccf0 con 0x5641127ac660 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.044+0000 7f2d06a00700 1 -- 172.21.15.45:0/498155037 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2d0c0a5070 con 0x5641127ac660 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.044+0000 7f2d324d6b80 1 -- 172.21.15.45:0/498155037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d14004dd0 con 0x5641127ac660 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.044+0000 7f2d06a00700 1 -- 172.21.15.45:0/498155037 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c0b9020 con 0x5641127ac660 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.044+0000 7f2d324d6b80 10 client.15420 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.044+0000 7f2d324d6b80 1 -- 172.21.15.45:0/498155037 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564112695bf0 con 0x5641127ac660 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.045+0000 7f2d06a00700 1 -- 172.21.15.45:0/498155037 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14004dd0 con 0x5641127ac660 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.045+0000 7f2d06a00700 1 -- 172.21.15.45:0/498155037 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f2d0c09dbc0 con 0x5641127ac660 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.045+0000 7f2d324d6b80 10 client.15420 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.045+0000 7f2d324d6b80 10 client.15420 resolve_mds: validated gid 24524 aka daemon mds.b 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.045+0000 7f2d324d6b80 20 client.15420 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:21.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.045+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5641127c6bb0 0x5641127c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.045+0000 7f2d324d6b80 4 client.15420 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.045+0000 7f2d324d6b80 1 -- 172.21.15.45:0/498155037 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56411270c780 con 0x5641127c6bb0 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.045+0000 7f2d25200700 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5641127c6bb0 0x5641127c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.046+0000 7f2d25200700 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5641127c6bb0 0x5641127c8fa0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aef13d8e044877a server_cookie=782d228055b57e29 in_seq=0 out_seq=0 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.046+0000 7f2d06a00700 10 client.15420 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d06a00700 1 -- 172.21.15.45:0/498155037 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56411270c780 con 0x5641127c6bb0 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d06a00700 10 client.15420 handle_command_reply: tid=0 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 client.15420 shutdown 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 -- 172.21.15.45:0/498155037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 msgr2=0x5641127b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b3f20 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c098ca0 tx=0x7f2d0c09c2e0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 -- 172.21.15.45:0/498155037 shutdown_connections 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5641127c6bb0 0x5641127c8fa0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127abd40 0x5641127b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/498155037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127ac660 0x5641127b3f20 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 -- 172.21.15.45:0/498155037 >> 172.21.15.45:0/498155037 conn(0x5641126db050 msgr2=0x5641127a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d06a00700 0 client.15420 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 -- 172.21.15.45:0/498155037 shutdown_connections 2024-04-25T19:02:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.047+0000 7f2d324d6b80 1 -- 172.21.15.45:0/498155037 wait complete. 2024-04-25T19:02:21.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.048+0000 7f2d324d6b80 20 client.15420 trim_cache size 0 max 16384 2024-04-25T19:02:21.071 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d324d6b80 1 Processor -- start 2024-04-25T19:02:21.072 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d324d6b80 1 -- start start 2024-04-25T19:02:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 0x5641127b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201b1f00 con 0x5641127abd40 2024-04-25T19:02:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201af790 con 0x5641127ac660 2024-04-25T19:02:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201ad5e0 con 0x5641127b74d0 2024-04-25T19:02:21.073 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d25200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 0x5641127b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d25200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 0x5641127b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39818/0 (socket says 172.21.15.45:39818) 2024-04-25T19:02:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.070+0000 7f2d25200700 1 -- 172.21.15.45:0/1555321592 learned_addr learned my addr 172.21.15.45:0/1555321592 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.071+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/1555321592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.071+0000 7f2d25c00700 1 --2- 172.21.15.45:0/1555321592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.071+0000 7f2d25200700 1 -- 172.21.15.45:0/1555321592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 msgr2=0x5641127bf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.071+0000 7f2d25200700 1 --2- 172.21.15.45:0/1555321592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127bf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.071+0000 7f2d25200700 1 -- 172.21.15.45:0/1555321592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 msgr2=0x5641127c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.071+0000 7f2d25200700 1 --2- 172.21.15.45:0/1555321592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.071+0000 7f2d25200700 1 -- 172.21.15.45:0/1555321592 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641127acce0 con 0x5641127abd40 2024-04-25T19:02:21.074 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.071+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/1555321592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:21.075 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.071+0000 7f2d25200700 1 --2- 172.21.15.45:0/1555321592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 0x5641127b2ec0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d14000c00 tx=0x7f2d14019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb29a903abc75a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.072+0000 7f2d11600700 1 -- 172.21.15.45:0/1555321592 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14004db0 con 0x5641127abd40 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.072+0000 7f2d11600700 1 -- 172.21.15.45:0/1555321592 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14040020 con 0x5641127abd40 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.072+0000 7f2d11600700 1 -- 172.21.15.45:0/1555321592 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1403c4a0 con 0x5641127abd40 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.072+0000 7f2d324d6b80 1 -- 172.21.15.45:0/1555321592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 msgr2=0x5641127b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.072+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/1555321592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 0x5641127b2ec0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2d14000c00 tx=0x7f2d14019040 comp rx=0 tx=0).stop 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.072+0000 7f2d324d6b80 1 -- 172.21.15.45:0/1555321592 shutdown_connections 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.072+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/1555321592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x5641127bf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.072+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/1555321592 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 0x5641127b2ec0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.072+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/1555321592 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.073+0000 7f2d324d6b80 1 -- 172.21.15.45:0/1555321592 >> 172.21.15.45:0/1555321592 conn(0x5641126db050 msgr2=0x5641127a7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.073+0000 7f2d324d6b80 1 -- 172.21.15.45:0/1555321592 shutdown_connections 2024-04-25T19:02:21.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.073+0000 7f2d324d6b80 1 -- 172.21.15.45:0/1555321592 wait complete. 2024-04-25T19:02:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.073+0000 7f2d324d6b80 1 -- start start 2024-04-25T19:02:21.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.073+0000 7f2d324d6b80 1 Processor -- start 2024-04-25T19:02:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 0x5641127be400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d324d6b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x56411282d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201adc50 con 0x5641127abd40 2024-04-25T19:02:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201af0c0 con 0x5641127ac660 2024-04-25T19:02:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d324d6b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2d201ae320 con 0x5641127b74d0 2024-04-25T19:02:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d324d6b80 10 client.0 _ll_register_callbacks cb 0x564112697820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:21.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d25200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 0x5641127be400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d1fe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d1fe00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127be940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:37622/0 (socket says 172.21.15.45:37622) 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d1fe00700 1 -- 172.21.15.45:0/2424168386 learned_addr learned my addr 172.21.15.45:0/2424168386 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.074+0000 7f2d25c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x56411282d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d1fe00700 1 -- 172.21.15.45:0/2424168386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 msgr2=0x56411282d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x56411282d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d1fe00700 1 -- 172.21.15.45:0/2424168386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 msgr2=0x5641127be400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 0x5641127be400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d1fe00700 1 -- 172.21.15.45:0/2424168386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56411282d6b0 con 0x5641127ac660 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d1fe00700 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127be940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c087010 tx=0x7f2d0c09c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79b1c3d0249810d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d06a00700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:02:21.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d06a00700 1 -- 172.21.15.45:0/2424168386 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c09ddc0 con 0x5641127ac660 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d324d6b80 1 -- 172.21.15.45:0/2424168386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d14040020 con 0x5641127ac660 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d06a00700 1 -- 172.21.15.45:0/2424168386 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2d0c09caf0 con 0x5641127ac660 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d06a00700 1 -- 172.21.15.45:0/2424168386 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d0c0a0c40 con 0x5641127ac660 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d324d6b80 10 client.25553 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.075+0000 7f2d324d6b80 1 -- 172.21.15.45:0/2424168386 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56411282e9e0 con 0x5641127ac660 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.076+0000 7f2d06a00700 1 -- 172.21.15.45:0/2424168386 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14040020 con 0x5641127ac660 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.076+0000 7f2d06a00700 1 -- 172.21.15.45:0/2424168386 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f2d0c018c40 con 0x5641127ac660 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.076+0000 7f2d324d6b80 10 client.25553 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.076+0000 7f2d324d6b80 10 client.25553 resolve_mds: validated gid 24524 aka daemon mds.b 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.076+0000 7f2d324d6b80 20 client.25553 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.076+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56411282efe0 0x564112831400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:21.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.076+0000 7f2d324d6b80 4 client.25553 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "695e6b8e", "await_for": 42.0}] 2024-04-25T19:02:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.076+0000 7f2d324d6b80 1 -- 172.21.15.45:0/2424168386 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "quiesce db", "set_id": "695e6b8e", "await_for": 42.0}) -- 0x564112706820 con 0x56411282efe0 2024-04-25T19:02:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.077+0000 7f2d25200700 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56411282efe0 0x564112831400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.077+0000 7f2d25200700 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56411282efe0 0x564112831400 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f6d01b23d7acb7 server_cookie=b1b0d368594e592a in_seq=0 out_seq=0 2024-04-25T19:02:21.082 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.077+0000 7f2d06a00700 10 client.25553 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.077+0000 7f2d06a00700 1 -- 172.21.15.45:0/2424168386 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: -110 ) ==== 8+0+590 (crc 0 0 0) 0x564112706820 con 0x56411282efe0 2024-04-25T19:02:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.077+0000 7f2d06a00700 10 client.25553 handle_command_reply: tid=0 2024-04-25T19:02:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.077+0000 7f2d324d6b80 1 client.25553 shutdown 2024-04-25T19:02:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 1 -- 172.21.15.45:0/2424168386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 msgr2=0x5641127be940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127be940 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c087010 tx=0x7f2d0c09c950 comp rx=0 tx=0).stop 2024-04-25T19:02:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 1 -- 172.21.15.45:0/2424168386 shutdown_connections 2024-04-25T19:02:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56411282efe0 0x564112831400 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5641127b74d0 0x56411282d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5641127abd40 0x5641127be400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 1 --2- 172.21.15.45:0/2424168386 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5641127ac660 0x5641127be940 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d06a00700 0 client.25553 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 1 -- 172.21.15.45:0/2424168386 >> 172.21.15.45:0/2424168386 conn(0x5641126db050 msgr2=0x5641127c1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 1 -- 172.21.15.45:0/2424168386 shutdown_connections 2024-04-25T19:02:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 1 -- 172.21.15.45:0/2424168386 wait complete. 2024-04-25T19:02:21.084 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:02:21.078+0000 7f2d324d6b80 20 client.25553 trim_cache size 0 max 16384 2024-04-25T19:02:21.084 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 28, 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24524, 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 6, 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": { 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "695e6b8e": { 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "version": 6, 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "age_ref": 8.5, 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "TIMEDOUT", 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 0.0 2024-04-25T19:02:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "timeout": 60.0, 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "expiration": 89.4, 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "members": { 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "file:/": { 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "excluded": false, 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "state": { 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "QUIESCING", 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "age": 60.0 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:02:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:02:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:02:21.087 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:02:21.087 INFO:teuthology.orchestra.run.smithi045.stderr:Error ETIMEDOUT: 2024-04-25T19:02:22.034 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-04-25T19:02:22.035 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't quiesce root with rc: 110 (ETIMEDOUT), stdout: { "epoch": 28, "leader": 24524, "set_version": 6, "sets": { "695e6b8e": { "version": 6, "age_ref": 8.5, "state": { "name": "TIMEDOUT", "age": 0.0 }, "timeout": 60.0, "expiration": 89.4, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCING", "age": 60.0 } } } } } } 2024-04-25T19:02:22.035 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks/quiescer.py", line 346, in _run self.do_quiesce(d) File "/home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks/quiescer.py", line 301, in do_quiesce raise RuntimeError(f"Error quiescing set '{set_id}': {rcinfo}") RuntimeError: Error quiescing set '695e6b8e': 110 (ETIMEDOUT) 2024-04-25T19:02:22.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:22 smithi062 ceph-mon[32844]: pgmap v94: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 436 KiB/s wr, 75 op/s 2024-04-25T19:02:22.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:22 smithi079 ceph-mon[33002]: pgmap v94: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 436 KiB/s wr, 75 op/s 2024-04-25T19:02:22.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:22 smithi045 ceph-mon[27776]: pgmap v94: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 349 KiB/s rd, 436 KiB/s wr, 75 op/s 2024-04-25T19:02:24.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:24 smithi062 ceph-mon[32844]: pgmap v95: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 436 KiB/s wr, 72 op/s 2024-04-25T19:02:24.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:24 smithi079 ceph-mon[33002]: pgmap v95: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 436 KiB/s wr, 72 op/s 2024-04-25T19:02:24.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:24 smithi045 ceph-mon[27776]: pgmap v95: 97 pgs: 97 active+clean; 27 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 436 KiB/s wr, 72 op/s 2024-04-25T19:02:25.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:25 smithi062 ceph-mon[32844]: pgmap v96: 97 pgs: 97 active+clean; 34 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-04-25T19:02:25.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:25 smithi079 ceph-mon[33002]: pgmap v96: 97 pgs: 97 active+clean; 34 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-04-25T19:02:25.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:25 smithi045 ceph-mon[27776]: pgmap v96: 97 pgs: 97 active+clean; 34 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 781 KiB/s rd, 1.2 MiB/s wr, 220 op/s 2024-04-25T19:02:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.052+0000 7fe8cde00640 1 -- 172.21.15.45:0/1874889598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c8103850 msgr2=0x7fe8c810bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.052+0000 7fe8cde00640 1 --2- 172.21.15.45:0/1874889598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c8103850 0x7fe8c810bd30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4009900 tx=0x7fe8b402a410 comp rx=0 tx=0).stop 2024-04-25T19:02:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.052+0000 7fe8cde00640 1 -- 172.21.15.45:0/1874889598 shutdown_connections 2024-04-25T19:02:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.052+0000 7fe8cde00640 1 --2- 172.21.15.45:0/1874889598 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c810c3a0 0x7fe8c810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.052+0000 7fe8cde00640 1 --2- 172.21.15.45:0/1874889598 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c8103850 0x7fe8c810bd30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.052+0000 7fe8cde00640 1 --2- 172.21.15.45:0/1874889598 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8c8102f30 0x7fe8c8103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.052+0000 7fe8cde00640 1 -- 172.21.15.45:0/1874889598 >> 172.21.15.45:0/1874889598 conn(0x7fe8c80fc790 msgr2=0x7fe8c80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.053+0000 7fe8cde00640 1 -- 172.21.15.45:0/1874889598 shutdown_connections 2024-04-25T19:02:26.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.053+0000 7fe8cde00640 1 -- 172.21.15.45:0/1874889598 wait complete. 2024-04-25T19:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.054+0000 7fe8cde00640 1 Processor -- start 2024-04-25T19:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.054+0000 7fe8cde00640 1 -- start start 2024-04-25T19:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.054+0000 7fe8cde00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8102f30 0x7fe8c81a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.054+0000 7fe8cde00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c8103850 0x7fe8c81a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.054+0000 7fe8cde00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8c810c3a0 0x7fe8c819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.054+0000 7fe8cde00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8111ff0 con 0x7fe8c8102f30 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.054+0000 7fe8cde00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8111e70 con 0x7fe8c8103850 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.054+0000 7fe8cde00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c8112170 con 0x7fe8c810c3a0 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.054+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c8103850 0x7fe8c81a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c8103850 0x7fe8c81a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:52964/0 (socket says 172.21.15.45:52964) 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8c7e00640 1 -- 172.21.15.45:0/3098446309 learned_addr learned my addr 172.21.15.45:0/3098446309 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8cca00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8102f30 0x7fe8c81a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8cd400640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8c810c3a0 0x7fe8c819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8c7e00640 1 -- 172.21.15.45:0/3098446309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8c810c3a0 msgr2=0x7fe8c819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8c7e00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8c810c3a0 0x7fe8c819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8c7e00640 1 -- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8102f30 msgr2=0x7fe8c81a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8c7e00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8102f30 0x7fe8c81a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8c7e00640 1 -- 172.21.15.45:0/3098446309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c819ad50 con 0x7fe8c8103850 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8cca00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8102f30 0x7fe8c81a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:26.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.055+0000 7fe8c7e00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c8103850 0x7fe8c81a0960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4032fd0 tx=0x7fe8b4031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.056+0000 7fe8c5600640 1 -- 172.21.15.45:0/3098446309 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b403f030 con 0x7fe8c8103850 2024-04-25T19:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.056+0000 7fe8c5600640 1 -- 172.21.15.45:0/3098446309 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b4004490 con 0x7fe8c8103850 2024-04-25T19:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.056+0000 7fe8c5600640 1 -- 172.21.15.45:0/3098446309 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4035070 con 0x7fe8c8103850 2024-04-25T19:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.056+0000 7fe8cde00640 1 -- 172.21.15.45:0/3098446309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c819afe0 con 0x7fe8c8103850 2024-04-25T19:02:26.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.056+0000 7fe8cde00640 1 -- 172.21.15.45:0/3098446309 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c81a72e0 con 0x7fe8c8103850 2024-04-25T19:02:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.059+0000 7fe8c5600640 1 -- 172.21.15.45:0/3098446309 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe8b4042030 con 0x7fe8c8103850 2024-04-25T19:02:26.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.059+0000 7fe8d2243740 1 Processor -- start 2024-04-25T19:02:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.060+0000 7fe8d2243740 1 -- start start 2024-04-25T19:02:26.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.060+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f683f1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.060+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f2210 0x557f68404e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.060+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68407790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.060+0000 7fe8d2243740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557f68357900 con 0x557f683f18f0 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.060+0000 7fe8d2243740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557f682fdef0 con 0x557f684053a0 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.060+0000 7fe8d2243740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557f683538c0 con 0x557f683f2210 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.060+0000 7fe8cd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f683f1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.060+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68407790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.060+0000 7fe8cca00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f2210 0x557f68404e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8cd400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f683f1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51870/0 (socket says 172.21.15.45:51870) 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68407790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:52972/0 (socket says 172.21.15.45:52972) 2024-04-25T19:02:26.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8cd400640 1 -- 172.21.15.45:0/1360737085 learned_addr learned my addr 172.21.15.45:0/1360737085 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8c5600640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe8b00a3d80 0x7fe8b00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8cd400640 1 -- 172.21.15.45:0/1360737085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f2210 msgr2=0x557f68404e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8cd400640 1 --2- 172.21.15.45:0/1360737085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f2210 0x557f68404e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8cd400640 1 -- 172.21.15.45:0/1360737085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 msgr2=0x557f68407790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8cd400640 1 --2- 172.21.15.45:0/1360737085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68407790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8cd400640 1 -- 172.21.15.45:0/1360737085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f6806a210 con 0x557f683f18f0 2024-04-25T19:02:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8c7e00640 1 --2- 172.21.15.45:0/1360737085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68407790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8c5600640 1 -- 172.21.15.45:0/3098446309 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fe8b40f0b90 con 0x7fe8c8103850 2024-04-25T19:02:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.061+0000 7fe8cd400640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe8b00a3d80 0x7fe8b00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.062+0000 7fe8cd400640 1 --2- 172.21.15.45:0/1360737085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f683f1cd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009f50 tx=0x7fe8c00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99b551ca3773cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.062+0000 7fe8b9c00640 1 -- 172.21.15.45:0/1360737085 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0004250 con 0x557f683f18f0 2024-04-25T19:02:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.062+0000 7fe8b9c00640 1 -- 172.21.15.45:0/1360737085 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8c000fe00 con 0x557f683f18f0 2024-04-25T19:02:26.064 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.062+0000 7fe8b9c00640 1 -- 172.21.15.45:0/1360737085 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c001b9d0 con 0x557f683f18f0 2024-04-25T19:02:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.063+0000 7fe8d2243740 1 -- 172.21.15.45:0/1360737085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 msgr2=0x557f683f1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.063+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1360737085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f683f1cd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0009f50 tx=0x7fe8c00118d0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.063+0000 7fe8cd400640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe8b00a3d80 0x7fe8b00a6240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0012fd0 tx=0x7fe8c0021000 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.063+0000 7fe8d2243740 1 -- 172.21.15.45:0/1360737085 shutdown_connections 2024-04-25T19:02:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.063+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1360737085 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68407790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.063+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1360737085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f2210 0x557f68404e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.063+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1360737085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f683f1cd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.063+0000 7fe8d2243740 1 -- 172.21.15.45:0/1360737085 >> 172.21.15.45:0/1360737085 conn(0x557f683edec0 msgr2=0x557f683ee2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.063+0000 7fe8d2243740 1 -- 172.21.15.45:0/1360737085 shutdown_connections 2024-04-25T19:02:26.067 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.063+0000 7fe8d2243740 1 -- 172.21.15.45:0/1360737085 wait complete. 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.064+0000 7fe8d2243740 1 -- start start 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.064+0000 7fe8d2243740 1 Processor -- start 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.064+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f18f0 0x557f684022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f2210 0x557f684027e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68402d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8d2243740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557f68407f70 con 0x557f683f2210 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8d2243740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557f68311310 con 0x557f684053a0 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8d2243740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557f684080f0 con 0x557f683f18f0 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8d2243740 10 client.0 _ll_register_callbacks cb 0x557f682dfcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8cd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68402d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f2210 0x557f684027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8cd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68402d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:52988/0 (socket says 172.21.15.45:52988) 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8cd400640 1 -- 172.21.15.45:0/4199792562 learned_addr learned my addr 172.21.15.45:0/4199792562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:26.068 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8cca00640 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f18f0 0x557f684022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8cd400640 1 -- 172.21.15.45:0/4199792562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f18f0 msgr2=0x557f684022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8cd400640 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f18f0 0x557f684022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8cd400640 1 -- 172.21.15.45:0/4199792562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f2210 msgr2=0x557f684027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8cd400640 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f2210 0x557f684027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8cd400640 1 -- 172.21.15.45:0/4199792562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f68136620 con 0x557f684053a0 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.065+0000 7fe8c7e00640 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f2210 0x557f684027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.066+0000 7fe8cd400640 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68402d20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe8c001b690 tx=0x7fe8c0003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66eb3dce20a1f95c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.066+0000 7fe8ae000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.066+0000 7fe8ae000640 1 -- 172.21.15.45:0/4199792562 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0033030 con 0x557f684053a0 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.066+0000 7fe8d2243740 1 -- 172.21.15.45:0/4199792562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c000fe00 con 0x557f684053a0 2024-04-25T19:02:26.069 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.066+0000 7fe8ae000640 1 -- 172.21.15.45:0/4199792562 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fe8c0004260 con 0x557f684053a0 2024-04-25T19:02:26.071 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.066+0000 7fe8ae000640 1 -- 172.21.15.45:0/4199792562 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8c003d050 con 0x557f684053a0 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.066+0000 7fe8d2243740 10 client.25565 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.066+0000 7fe8d2243740 1 -- 172.21.15.45:0/4199792562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f684141b0 con 0x557f684053a0 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.066+0000 7fe8ae000640 1 -- 172.21.15.45:0/4199792562 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8c000fe00 con 0x557f684053a0 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.067+0000 7fe8ae000640 1 -- 172.21.15.45:0/4199792562 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fe8c001bce0 con 0x557f684053a0 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.067+0000 7fe8d2243740 10 client.25565 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.067+0000 7fe8d2243740 10 client.25565 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.067+0000 7fe8d2243740 20 client.25565 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.067+0000 7fe8d2243740 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557f684155c0 0x557f684179b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.067+0000 7fe8d2243740 4 client.25565 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.067+0000 7fe8d2243740 1 -- 172.21.15.45:0/4199792562 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f6836b8b0 con 0x557f684155c0 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.067+0000 7fe8cca00640 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557f684155c0 0x557f684179b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.068+0000 7fe8cca00640 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557f684155c0 0x557f684179b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8151491379e628ae server_cookie=4b5d615911d002a1 in_seq=0 out_seq=0 2024-04-25T19:02:26.072 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.068+0000 7fe8ae000640 10 client.25565 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8ae000640 1 -- 172.21.15.45:0/4199792562 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557f6836b8b0 con 0x557f684155c0 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8ae000640 10 client.25565 handle_command_reply: tid=0 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 client.25565 shutdown 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 -- 172.21.15.45:0/4199792562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 msgr2=0x557f68402d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68402d20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe8c001b690 tx=0x7fe8c0003f20 comp rx=0 tx=0).stop 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 -- 172.21.15.45:0/4199792562 shutdown_connections 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557f684155c0 0x557f684179b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f68402d20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f2210 0x557f684027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 --2- 172.21.15.45:0/4199792562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f18f0 0x557f684022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8ae000640 0 client.25565 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 -- 172.21.15.45:0/4199792562 >> 172.21.15.45:0/4199792562 conn(0x557f683edec0 msgr2=0x557f683f3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 -- 172.21.15.45:0/4199792562 shutdown_connections 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 1 -- 172.21.15.45:0/4199792562 wait complete. 2024-04-25T19:02:26.073 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.069+0000 7fe8d2243740 20 client.25565 trim_cache size 0 max 16384 2024-04-25T19:02:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.087+0000 7fe8d2243740 1 Processor -- start 2024-04-25T19:02:26.088 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.087+0000 7fe8d2243740 1 -- start start 2024-04-25T19:02:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f18f0 0x557f68410e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f683f2210 0x557f6841d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f684053a0 0x557f6841d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8d2243740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81ae170 con 0x557f684053a0 2024-04-25T19:02:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8d2243740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81aba40 con 0x557f683f2210 2024-04-25T19:02:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8d2243740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81a9890 con 0x557f683f18f0 2024-04-25T19:02:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f683f2210 0x557f6841d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8c7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f683f2210 0x557f6841d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:52996/0 (socket says 172.21.15.45:52996) 2024-04-25T19:02:26.090 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8c7e00640 1 -- 172.21.15.45:0/1873956307 learned_addr learned my addr 172.21.15.45:0/1873956307 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8cca00640 1 --2- 172.21.15.45:0/1873956307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f18f0 0x557f68410e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8cd400640 1 --2- 172.21.15.45:0/1873956307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f684053a0 0x557f6841d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8c7e00640 1 -- 172.21.15.45:0/1873956307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f18f0 msgr2=0x557f68410e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8c7e00640 1 --2- 172.21.15.45:0/1873956307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f18f0 0x557f68410e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8c7e00640 1 -- 172.21.15.45:0/1873956307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f684053a0 msgr2=0x557f6841d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8c7e00640 1 --2- 172.21.15.45:0/1873956307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f684053a0 0x557f6841d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.088+0000 7fe8c7e00640 1 -- 172.21.15.45:0/1873956307 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f683f2040 con 0x557f683f2210 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.089+0000 7fe8c7e00640 1 --2- 172.21.15.45:0/1873956307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f683f2210 0x557f6841d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40b94a0 tx=0x7fe8b40f4640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f773604391b63bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.089+0000 7fe8b9200640 1 -- 172.21.15.45:0/1873956307 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4049040 con 0x557f683f2210 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.089+0000 7fe8b9200640 1 -- 172.21.15.45:0/1873956307 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b40bd1d0 con 0x557f683f2210 2024-04-25T19:02:26.091 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.089+0000 7fe8b9200640 1 -- 172.21.15.45:0/1873956307 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40f5040 con 0x557f683f2210 2024-04-25T19:02:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.090+0000 7fe8d2243740 1 -- 172.21.15.45:0/1873956307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f683f2210 msgr2=0x557f6841d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:26.094 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.090+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1873956307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f683f2210 0x557f6841d190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40b94a0 tx=0x7fe8b40f4640 comp rx=0 tx=0).stop 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.090+0000 7fe8d2243740 1 -- 172.21.15.45:0/1873956307 shutdown_connections 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.090+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1873956307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f684053a0 0x557f6841d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.090+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1873956307 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f683f2210 0x557f6841d190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.090+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1873956307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f18f0 0x557f68410e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.090+0000 7fe8d2243740 1 -- 172.21.15.45:0/1873956307 >> 172.21.15.45:0/1873956307 conn(0x557f683edec0 msgr2=0x557f683fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.090+0000 7fe8d2243740 1 -- 172.21.15.45:0/1873956307 shutdown_connections 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.090+0000 7fe8d2243740 1 -- 172.21.15.45:0/1873956307 wait complete. 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.090+0000 7fe8d2243740 1 -- start start 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8d2243740 1 Processor -- start 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f6840a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f2210 0x557f6840aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8d2243740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f684739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.095 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8d2243740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81a9f00 con 0x557f683f18f0 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8d2243740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81ab370 con 0x557f684053a0 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8d2243740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c81aa5d0 con 0x557f683f2210 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8d2243740 10 client.0 _ll_register_callbacks cb 0x7fe8c0009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8cca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f6840a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8cd400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f684739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8cca00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f6840a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51886/0 (socket says 172.21.15.45:51886) 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.091+0000 7fe8cca00640 1 -- 172.21.15.45:0/1387202284 learned_addr learned my addr 172.21.15.45:0/1387202284 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.092+0000 7fe8c7e00640 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f2210 0x557f6840aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.092+0000 7fe8cd400640 1 -- 172.21.15.45:0/1387202284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f2210 msgr2=0x557f6840aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.092+0000 7fe8cd400640 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f2210 0x557f6840aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.092+0000 7fe8cd400640 1 -- 172.21.15.45:0/1387202284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 msgr2=0x557f6840a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.092+0000 7fe8cd400640 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f6840a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.092+0000 7fe8cd400640 1 -- 172.21.15.45:0/1387202284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f68474040 con 0x557f684053a0 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.092+0000 7fe8cca00640 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f6840a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:26.096 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.092+0000 7fe8cd400640 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f684739d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0011910 tx=0x7fe8c0003a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2843b8954b0846ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.092+0000 7fe8ae000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.092+0000 7fe8ae000640 1 -- 172.21.15.45:0/1387202284 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0033040 con 0x557f684053a0 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8ae000640 1 -- 172.21.15.45:0/1387202284 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fe8c001b430 con 0x557f684053a0 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8d2243740 1 -- 172.21.15.45:0/1387202284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8b40bd1d0 con 0x557f684053a0 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8ae000640 1 -- 172.21.15.45:0/1387202284 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8c0051050 con 0x557f684053a0 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8d2243740 10 client.25571 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8d2243740 1 -- 172.21.15.45:0/1387202284 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557f6847b510 con 0x557f684053a0 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8ae000640 1 -- 172.21.15.45:0/1387202284 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8b40bd1d0 con 0x557f684053a0 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8ae000640 1 -- 172.21.15.45:0/1387202284 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fe8c001bce0 con 0x557f684053a0 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8d2243740 10 client.25571 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8d2243740 10 client.25571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8d2243740 20 client.25571 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557f6847bba0 0x557f6847dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8d2243740 4 client.25571 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:02:26.097 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.093+0000 7fe8d2243740 1 -- 172.21.15.45:0/1387202284 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557f68365320 con 0x557f6847bba0 2024-04-25T19:02:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.094+0000 7fe8cca00640 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557f6847bba0 0x557f6847dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.094+0000 7fe8cca00640 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557f6847bba0 0x557f6847dfc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3afb37319a121de server_cookie=7c955eb672a68c4a in_seq=0 out_seq=0 2024-04-25T19:02:26.098 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:26.094+0000 7fe8ae000640 10 client.25571 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.012+0000 7fe8ae000640 1 -- 172.21.15.45:0/1387202284 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557f68365320 con 0x557f6847bba0 2024-04-25T19:02:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.012+0000 7fe8ae000640 10 client.25571 handle_command_reply: tid=0 2024-04-25T19:02:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.012+0000 7fe8d2243740 1 client.25571 shutdown 2024-04-25T19:02:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 1 -- 172.21.15.45:0/1387202284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 msgr2=0x557f684739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f684739d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0011910 tx=0x7fe8c0003a50 comp rx=0 tx=0).stop 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 1 -- 172.21.15.45:0/1387202284 shutdown_connections 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557f6847bba0 0x557f6847dfc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557f684053a0 0x557f684739d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557f683f2210 0x557f6840aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 1 --2- 172.21.15.45:0/1387202284 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557f683f18f0 0x557f6840a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8ae000640 0 client.25571 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 1 -- 172.21.15.45:0/1387202284 >> 172.21.15.45:0/1387202284 conn(0x557f683edec0 msgr2=0x557f68402240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 1 -- 172.21.15.45:0/1387202284 shutdown_connections 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 1 -- 172.21.15.45:0/1387202284 wait complete. 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.013+0000 7fe8d2243740 20 client.25571 trim_cache size 0 max 16384 2024-04-25T19:02:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:02:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:02:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:02:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:02:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 -- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe8b00a3d80 msgr2=0x7fe8b00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe8b00a3d80 0x7fe8b00a6240 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0012fd0 tx=0x7fe8c0021000 comp rx=0 tx=0).stop 2024-04-25T19:02:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 -- 172.21.15.45:0/3098446309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c8103850 msgr2=0x7fe8c81a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c8103850 0x7fe8c81a0960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe8b4032fd0 tx=0x7fe8b4031a20 comp rx=0 tx=0).stop 2024-04-25T19:02:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 -- 172.21.15.45:0/3098446309 shutdown_connections 2024-04-25T19:02:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe8b00a3d80 0x7fe8b00a6240 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe8c810c3a0 0x7fe8c819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe8c8103850 0x7fe8c81a0960 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 --2- 172.21.15.45:0/3098446309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe8c8102f30 0x7fe8c81a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 -- 172.21.15.45:0/3098446309 >> 172.21.15.45:0/3098446309 conn(0x7fe8c80fc790 msgr2=0x7fe8c80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 -- 172.21.15.45:0/3098446309 shutdown_connections 2024-04-25T19:02:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:27.014+0000 7fe8cde00640 1 -- 172.21.15.45:0/3098446309 wait complete. 2024-04-25T19:02:27.660 INFO:tasks.workunit.client.0.smithi045.stderr:Updating files: 7% (321/4290) Updating files: 8% (344/4290) Updating files: 9% (387/4290) Updating files: 10% (429/4290) Updating files: 10% (444/4290) Updating files: 11% (472/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 20% (874/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 30% (1289/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 39% (1685/4290) Updating files: 40% (1716/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 46% (1988/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 54% (2317/4290) Updating files: 54% (2327/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 63% (2724/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 68% (2924/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 73% (3165/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 85% (3683/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 93% (3994/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-25T19:02:27.733 INFO:tasks.workunit.client.0.smithi045.stderr:+ cd xfstests-dev 2024-04-25T19:02:27.734 INFO:tasks.workunit.client.0.smithi045.stderr:+ make -j4 2024-04-25T19:02:27.764 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize -cfi 2024-04-25T19:02:28.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:27 smithi079 ceph-mon[33002]: pgmap v97: 97 pgs: 97 active+clean; 34 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 952 KiB/s wr, 163 op/s 2024-04-25T19:02:28.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:27 smithi045 ceph-mon[27776]: pgmap v97: 97 pgs: 97 active+clean; 34 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 952 KiB/s wr, 163 op/s 2024-04-25T19:02:28.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:27 smithi062 ceph-mon[32844]: pgmap v97: 97 pgs: 97 active+clean; 34 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 952 KiB/s wr, 163 op/s 2024-04-25T19:02:28.794 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-25T19:02:28.794 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file './config.guess' 2024-04-25T19:02:28.809 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file './config.sub' 2024-04-25T19:02:28.828 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file './install-sh' 2024-04-25T19:02:28.849 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file './ltmain.sh' 2024-04-25T19:02:28.957 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-25T19:02:28.957 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-25T19:02:29.011 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-25T19:02:29.042 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-25T19:02:29.073 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-25T19:02:29.106 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-25T19:02:29.136 INFO:tasks.workunit.client.0.smithi045.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-25T19:02:29.141 INFO:tasks.workunit.client.0.smithi045.stdout:cp include/install-sh . 2024-04-25T19:02:29.156 INFO:tasks.workunit.client.0.smithi045.stdout:aclocal -I m4 2024-04-25T19:02:30.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:29 smithi079 ceph-mon[33002]: pgmap v98: 97 pgs: 97 active+clean; 34 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 952 KiB/s wr, 163 op/s 2024-04-25T19:02:30.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:29 smithi045 ceph-mon[27776]: pgmap v98: 97 pgs: 97 active+clean; 34 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 952 KiB/s wr, 163 op/s 2024-04-25T19:02:30.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:29 smithi062 ceph-mon[32844]: pgmap v98: 97 pgs: 97 active+clean; 34 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 705 KiB/s rd, 952 KiB/s wr, 163 op/s 2024-04-25T19:02:30.641 INFO:tasks.workunit.client.0.smithi045.stdout:autoheader 2024-04-25T19:02:31.108 INFO:tasks.workunit.client.0.smithi045.stdout:autoconf 2024-04-25T19:02:31.374 INFO:tasks.workunit.client.0.smithi045.stdout:./configure \ 2024-04-25T19:02:31.374 INFO:tasks.workunit.client.0.smithi045.stdout: --libexecdir=/usr/lib \ 2024-04-25T19:02:31.374 INFO:tasks.workunit.client.0.smithi045.stdout: --exec_prefix=/var/lib 2024-04-25T19:02:31.817 INFO:tasks.workunit.client.0.smithi045.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-25T19:02:31.817 INFO:tasks.workunit.client.0.smithi045.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-25T19:02:31.820 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to print strings... printf 2024-04-25T19:02:31.821 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc... gcc 2024-04-25T19:02:31.986 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether the C compiler works... yes 2024-04-25T19:02:31.986 INFO:tasks.workunit.client.0.smithi045.stdout:checking for C compiler default output file name... a.out 2024-04-25T19:02:32.065 INFO:tasks.workunit.client.0.smithi045.stdout:checking for suffix of executables... 2024-04-25T19:02:32.146 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are cross compiling... no 2024-04-25T19:02:32.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:31 smithi079 ceph-mon[33002]: pgmap v99: 97 pgs: 97 active+clean; 25 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.4 MiB/s wr, 645 op/s 2024-04-25T19:02:32.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:31 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:02:32.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:31 smithi045 ceph-mon[27776]: pgmap v99: 97 pgs: 97 active+clean; 25 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.4 MiB/s wr, 645 op/s 2024-04-25T19:02:32.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:31 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:02:32.221 INFO:tasks.workunit.client.0.smithi045.stdout:checking for suffix of object files... o 2024-04-25T19:02:32.286 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T19:02:32.337 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether gcc accepts -g... yes 2024-04-25T19:02:32.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:31 smithi062 ceph-mon[32844]: pgmap v99: 97 pgs: 97 active+clean; 25 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 956 KiB/s rd, 1.4 MiB/s wr, 645 op/s 2024-04-25T19:02:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:31 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:02:32.465 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T19:02:32.480 INFO:tasks.workunit.client.0.smithi045.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-25T19:02:32.494 INFO:tasks.workunit.client.0.smithi045.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T19:02:32.498 INFO:tasks.workunit.client.0.smithi045.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T19:02:32.503 INFO:tasks.workunit.client.0.smithi045.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-25T19:02:32.517 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-25T19:02:32.525 INFO:tasks.workunit.client.0.smithi045.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-25T19:02:32.537 INFO:tasks.workunit.client.0.smithi045.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-25T19:02:33.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:32 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:02:33.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:32 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:02:33.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:32 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:02:33.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:32 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:02:33.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:32 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:02:33.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:32 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:02:33.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:32 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:02:33.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:32 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:02:33.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:32 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:02:33.621 INFO:tasks.workunit.client.0.smithi045.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-25T19:02:33.621 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether ln -s works... yes 2024-04-25T19:02:33.670 INFO:tasks.workunit.client.0.smithi045.stdout:checking the maximum length of command line arguments... 1572864 2024-04-25T19:02:33.682 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-25T19:02:33.682 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-25T19:02:33.682 INFO:tasks.workunit.client.0.smithi045.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-25T19:02:33.683 INFO:tasks.workunit.client.0.smithi045.stdout:checking for objdump... objdump 2024-04-25T19:02:33.683 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to recognize dependent libraries... pass_all 2024-04-25T19:02:33.684 INFO:tasks.workunit.client.0.smithi045.stdout:checking for dlltool... no 2024-04-25T19:02:33.684 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-25T19:02:33.684 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ar... ar 2024-04-25T19:02:33.860 INFO:tasks.workunit.client.0.smithi045.stdout:checking for archiver @FILE support... @ 2024-04-25T19:02:33.861 INFO:tasks.workunit.client.0.smithi045.stdout:checking for strip... strip 2024-04-25T19:02:33.862 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ranlib... ranlib 2024-04-25T19:02:33.863 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gawk... gawk 2024-04-25T19:02:34.104 INFO:tasks.workunit.client.0.smithi045.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-25T19:02:34.110 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sysroot... no 2024-04-25T19:02:34.133 INFO:tasks.workunit.client.0.smithi045.stdout:checking for a working dd... /usr/bin/dd 2024-04-25T19:02:34.154 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-25T19:02:34.227 INFO:tasks.workunit.client.0.smithi045.stdout:checking for mt... no 2024-04-25T19:02:34.256 INFO:tasks.workunit.client.0.smithi045.stdout:checking if : is a manifest tool... no 2024-04-25T19:02:34.440 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T19:02:34.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:34 smithi062 ceph-mon[32844]: pgmap v100: 97 pgs: 97 active+clean; 25 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.2 MiB/s wr, 628 op/s 2024-04-25T19:02:34.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:34 smithi079 ceph-mon[33002]: pgmap v100: 97 pgs: 97 active+clean; 25 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.2 MiB/s wr, 628 op/s 2024-04-25T19:02:34.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:34 smithi045 ceph-mon[27776]: pgmap v100: 97 pgs: 97 active+clean; 25 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.2 MiB/s wr, 628 op/s 2024-04-25T19:02:34.860 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ANSI C header files... yes 2024-04-25T19:02:34.942 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/types.h... yes 2024-04-25T19:02:35.041 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/stat.h... yes 2024-04-25T19:02:35.135 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stdlib.h... yes 2024-04-25T19:02:35.231 INFO:tasks.workunit.client.0.smithi045.stdout:checking for string.h... yes 2024-04-25T19:02:35.330 INFO:tasks.workunit.client.0.smithi045.stdout:checking for memory.h... yes 2024-04-25T19:02:35.431 INFO:tasks.workunit.client.0.smithi045.stdout:checking for strings.h... yes 2024-04-25T19:02:35.525 INFO:tasks.workunit.client.0.smithi045.stdout:checking for inttypes.h... yes 2024-04-25T19:02:35.629 INFO:tasks.workunit.client.0.smithi045.stdout:checking for stdint.h... yes 2024-04-25T19:02:35.740 INFO:tasks.workunit.client.0.smithi045.stdout:checking for unistd.h... yes 2024-04-25T19:02:35.844 INFO:tasks.workunit.client.0.smithi045.stdout:checking for dlfcn.h... yes 2024-04-25T19:02:35.862 INFO:tasks.workunit.client.0.smithi045.stdout:checking for objdir... .libs 2024-04-25T19:02:36.149 INFO:tasks.workunit.client.0.smithi045.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-25T19:02:36.149 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-25T19:02:36.248 INFO:tasks.workunit.client.0.smithi045.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-25T19:02:36.348 INFO:tasks.workunit.client.0.smithi045.stdout:checking if gcc static flag -static works... no 2024-04-25T19:02:36.515 INFO:tasks.workunit.client.0.smithi045.stdout:checking if gcc supports -c -o file.o... yes 2024-04-25T19:02:36.515 INFO:tasks.workunit.client.0.smithi045.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-25T19:02:36.542 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-25T19:02:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:36 smithi062 ceph-mon[32844]: pgmap v101: 97 pgs: 97 active+clean; 26 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.3 MiB/s wr, 636 op/s 2024-04-25T19:02:36.642 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether -lc should be explicitly linked in... no 2024-04-25T19:02:36.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:36 smithi079 ceph-mon[33002]: pgmap v101: 97 pgs: 97 active+clean; 26 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.3 MiB/s wr, 636 op/s 2024-04-25T19:02:36.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:36 smithi045 ceph-mon[27776]: pgmap v101: 97 pgs: 97 active+clean; 26 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 1.3 MiB/s wr, 636 op/s 2024-04-25T19:02:36.804 INFO:tasks.workunit.client.0.smithi045.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-25T19:02:36.804 INFO:tasks.workunit.client.0.smithi045.stdout:checking how to hardcode library paths into programs... immediate 2024-04-25T19:02:36.808 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether stripping libraries is possible... yes 2024-04-25T19:02:36.808 INFO:tasks.workunit.client.0.smithi045.stdout:checking if libtool supports shared libraries... yes 2024-04-25T19:02:36.809 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether to build shared libraries... yes 2024-04-25T19:02:36.809 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether to build static libraries... yes 2024-04-25T19:02:36.830 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc... (cached) gcc 2024-04-25T19:02:36.915 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T19:02:36.915 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T19:02:36.916 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T19:02:36.917 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gmake... /usr/bin/gmake 2024-04-25T19:02:36.918 INFO:tasks.workunit.client.0.smithi045.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-25T19:02:36.919 INFO:tasks.workunit.client.0.smithi045.stdout:checking for tar... /bin/tar 2024-04-25T19:02:36.920 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gzip... /bin/gzip 2024-04-25T19:02:36.979 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether gcc -MM is supported... yes 2024-04-25T19:02:36.981 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sort... /bin/sort 2024-04-25T19:02:36.981 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether ln -s works... yes 2024-04-25T19:02:36.982 INFO:tasks.workunit.client.0.smithi045.stdout:checking for rpm... /bin/rpm 2024-04-25T19:02:36.996 INFO:tasks.workunit.client.0.smithi045.stdout:checking for rpmbuild... no 2024-04-25T19:02:37.134 INFO:tasks.workunit.client.0.smithi045.stdout:checking assert.h usability... yes 2024-04-25T19:02:37.174 INFO:tasks.workunit.client.0.smithi045.stdout:checking assert.h presence... yes 2024-04-25T19:02:37.174 INFO:tasks.workunit.client.0.smithi045.stdout:checking for assert.h... yes 2024-04-25T19:02:37.295 INFO:tasks.workunit.client.0.smithi045.stdout:checking bstring.h usability... no 2024-04-25T19:02:37.362 INFO:tasks.workunit.client.0.smithi045.stdout:checking bstring.h presence... no 2024-04-25T19:02:37.362 INFO:tasks.workunit.client.0.smithi045.stdout:checking for bstring.h... no 2024-04-25T19:02:37.462 INFO:tasks.workunit.client.0.smithi045.stdout:checking libgen.h usability... yes 2024-04-25T19:02:37.501 INFO:tasks.workunit.client.0.smithi045.stdout:checking libgen.h presence... yes 2024-04-25T19:02:37.501 INFO:tasks.workunit.client.0.smithi045.stdout:checking for libgen.h... yes 2024-04-25T19:02:37.618 INFO:tasks.workunit.client.0.smithi045.stdout:checking dirent.h usability... yes 2024-04-25T19:02:37.662 INFO:tasks.workunit.client.0.smithi045.stdout:checking dirent.h presence... yes 2024-04-25T19:02:37.662 INFO:tasks.workunit.client.0.smithi045.stdout:checking for dirent.h... yes 2024-04-25T19:02:37.774 INFO:tasks.workunit.client.0.smithi045.stdout:checking errno.h usability... yes 2024-04-25T19:02:37.813 INFO:tasks.workunit.client.0.smithi045.stdout:checking errno.h presence... yes 2024-04-25T19:02:37.813 INFO:tasks.workunit.client.0.smithi045.stdout:checking for errno.h... yes 2024-04-25T19:02:37.920 INFO:tasks.workunit.client.0.smithi045.stdout:checking malloc.h usability... yes 2024-04-25T19:02:37.962 INFO:tasks.workunit.client.0.smithi045.stdout:checking malloc.h presence... yes 2024-04-25T19:02:37.963 INFO:tasks.workunit.client.0.smithi045.stdout:checking for malloc.h... yes 2024-04-25T19:02:38.082 INFO:tasks.workunit.client.0.smithi045.stdout:checking uuid.h usability... no 2024-04-25T19:02:38.151 INFO:tasks.workunit.client.0.smithi045.stdout:checking uuid.h presence... no 2024-04-25T19:02:38.151 INFO:tasks.workunit.client.0.smithi045.stdout:checking for uuid.h... no 2024-04-25T19:02:38.251 INFO:tasks.workunit.client.0.smithi045.stdout:checking uuid/uuid.h usability... yes 2024-04-25T19:02:38.294 INFO:tasks.workunit.client.0.smithi045.stdout:checking uuid/uuid.h presence... yes 2024-04-25T19:02:38.294 INFO:tasks.workunit.client.0.smithi045.stdout:checking for uuid/uuid.h... yes 2024-04-25T19:02:38.418 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/uuid.h usability... no 2024-04-25T19:02:38.486 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/uuid.h presence... no 2024-04-25T19:02:38.486 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/uuid.h... no 2024-04-25T19:02:38.590 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/file.h usability... yes 2024-04-25T19:02:38.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:38 smithi062 ceph-mon[32844]: pgmap v102: 97 pgs: 97 active+clean; 26 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 587 KiB/s wr, 489 op/s 2024-04-25T19:02:38.632 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/file.h presence... yes 2024-04-25T19:02:38.632 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/file.h... yes 2024-04-25T19:02:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:38 smithi079 ceph-mon[33002]: pgmap v102: 97 pgs: 97 active+clean; 26 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 587 KiB/s wr, 489 op/s 2024-04-25T19:02:38.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:38 smithi045 ceph-mon[27776]: pgmap v102: 97 pgs: 97 active+clean; 26 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 587 KiB/s wr, 489 op/s 2024-04-25T19:02:38.749 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fcntl.h usability... yes 2024-04-25T19:02:38.792 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fcntl.h presence... yes 2024-04-25T19:02:38.792 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/fcntl.h... yes 2024-04-25T19:02:38.912 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/syssgi.h usability... no 2024-04-25T19:02:38.980 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/syssgi.h presence... no 2024-04-25T19:02:38.980 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/syssgi.h... no 2024-04-25T19:02:39.114 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/param.h usability... yes 2024-04-25T19:02:39.163 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/param.h presence... yes 2024-04-25T19:02:39.163 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/param.h... yes 2024-04-25T19:02:39.200 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/stat.h... (cached) yes 2024-04-25T19:02:39.292 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/statvfs.h usability... yes 2024-04-25T19:02:39.334 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/statvfs.h presence... yes 2024-04-25T19:02:39.334 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/statvfs.h... yes 2024-04-25T19:02:39.434 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/time.h usability... yes 2024-04-25T19:02:39.475 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/time.h presence... yes 2024-04-25T19:02:39.475 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/time.h... yes 2024-04-25T19:02:39.586 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/ioctl.h usability... yes 2024-04-25T19:02:39.628 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/ioctl.h presence... yes 2024-04-25T19:02:39.628 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/ioctl.h... yes 2024-04-25T19:02:39.748 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/wait.h usability... yes 2024-04-25T19:02:39.799 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/wait.h presence... yes 2024-04-25T19:02:39.799 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/wait.h... yes 2024-04-25T19:02:39.839 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/types.h... (cached) yes 2024-04-25T19:02:39.858 INFO:tasks.workunit.client.0.smithi045.stdout:checking for strings.h... (cached) yes 2024-04-25T19:02:39.948 INFO:tasks.workunit.client.0.smithi045.stdout:checking err.h usability... yes 2024-04-25T19:02:39.993 INFO:tasks.workunit.client.0.smithi045.stdout:checking err.h presence... yes 2024-04-25T19:02:39.993 INFO:tasks.workunit.client.0.smithi045.stdout:checking for err.h... yes 2024-04-25T19:02:40.109 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/falloc.h usability... yes 2024-04-25T19:02:40.153 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/falloc.h presence... yes 2024-04-25T19:02:40.154 INFO:tasks.workunit.client.0.smithi045.stdout:checking for linux/falloc.h... yes 2024-04-25T19:02:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.162+0000 7f5143200640 1 -- 172.21.15.45:0/1759981250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51441038f0 msgr2=0x7f514410bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.162+0000 7f5143200640 1 --2- 172.21.15.45:0/1759981250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51441038f0 0x7f514410bdd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5134009900 tx=0x7f513402a480 comp rx=0 tx=0).stop 2024-04-25T19:02:40.163 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.162+0000 7f5143200640 1 -- 172.21.15.45:0/1759981250 shutdown_connections 2024-04-25T19:02:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.162+0000 7f5143200640 1 --2- 172.21.15.45:0/1759981250 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f514410c3a0 0x7f514410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.162+0000 7f5143200640 1 --2- 172.21.15.45:0/1759981250 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f51441038f0 0x7f514410bdd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.162+0000 7f5143200640 1 --2- 172.21.15.45:0/1759981250 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5144102fd0 0x7f51441033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.162+0000 7f5143200640 1 -- 172.21.15.45:0/1759981250 >> 172.21.15.45:0/1759981250 conn(0x7f51440fc730 msgr2=0x7f51440feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.162+0000 7f5143200640 1 -- 172.21.15.45:0/1759981250 shutdown_connections 2024-04-25T19:02:40.164 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.162+0000 7f5143200640 1 -- 172.21.15.45:0/1759981250 wait complete. 2024-04-25T19:02:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.163+0000 7f5143200640 1 Processor -- start 2024-04-25T19:02:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.163+0000 7f5143200640 1 -- start start 2024-04-25T19:02:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f5143200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5144102fd0 0x7f5144198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f5143200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51441038f0 0x7f5144198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f5143200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514410c3a0 0x7f514419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.165 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f5143200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5144112010 con 0x7f5144102fd0 2024-04-25T19:02:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f5143200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f5144111e90 con 0x7f514410c3a0 2024-04-25T19:02:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f5143200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5144112190 con 0x7f51441038f0 2024-04-25T19:02:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f513be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51441038f0 0x7f5144198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f5141e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5144102fd0 0x7f5144198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f513be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51441038f0 0x7f5144198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:47706/0 (socket says 172.21.15.45:47706) 2024-04-25T19:02:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f513be00640 1 -- 172.21.15.45:0/4144289956 learned_addr learned my addr 172.21.15.45:0/4144289956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:40.166 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.164+0000 7f5142800640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514410c3a0 0x7f514419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.165+0000 7f513be00640 1 -- 172.21.15.45:0/4144289956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514410c3a0 msgr2=0x7f514419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.165+0000 7f513be00640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514410c3a0 0x7f514419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.165+0000 7f513be00640 1 -- 172.21.15.45:0/4144289956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5144102fd0 msgr2=0x7f5144198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.165+0000 7f513be00640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5144102fd0 0x7f5144198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.165+0000 7f513be00640 1 -- 172.21.15.45:0/4144289956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514419d560 con 0x7f51441038f0 2024-04-25T19:02:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.165+0000 7f5141e00640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5144102fd0 0x7f5144198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:40.167 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.165+0000 7f5142800640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514410c3a0 0x7f514419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.165+0000 7f513be00640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51441038f0 0x7f5144198ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5134005ab0 tx=0x7f5134003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.165+0000 7f513aa00640 1 -- 172.21.15.45:0/4144289956 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513403f070 con 0x7f51441038f0 2024-04-25T19:02:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.166+0000 7f513aa00640 1 -- 172.21.15.45:0/4144289956 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5134031b60 con 0x7f51441038f0 2024-04-25T19:02:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.166+0000 7f5143200640 1 -- 172.21.15.45:0/4144289956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514419d7f0 con 0x7f51441038f0 2024-04-25T19:02:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.166+0000 7f513aa00640 1 -- 172.21.15.45:0/4144289956 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134035030 con 0x7f51441038f0 2024-04-25T19:02:40.168 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.166+0000 7f5143200640 1 -- 172.21.15.45:0/4144289956 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51441a50e0 con 0x7f51441038f0 2024-04-25T19:02:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.169+0000 7f513aa00640 1 -- 172.21.15.45:0/4144289956 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f5134042070 con 0x7f51441038f0 2024-04-25T19:02:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.169+0000 7f514be76740 1 Processor -- start 2024-04-25T19:02:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.169+0000 7f514be76740 1 -- start start 2024-04-25T19:02:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f514be76740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e629b0 0x5588c6e62d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f514be76740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e632d0 0x5588c6e6d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f514be76740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e6db90 0x5588c6e6ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f514be76740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5588c6dc8900 con 0x5588c6e629b0 2024-04-25T19:02:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f514be76740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5588c6d6eef0 con 0x5588c6e6db90 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f514be76740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5588c6dc48c0 con 0x5588c6e632d0 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f5142800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e629b0 0x5588c6e62d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f5141e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e632d0 0x5588c6e6d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f5141e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e632d0 0x5588c6e6d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:47716/0 (socket says 172.21.15.45:47716) 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f513be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e6db90 0x5588c6e6ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.170+0000 7f5141e00640 1 -- 172.21.15.45:0/559876428 learned_addr learned my addr 172.21.15.45:0/559876428 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f5141e00640 1 -- 172.21.15.45:0/559876428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e6db90 msgr2=0x5588c6e6ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f5141e00640 1 --2- 172.21.15.45:0/559876428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e6db90 0x5588c6e6ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f5141e00640 1 -- 172.21.15.45:0/559876428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e629b0 msgr2=0x5588c6e62d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f5141e00640 1 --2- 172.21.15.45:0/559876428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e629b0 0x5588c6e62d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f5141e00640 1 -- 172.21.15.45:0/559876428 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588c6adb210 con 0x5588c6e632d0 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f513aa00640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51280a3d00 0x7f51280a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f5141e00640 1 --2- 172.21.15.45:0/559876428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e632d0 0x5588c6e6d650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f512c0115f0 tx=0x7f512c011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d121c3d03af97121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f513aa00640 1 -- 172.21.15.45:0/4144289956 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f51340f0ea0 con 0x7f51441038f0 2024-04-25T19:02:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f5133400640 1 -- 172.21.15.45:0/559876428 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c01e070 con 0x5588c6e632d0 2024-04-25T19:02:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f5142800640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51280a3d00 0x7f51280a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f5133400640 1 -- 172.21.15.45:0/559876428 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512c00faf0 con 0x5588c6e632d0 2024-04-25T19:02:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.171+0000 7f5133400640 1 -- 172.21.15.45:0/559876428 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c019bc0 con 0x5588c6e632d0 2024-04-25T19:02:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.172+0000 7f514be76740 1 -- 172.21.15.45:0/559876428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e632d0 msgr2=0x5588c6e6d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.172+0000 7f514be76740 1 --2- 172.21.15.45:0/559876428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e632d0 0x5588c6e6d650 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f512c0115f0 tx=0x7f512c011ab0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.172+0000 7f5142800640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51280a3d00 0x7f51280a61c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f513c009fd0 tx=0x7f513c009500 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.172+0000 7f514be76740 1 -- 172.21.15.45:0/559876428 shutdown_connections 2024-04-25T19:02:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.172+0000 7f514be76740 1 --2- 172.21.15.45:0/559876428 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e6db90 0x5588c6e6ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.172+0000 7f514be76740 1 --2- 172.21.15.45:0/559876428 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e632d0 0x5588c6e6d650 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.172+0000 7f514be76740 1 --2- 172.21.15.45:0/559876428 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e629b0 0x5588c6e62d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.172+0000 7f514be76740 1 -- 172.21.15.45:0/559876428 >> 172.21.15.45:0/559876428 conn(0x5588c6e5eec0 msgr2=0x5588c6e5f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.172+0000 7f514be76740 1 -- 172.21.15.45:0/559876428 shutdown_connections 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.173+0000 7f514be76740 1 -- 172.21.15.45:0/559876428 wait complete. 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.173+0000 7f514be76740 1 -- start start 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.173+0000 7f514be76740 1 Processor -- start 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f514be76740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 0x5588c6e6d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f514be76740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e6ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f514be76740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 0x5588c6e6b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f514be76740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5588c6e707c0 con 0x5588c6e6db90 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f514be76740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5588c6d82310 con 0x5588c6e632d0 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f514be76740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5588c6e70940 con 0x5588c6e629b0 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f514be76740 10 client.0 _ll_register_callbacks cb 0x5588c6d50cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f513be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e6ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f513be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e6ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:55476/0 (socket says 172.21.15.45:55476) 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f513be00640 1 -- 172.21.15.45:0/27356808 learned_addr learned my addr 172.21.15.45:0/27356808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f5141e00640 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 0x5588c6e6d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f5142800640 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 0x5588c6e6b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f513be00640 1 -- 172.21.15.45:0/27356808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 msgr2=0x5588c6e6d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.174+0000 7f513be00640 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 0x5588c6e6d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f513be00640 1 -- 172.21.15.45:0/27356808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 msgr2=0x5588c6e6b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f513be00640 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 0x5588c6e6b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f513be00640 1 -- 172.21.15.45:0/27356808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588c6ba7620 con 0x5588c6e632d0 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f5142800640 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 0x5588c6e6b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f513be00640 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e6ab10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5134005ec0 tx=0x7f513402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f283da3792cccf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f5127e00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f5127e00640 1 -- 172.21.15.45:0/27356808 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134049070 con 0x5588c6e632d0 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f5127e00640 1 -- 172.21.15.45:0/27356808 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f513403b460 con 0x5588c6e632d0 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f514be76740 1 -- 172.21.15.45:0/27356808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f512c00faf0 con 0x5588c6e632d0 2024-04-25T19:02:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f5127e00640 1 -- 172.21.15.45:0/27356808 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f51340bc7f0 con 0x5588c6e632d0 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f514be76740 10 client.25583 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f514be76740 1 -- 172.21.15.45:0/27356808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588c6e85250 con 0x5588c6e632d0 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.175+0000 7f5127e00640 1 -- 172.21.15.45:0/27356808 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512c00faf0 con 0x5588c6e632d0 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.176+0000 7f5127e00640 1 -- 172.21.15.45:0/27356808 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f51340eece0 con 0x5588c6e632d0 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.176+0000 7f514be76740 10 client.25583 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.176+0000 7f514be76740 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.176+0000 7f514be76740 20 client.25583 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.176+0000 7f514be76740 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5588c6e867a0 0x5588c6e88c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.176+0000 7f514be76740 4 client.25583 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.176+0000 7f514be76740 1 -- 172.21.15.45:0/27356808 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588c6ddc8b0 con 0x5588c6e867a0 2024-04-25T19:02:40.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.176+0000 7f5141e00640 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5588c6e867a0 0x5588c6e88c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.177+0000 7f5141e00640 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5588c6e867a0 0x5588c6e88c20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73ecf644c4863e1a server_cookie=eaa3639e6f38aa22 in_seq=0 out_seq=0 2024-04-25T19:02:40.181 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.177+0000 7f5127e00640 10 client.25583 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.177+0000 7f5127e00640 1 -- 172.21.15.45:0/27356808 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588c6ddc8b0 con 0x5588c6e867a0 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.177+0000 7f5127e00640 10 client.25583 handle_command_reply: tid=0 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 client.25583 shutdown 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 -- 172.21.15.45:0/27356808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 msgr2=0x5588c6e6ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e6ab10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5134005ec0 tx=0x7f513402a480 comp rx=0 tx=0).stop 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 -- 172.21.15.45:0/27356808 shutdown_connections 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5588c6e867a0 0x5588c6e88c20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 0x5588c6e6b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e6ab10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f5127e00640 0 client.25583 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 --2- 172.21.15.45:0/27356808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 0x5588c6e6d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 -- 172.21.15.45:0/27356808 >> 172.21.15.45:0/27356808 conn(0x5588c6e5eec0 msgr2=0x5588c6e64c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 -- 172.21.15.45:0/27356808 shutdown_connections 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 1 -- 172.21.15.45:0/27356808 wait complete. 2024-04-25T19:02:40.182 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.178+0000 7f514be76740 20 client.25583 trim_cache size 0 max 16384 2024-04-25T19:02:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f514be76740 1 Processor -- start 2024-04-25T19:02:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f514be76740 1 -- start start 2024-04-25T19:02:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f514be76740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e629b0 0x5588c6e6ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f514be76740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e632d0 0x5588c6e69c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f514be76740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e6db90 0x5588c6e6d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f514be76740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ae150 con 0x5588c6e632d0 2024-04-25T19:02:40.199 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f514be76740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51441aba20 con 0x5588c6e629b0 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f514be76740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51441a9870 con 0x5588c6e6db90 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f5141e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e629b0 0x5588c6e6ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f513be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e632d0 0x5588c6e69c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f5142800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e6db90 0x5588c6e6d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f5141e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e629b0 0x5588c6e6ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:55480/0 (socket says 172.21.15.45:55480) 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.197+0000 7f5141e00640 1 -- 172.21.15.45:0/1933161516 learned_addr learned my addr 172.21.15.45:0/1933161516 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.198+0000 7f5141e00640 1 -- 172.21.15.45:0/1933161516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e6db90 msgr2=0x5588c6e6d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.198+0000 7f5141e00640 1 --2- 172.21.15.45:0/1933161516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e6db90 0x5588c6e6d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.198+0000 7f5141e00640 1 -- 172.21.15.45:0/1933161516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e632d0 msgr2=0x5588c6e69c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.198+0000 7f5141e00640 1 --2- 172.21.15.45:0/1933161516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e632d0 0x5588c6e69c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.198+0000 7f5141e00640 1 -- 172.21.15.45:0/1933161516 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588c6e701b0 con 0x5588c6e629b0 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.198+0000 7f513be00640 1 --2- 172.21.15.45:0/1933161516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e632d0 0x5588c6e69c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.198+0000 7f5141e00640 1 --2- 172.21.15.45:0/1933161516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e629b0 0x5588c6e6ad00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f512c019810 tx=0x7f512c019700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b6d02e575ff0718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:40.200 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.198+0000 7f5132a00640 1 -- 172.21.15.45:0/1933161516 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c02a030 con 0x5588c6e629b0 2024-04-25T19:02:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.198+0000 7f5132a00640 1 -- 172.21.15.45:0/1933161516 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512c005560 con 0x5588c6e629b0 2024-04-25T19:02:40.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.198+0000 7f5132a00640 1 -- 172.21.15.45:0/1933161516 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c01e040 con 0x5588c6e629b0 2024-04-25T19:02:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.199+0000 7f514be76740 1 -- 172.21.15.45:0/1933161516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e629b0 msgr2=0x5588c6e6ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.199+0000 7f514be76740 1 --2- 172.21.15.45:0/1933161516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e629b0 0x5588c6e6ad00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f512c019810 tx=0x7f512c019700 comp rx=0 tx=0).stop 2024-04-25T19:02:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.199+0000 7f514be76740 1 -- 172.21.15.45:0/1933161516 shutdown_connections 2024-04-25T19:02:40.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.199+0000 7f514be76740 1 --2- 172.21.15.45:0/1933161516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e6db90 0x5588c6e6d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.199+0000 7f514be76740 1 --2- 172.21.15.45:0/1933161516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e632d0 0x5588c6e69c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.199+0000 7f514be76740 1 --2- 172.21.15.45:0/1933161516 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e629b0 0x5588c6e6ad00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.199+0000 7f514be76740 1 -- 172.21.15.45:0/1933161516 >> 172.21.15.45:0/1933161516 conn(0x5588c6e5eec0 msgr2=0x5588c6e72b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.199+0000 7f514be76740 1 -- 172.21.15.45:0/1933161516 shutdown_connections 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.199+0000 7f514be76740 1 -- 172.21.15.45:0/1933161516 wait complete. 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.200+0000 7f514be76740 1 -- start start 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.200+0000 7f514be76740 1 Processor -- start 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f514be76740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 0x5588c6e80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f514be76740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e81470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f514be76740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 0x5588c6eebdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f514be76740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51441a9ee0 con 0x5588c6e6db90 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f514be76740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f51441ab350 con 0x5588c6e632d0 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f514be76740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f51441aa5b0 con 0x5588c6e629b0 2024-04-25T19:02:40.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f514be76740 10 client.0 _ll_register_callbacks cb 0x7f5134009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f5141e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 0x5588c6e80f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f5141e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 0x5588c6e80f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:47742/0 (socket says 172.21.15.45:47742) 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f513be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e81470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f5141e00640 1 -- 172.21.15.45:0/1232106662 learned_addr learned my addr 172.21.15.45:0/1232106662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f5142800640 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 0x5588c6eebdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f5141e00640 1 -- 172.21.15.45:0/1232106662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 msgr2=0x5588c6e81470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f5141e00640 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e81470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f5141e00640 1 -- 172.21.15.45:0/1232106662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 msgr2=0x5588c6eebdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f5141e00640 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 0x5588c6eebdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f5141e00640 1 -- 172.21.15.45:0/1232106662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588c6eec340 con 0x5588c6e629b0 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.201+0000 7f513be00640 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e81470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.202+0000 7f5141e00640 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 0x5588c6e80f30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f512c0115c0 tx=0x7f512c005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=160c94644f92aad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.202+0000 7f5127e00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.202+0000 7f5127e00640 1 -- 172.21.15.45:0/1232106662 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c032020 con 0x5588c6e629b0 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.202+0000 7f514be76740 1 -- 172.21.15.45:0/1232106662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f512c005560 con 0x5588c6e629b0 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.202+0000 7f5127e00640 1 -- 172.21.15.45:0/1232106662 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f512c016070 con 0x5588c6e629b0 2024-04-25T19:02:40.205 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.202+0000 7f5127e00640 1 -- 172.21.15.45:0/1232106662 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f512c03d050 con 0x5588c6e629b0 2024-04-25T19:02:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.202+0000 7f514be76740 10 client.25609 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.202+0000 7f514be76740 1 -- 172.21.15.45:0/1232106662 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5588c6eed650 con 0x5588c6e629b0 2024-04-25T19:02:40.206 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.203+0000 7f5127e00640 1 -- 172.21.15.45:0/1232106662 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f512c005560 con 0x5588c6e629b0 2024-04-25T19:02:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.203+0000 7f5127e00640 1 -- 172.21.15.45:0/1232106662 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f512c00f740 con 0x5588c6e629b0 2024-04-25T19:02:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.203+0000 7f514be76740 10 client.25609 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.203+0000 7f514be76740 10 client.25609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:02:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.203+0000 7f514be76740 20 client.25609 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.203+0000 7f514be76740 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5588c6eedc00 0x5588c6ef0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.203+0000 7f514be76740 4 client.25609 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:02:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.203+0000 7f514be76740 1 -- 172.21.15.45:0/1232106662 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588c6e6b3e0 con 0x5588c6eedc00 2024-04-25T19:02:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.203+0000 7f513be00640 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5588c6eedc00 0x5588c6ef0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.204+0000 7f513be00640 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5588c6eedc00 0x5588c6ef0020 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81176d62cdc155ee server_cookie=56f8e32f80345a0a in_seq=0 out_seq=0 2024-04-25T19:02:40.207 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.204+0000 7f5127e00640 10 client.25609 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:40.361 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-25T19:02:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.360+0000 7f5127e00640 1 -- 172.21.15.45:0/1232106662 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588c6e6b3e0 con 0x5588c6eedc00 2024-04-25T19:02:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.360+0000 7f5127e00640 10 client.25609 handle_command_reply: tid=0 2024-04-25T19:02:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.360+0000 7f514be76740 1 client.25609 shutdown 2024-04-25T19:02:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.360+0000 7f514be76740 1 -- 172.21.15.45:0/1232106662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 msgr2=0x5588c6e80f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f514be76740 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 0x5588c6e80f30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f512c0115c0 tx=0x7f512c005780 comp rx=0 tx=0).stop 2024-04-25T19:02:40.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f514be76740 1 -- 172.21.15.45:0/1232106662 shutdown_connections 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f514be76740 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5588c6eedc00 0x5588c6ef0020 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f514be76740 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5588c6e6db90 0x5588c6eebdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f514be76740 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5588c6e632d0 0x5588c6e81470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f514be76740 1 --2- 172.21.15.45:0/1232106662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5588c6e629b0 0x5588c6e80f30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f5127e00640 0 client.25609 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f514be76740 1 -- 172.21.15.45:0/1232106662 >> 172.21.15.45:0/1232106662 conn(0x5588c6e5eec0 msgr2=0x5588c6e733f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f514be76740 1 -- 172.21.15.45:0/1232106662 shutdown_connections 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f514be76740 1 -- 172.21.15.45:0/1232106662 wait complete. 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.361+0000 7f514be76740 20 client.25609 trim_cache size 0 max 16384 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:02:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:02:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:02:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:02:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 -- 172.21.15.45:0/4144289956 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51280a3d00 msgr2=0x7f51280a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51280a3d00 0x7f51280a61c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f513c009fd0 tx=0x7f513c009500 comp rx=0 tx=0).stop 2024-04-25T19:02:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 -- 172.21.15.45:0/4144289956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51441038f0 msgr2=0x7f5144198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51441038f0 0x7f5144198ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5134005ab0 tx=0x7f5134003c10 comp rx=0 tx=0).stop 2024-04-25T19:02:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 -- 172.21.15.45:0/4144289956 shutdown_connections 2024-04-25T19:02:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f51280a3d00 0x7f51280a61c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f514410c3a0 0x7f514419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f51441038f0 0x7f5144198ab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 --2- 172.21.15.45:0/4144289956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5144102fd0 0x7f5144198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 -- 172.21.15.45:0/4144289956 >> 172.21.15.45:0/4144289956 conn(0x7f51440fc730 msgr2=0x7f51440fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 -- 172.21.15.45:0/4144289956 shutdown_connections 2024-04-25T19:02:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:40.362+0000 7f5143200640 1 -- 172.21.15.45:0/4144289956 wait complete. 2024-04-25T19:02:40.412 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-25T19:02:40.412 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-25T19:02:40.490 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-25T19:02:40.547 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-25T19:02:40.547 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-25T19:02:40.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:40 smithi062 ceph-mon[32844]: pgmap v103: 97 pgs: 97 active+clean; 26 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 587 KiB/s wr, 489 op/s 2024-04-25T19:02:40.645 INFO:tasks.workunit.client.0.smithi045.stdout:checking xfs/platform_defs.h usability... no 2024-04-25T19:02:40.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:40 smithi079 ceph-mon[33002]: pgmap v103: 97 pgs: 97 active+clean; 26 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 587 KiB/s wr, 489 op/s 2024-04-25T19:02:40.712 INFO:tasks.workunit.client.0.smithi045.stdout:checking xfs/platform_defs.h presence... no 2024-04-25T19:02:40.712 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/platform_defs.h... no 2024-04-25T19:02:40.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:40 smithi045 ceph-mon[27776]: pgmap v103: 97 pgs: 97 active+clean; 26 MiB data, 537 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 587 KiB/s wr, 489 op/s 2024-04-25T19:02:40.813 INFO:tasks.workunit.client.0.smithi045.stdout:checking btrfs/ioctl.h usability... no 2024-04-25T19:02:40.884 INFO:tasks.workunit.client.0.smithi045.stdout:checking btrfs/ioctl.h presence... no 2024-04-25T19:02:40.884 INFO:tasks.workunit.client.0.smithi045.stdout:checking for btrfs/ioctl.h... no 2024-04-25T19:02:40.988 INFO:tasks.workunit.client.0.smithi045.stdout:checking cifs/ioctl.h usability... no 2024-04-25T19:02:41.051 INFO:tasks.workunit.client.0.smithi045.stdout:checking cifs/ioctl.h presence... no 2024-04-25T19:02:41.052 INFO:tasks.workunit.client.0.smithi045.stdout:checking for cifs/ioctl.h... no 2024-04-25T19:02:41.159 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/mman.h usability... yes 2024-04-25T19:02:41.208 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/mman.h presence... yes 2024-04-25T19:02:41.208 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/mman.h... yes 2024-04-25T19:02:41.330 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/ext4.h usability... no 2024-04-25T19:02:41.401 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/ext4.h presence... no 2024-04-25T19:02:41.401 INFO:tasks.workunit.client.0.smithi045.stdout:checking for linux/ext4.h... no 2024-04-25T19:02:41.479 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/xfs_log_format.h... no 2024-04-25T19:02:41.615 INFO:tasks.workunit.client.0.smithi045.stdout:checking for uuid_compare... no 2024-04-25T19:02:41.713 INFO:tasks.workunit.client.0.smithi045.stdout:checking for uuid_compare in -luuid... yes 2024-04-25T19:02:41.842 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/xfs.h... yes 2024-04-25T19:02:41.917 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/libxfs.h... no 2024-04-25T19:02:41.995 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-25T19:02:42.088 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/xqm.h... yes 2024-04-25T19:02:42.211 INFO:tasks.workunit.client.0.smithi045.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-25T19:02:42.299 INFO:tasks.workunit.client.0.smithi045.stdout:checking xfs/handle.h usability... yes 2024-04-25T19:02:42.345 INFO:tasks.workunit.client.0.smithi045.stdout:checking xfs/handle.h presence... yes 2024-04-25T19:02:42.345 INFO:tasks.workunit.client.0.smithi045.stdout:checking for xfs/handle.h... yes 2024-04-25T19:02:42.469 INFO:tasks.workunit.client.0.smithi045.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-25T19:02:42.569 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/acl.h usability... yes 2024-04-25T19:02:42.606 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/acl.h presence... yes 2024-04-25T19:02:42.606 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/acl.h... yes 2024-04-25T19:02:42.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:42 smithi062 ceph-mon[32844]: pgmap v104: 97 pgs: 97 active+clean; 28 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 732 KiB/s wr, 498 op/s 2024-04-25T19:02:42.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:42 smithi079 ceph-mon[33002]: pgmap v104: 97 pgs: 97 active+clean; 28 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 732 KiB/s wr, 498 op/s 2024-04-25T19:02:42.697 INFO:tasks.workunit.client.0.smithi045.stdout:checking acl/libacl.h usability... yes 2024-04-25T19:02:42.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:42 smithi045 ceph-mon[27776]: pgmap v104: 97 pgs: 97 active+clean; 28 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 732 KiB/s wr, 498 op/s 2024-04-25T19:02:42.736 INFO:tasks.workunit.client.0.smithi045.stdout:checking acl/libacl.h presence... yes 2024-04-25T19:02:42.736 INFO:tasks.workunit.client.0.smithi045.stdout:checking for acl/libacl.h... yes 2024-04-25T19:02:42.836 INFO:tasks.workunit.client.0.smithi045.stdout:checking for acl_init in -lacl... yes 2024-04-25T19:02:42.970 INFO:tasks.workunit.client.0.smithi045.stdout:checking gdbm-ndbm.h usability... no 2024-04-25T19:02:43.037 INFO:tasks.workunit.client.0.smithi045.stdout:checking gdbm-ndbm.h presence... no 2024-04-25T19:02:43.038 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gdbm-ndbm.h... no 2024-04-25T19:02:43.131 INFO:tasks.workunit.client.0.smithi045.stdout:checking gdbm/ndbm.h usability... no 2024-04-25T19:02:43.197 INFO:tasks.workunit.client.0.smithi045.stdout:checking gdbm/ndbm.h presence... no 2024-04-25T19:02:43.198 INFO:tasks.workunit.client.0.smithi045.stdout:checking for gdbm/ndbm.h... no 2024-04-25T19:02:43.289 INFO:tasks.workunit.client.0.smithi045.stdout:checking ndbm.h usability... no 2024-04-25T19:02:43.349 INFO:tasks.workunit.client.0.smithi045.stdout:checking ndbm.h presence... no 2024-04-25T19:02:43.349 INFO:tasks.workunit.client.0.smithi045.stdout:checking for ndbm.h... no 2024-04-25T19:02:43.427 INFO:tasks.workunit.client.0.smithi045.stdout:checking libaio.h usability... yes 2024-04-25T19:02:43.472 INFO:tasks.workunit.client.0.smithi045.stdout:checking libaio.h presence... yes 2024-04-25T19:02:43.472 INFO:tasks.workunit.client.0.smithi045.stdout:checking for libaio.h... yes 2024-04-25T19:02:43.570 INFO:tasks.workunit.client.0.smithi045.stdout:checking liburing.h usability... no 2024-04-25T19:02:43.646 INFO:tasks.workunit.client.0.smithi045.stdout:checking liburing.h presence... no 2024-04-25T19:02:43.646 INFO:tasks.workunit.client.0.smithi045.stdout:checking for liburing.h... no 2024-04-25T19:02:44.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:44 smithi062 ceph-mon[32844]: pgmap v105: 97 pgs: 97 active+clean; 28 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-04-25T19:02:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:44 smithi079 ceph-mon[33002]: pgmap v105: 97 pgs: 97 active+clean; 28 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-04-25T19:02:44.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:44 smithi045 ceph-mon[27776]: pgmap v105: 97 pgs: 97 active+clean; 28 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-04-25T19:02:44.838 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/capability.h usability... no 2024-04-25T19:02:44.900 INFO:tasks.workunit.client.0.smithi045.stdout:checking sys/capability.h presence... no 2024-04-25T19:02:44.901 INFO:tasks.workunit.client.0.smithi045.stdout:checking for sys/capability.h... no 2024-04-25T19:02:44.982 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/fiemap.h usability... yes 2024-04-25T19:02:45.023 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/fiemap.h presence... yes 2024-04-25T19:02:45.023 INFO:tasks.workunit.client.0.smithi045.stdout:checking for linux/fiemap.h... yes 2024-04-25T19:02:45.126 INFO:tasks.workunit.client.0.smithi045.stdout:checking for fallocate... yes 2024-04-25T19:02:45.233 INFO:tasks.workunit.client.0.smithi045.stdout:checking for open_by_handle_at... yes 2024-04-25T19:02:45.313 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/fs.h usability... yes 2024-04-25T19:02:45.352 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/fs.h presence... yes 2024-04-25T19:02:45.352 INFO:tasks.workunit.client.0.smithi045.stdout:checking for linux/fs.h... yes 2024-04-25T19:02:45.461 INFO:tasks.workunit.client.0.smithi045.stdout:checking btrfsutil.h usability... no 2024-04-25T19:02:45.524 INFO:tasks.workunit.client.0.smithi045.stdout:checking btrfsutil.h presence... no 2024-04-25T19:02:45.524 INFO:tasks.workunit.client.0.smithi045.stdout:checking for btrfsutil.h... no 2024-04-25T19:02:45.637 INFO:tasks.workunit.client.0.smithi045.stdout:checking for copy_file_range... yes 2024-04-25T19:02:45.723 INFO:tasks.workunit.client.0.smithi045.stdout:checking for SEEK_DATA... yes 2024-04-25T19:02:45.845 INFO:tasks.workunit.client.0.smithi045.stdout:checking for BMV_OF_SHARED... yes 2024-04-25T19:02:45.953 INFO:tasks.workunit.client.0.smithi045.stdout:checking for nftw... yes 2024-04-25T19:02:46.069 INFO:tasks.workunit.client.0.smithi045.stdout:checking for RLIMIT_NOFILE... yes 2024-04-25T19:02:46.170 INFO:tasks.workunit.client.0.smithi045.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-25T19:02:46.312 INFO:tasks.workunit.client.0.smithi045.stdout:checking for FICLONE... yes 2024-04-25T19:02:46.417 INFO:tasks.workunit.client.0.smithi045.stdout:checking for renameat2... yes 2024-04-25T19:02:46.523 INFO:tasks.workunit.client.0.smithi045.stdout:checking for reallocarray... yes 2024-04-25T19:02:46.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:46 smithi062 ceph-mon[32844]: pgmap v106: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.2 MiB/s wr, 313 op/s 2024-04-25T19:02:46.663 INFO:tasks.workunit.client.0.smithi045.stdout:checking for struct mount_attr... yes 2024-04-25T19:02:46.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:46 smithi079 ceph-mon[33002]: pgmap v106: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.2 MiB/s wr, 313 op/s 2024-04-25T19:02:46.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:46 smithi045 ceph-mon[27776]: pgmap v106: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 1.2 MiB/s wr, 313 op/s 2024-04-25T19:02:46.816 INFO:tasks.workunit.client.0.smithi045.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-25T19:02:46.963 INFO:tasks.workunit.client.0.smithi045.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-25T19:02:47.105 INFO:tasks.workunit.client.0.smithi045.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-25T19:02:47.249 INFO:tasks.workunit.client.0.smithi045.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-25T19:02:47.389 INFO:tasks.workunit.client.0.smithi045.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-25T19:02:47.532 INFO:tasks.workunit.client.0.smithi045.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-25T19:02:47.676 INFO:tasks.workunit.client.0.smithi045.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-25T19:02:47.777 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/btrfs.h usability... yes 2024-04-25T19:02:47.823 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/btrfs.h presence... yes 2024-04-25T19:02:47.823 INFO:tasks.workunit.client.0.smithi045.stdout:checking for linux/btrfs.h... yes 2024-04-25T19:02:47.932 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-25T19:02:47.979 INFO:tasks.workunit.client.0.smithi045.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-25T19:02:47.980 INFO:tasks.workunit.client.0.smithi045.stdout:checking for linux/btrfs_tree.h... yes 2024-04-25T19:02:48.072 INFO:tasks.workunit.client.0.smithi045.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-25T19:02:48.155 INFO:tasks.workunit.client.0.smithi045.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-25T19:02:48.200 INFO:tasks.workunit.client.0.smithi045.stdout:configure: creating ./config.status 2024-04-25T19:02:48.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:48 smithi062 ceph-mon[32844]: pgmap v107: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-04-25T19:02:48.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:48 smithi079 ceph-mon[33002]: pgmap v107: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-04-25T19:02:48.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:48 smithi045 ceph-mon[27776]: pgmap v107: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-04-25T19:02:49.578 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: creating include/builddefs 2024-04-25T19:02:49.631 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: creating include/config.h 2024-04-25T19:02:49.671 INFO:tasks.workunit.client.0.smithi045.stdout:config.status: executing libtool commands 2024-04-25T19:02:49.865 INFO:tasks.workunit.client.0.smithi045.stdout:Building include 2024-04-25T19:02:49.865 INFO:tasks.workunit.client.0.smithi045.stdout:Building lib 2024-04-25T19:02:49.866 INFO:tasks.workunit.client.0.smithi045.stdout:Building tests 2024-04-25T19:02:49.882 INFO:tasks.workunit.client.0.smithi045.stdout: [LTDEP] 2024-04-25T19:02:49.886 INFO:tasks.workunit.client.0.smithi045.stdout:Building btrfs 2024-04-25T19:02:49.887 INFO:tasks.workunit.client.0.smithi045.stdout:Building ceph 2024-04-25T19:02:49.888 INFO:tasks.workunit.client.0.smithi045.stdout:Building cifs 2024-04-25T19:02:49.911 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-04-25T19:02:49.911 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-04-25T19:02:49.941 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-04-25T19:02:49.971 INFO:tasks.workunit.client.0.smithi045.stdout:Building ext4 2024-04-25T19:02:49.983 INFO:tasks.workunit.client.0.smithi045.stdout:Building f2fs 2024-04-25T19:02:49.998 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-04-25T19:02:50.000 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-04-25T19:02:50.067 INFO:tasks.workunit.client.0.smithi045.stdout:Building generic 2024-04-25T19:02:50.266 INFO:tasks.workunit.client.0.smithi045.stdout:Building nfs 2024-04-25T19:02:50.273 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-04-25T19:02:50.282 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-04-25T19:02:50.336 INFO:tasks.workunit.client.0.smithi045.stdout:Building ocfs2 2024-04-25T19:02:50.338 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] dataascii.lo 2024-04-25T19:02:50.350 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-04-25T19:02:50.401 INFO:tasks.workunit.client.0.smithi045.stdout:Building overlay 2024-04-25T19:02:50.431 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-04-25T19:02:50.582 INFO:tasks.workunit.client.0.smithi045.stdout:Building perf 2024-04-25T19:02:50.597 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-04-25T19:02:50.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:50 smithi062 ceph-mon[32844]: pgmap v108: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-04-25T19:02:50.639 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] databin.lo 2024-04-25T19:02:50.645 INFO:tasks.workunit.client.0.smithi045.stdout:Building selftest 2024-04-25T19:02:50.660 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-04-25T19:02:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:50 smithi079 ceph-mon[33002]: pgmap v108: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-04-25T19:02:50.703 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] datapid.lo 2024-04-25T19:02:50.703 INFO:tasks.workunit.client.0.smithi045.stdout:Building shared 2024-04-25T19:02:50.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:50 smithi045 ceph-mon[27776]: pgmap v108: 97 pgs: 97 active+clean; 34 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.1 MiB/s wr, 306 op/s 2024-04-25T19:02:50.718 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-04-25T19:02:50.776 INFO:tasks.workunit.client.0.smithi045.stdout:Building tmpfs 2024-04-25T19:02:50.790 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-04-25T19:02:50.811 INFO:tasks.workunit.client.0.smithi045.stdout:Building udf 2024-04-25T19:02:50.825 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-04-25T19:02:50.843 INFO:tasks.workunit.client.0.smithi045.stdout:Building xfs 2024-04-25T19:02:50.869 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] file_lock.lo 2024-04-25T19:02:51.026 INFO:tasks.workunit.client.0.smithi045.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-04-25T19:02:51.123 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] forker.lo 2024-04-25T19:02:51.189 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] pattern.lo 2024-04-25T19:02:51.293 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] open_flags.lo 2024-04-25T19:02:51.418 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] random_range.lo 2024-04-25T19:02:51.460 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] string_to_tokens.lo 2024-04-25T19:02:51.473 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] str_to_bytes.lo 2024-04-25T19:02:51.610 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] tlibio.lo 2024-04-25T19:02:51.722 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] write_log.lo 2024-04-25T19:02:51.794 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] random.lo 2024-04-25T19:02:52.166 INFO:tasks.workunit.client.0.smithi045.stdout: [LD] 2024-04-25T19:02:52.429 INFO:tasks.workunit.client.0.smithi045.stdout:Building ltp 2024-04-25T19:02:52.430 INFO:tasks.workunit.client.0.smithi045.stdout:Building src 2024-04-25T19:02:52.430 INFO:tasks.workunit.client.0.smithi045.stdout:Building m4 2024-04-25T19:02:52.433 INFO:tasks.workunit.client.0.smithi045.stdout:Building common 2024-04-25T19:02:52.445 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-04-25T19:02:52.468 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-04-25T19:02:52.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:52 smithi062 ceph-mon[32844]: pgmap v109: 97 pgs: 97 active+clean; 34 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 333 op/s 2024-04-25T19:02:52.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:52 smithi079 ceph-mon[33002]: pgmap v109: 97 pgs: 97 active+clean; 34 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 333 op/s 2024-04-25T19:02:52.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:52 smithi045 ceph-mon[27776]: pgmap v109: 97 pgs: 97 active+clean; 34 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 670 KiB/s rd, 1.2 MiB/s wr, 333 op/s 2024-04-25T19:02:52.854 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] doio 2024-04-25T19:02:52.855 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fsstress 2024-04-25T19:02:52.856 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fsx 2024-04-25T19:02:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.554+0000 7f070e800640 1 -- 172.21.15.45:0/1055901500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07081043c0 msgr2=0x7f07081047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.554+0000 7f070e800640 1 --2- 172.21.15.45:0/1055901500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07081043c0 0x7f07081047a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009860 tx=0x7f06f802a340 comp rx=0 tx=0).stop 2024-04-25T19:02:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.554+0000 7f070e800640 1 -- 172.21.15.45:0/1055901500 shutdown_connections 2024-04-25T19:02:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.554+0000 7f070e800640 1 --2- 172.21.15.45:0/1055901500 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0708109490 0x7f0708111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.554+0000 7f070e800640 1 --2- 172.21.15.45:0/1055901500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0708104d70 0x7f0708108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.554+0000 7f070e800640 1 --2- 172.21.15.45:0/1055901500 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f07081043c0 0x7f07081047a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.554+0000 7f070e800640 1 -- 172.21.15.45:0/1055901500 >> 172.21.15.45:0/1055901500 conn(0x7f07080ffc20 msgr2=0x7f0708102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.554+0000 7f070e800640 1 -- 172.21.15.45:0/1055901500 shutdown_connections 2024-04-25T19:02:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.554+0000 7f070e800640 1 -- 172.21.15.45:0/1055901500 wait complete. 2024-04-25T19:02:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.555+0000 7f070e800640 1 Processor -- start 2024-04-25T19:02:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.556+0000 7f070e800640 1 -- start start 2024-04-25T19:02:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.556+0000 7f070e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07081043c0 0x7f070819aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.556+0000 7f070e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0708109490 0x7f070819afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.556+0000 7f070e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f070819f390 0x7f070819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.556+0000 7f070e800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f07081143d0 con 0x7f0708109490 2024-04-25T19:02:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.556+0000 7f070e800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f0708114250 con 0x7f070819f390 2024-04-25T19:02:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.556+0000 7f070e800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0708114550 con 0x7f07081043c0 2024-04-25T19:02:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.557+0000 7f0706a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0708109490 0x7f070819afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.557+0000 7f0707400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07081043c0 0x7f070819aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.557+0000 7f0707400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07081043c0 0x7f070819aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:53642/0 (socket says 172.21.15.45:53642) 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.557+0000 7f0707400640 1 -- 172.21.15.45:0/3393210883 learned_addr learned my addr 172.21.15.45:0/3393210883 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.557+0000 7f0706a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0708109490 0x7f070819afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54478/0 (socket says 172.21.15.45:54478) 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.557+0000 7f0707e00640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f070819f390 0x7f070819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.557+0000 7f0706a00640 1 -- 172.21.15.45:0/3393210883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07081043c0 msgr2=0x7f070819aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.557+0000 7f0706a00640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07081043c0 0x7f070819aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.558+0000 7f0706a00640 1 -- 172.21.15.45:0/3393210883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f070819f390 msgr2=0x7f070819f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.558+0000 7f0706a00640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f070819f390 0x7f070819f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.558+0000 7f0706a00640 1 -- 172.21.15.45:0/3393210883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070819fe80 con 0x7f0708109490 2024-04-25T19:02:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.558+0000 7f0707e00640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f070819f390 0x7f070819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.558+0000 7f0706a00640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0708109490 0x7f070819afe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f06f4005fd0 tx=0x7f06f400eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.558+0000 7f06ffe00640 1 -- 172.21.15.45:0/3393210883 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f400cab0 con 0x7f0708109490 2024-04-25T19:02:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.559+0000 7f070e800640 1 -- 172.21.15.45:0/3393210883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07081a0170 con 0x7f0708109490 2024-04-25T19:02:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.559+0000 7f06ffe00640 1 -- 172.21.15.45:0/3393210883 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f400cc50 con 0x7f0708109490 2024-04-25T19:02:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.559+0000 7f06ffe00640 1 -- 172.21.15.45:0/3393210883 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4018680 con 0x7f0708109490 2024-04-25T19:02:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.559+0000 7f070e800640 1 -- 172.21.15.45:0/3393210883 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07081a7980 con 0x7f0708109490 2024-04-25T19:02:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.565+0000 7f071157e740 1 Processor -- start 2024-04-25T19:02:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.565+0000 7f071157e740 1 -- start start 2024-04-25T19:02:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.565+0000 7f071157e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af4820 0x563238af4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.565+0000 7f071157e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af5140 0x563238aff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.566+0000 7f071157e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238affa00 0x563238b01df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.566+0000 7f071157e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563238a5a900 con 0x563238af5140 2024-04-25T19:02:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.566+0000 7f071157e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563238a00ef0 con 0x563238af4820 2024-04-25T19:02:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.566+0000 7f071157e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563238a568c0 con 0x563238affa00 2024-04-25T19:02:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.566+0000 7f0707e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af4820 0x563238af4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.566+0000 7f0707400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af5140 0x563238aff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.566+0000 7f0707e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af4820 0x563238af4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41000/0 (socket says 172.21.15.45:41000) 2024-04-25T19:02:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.566+0000 7f0707e00640 1 -- 172.21.15.45:0/1779743060 learned_addr learned my addr 172.21.15.45:0/1779743060 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.567+0000 7f0707400640 1 -- 172.21.15.45:0/1779743060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238affa00 msgr2=0x563238b01df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T19:02:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.567+0000 7f0707400640 1 --2- 172.21.15.45:0/1779743060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238affa00 0x563238b01df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.567+0000 7f0707400640 1 -- 172.21.15.45:0/1779743060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af4820 msgr2=0x563238af4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.567+0000 7f0707400640 1 --2- 172.21.15.45:0/1779743060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af4820 0x563238af4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.567+0000 7f0707400640 1 -- 172.21.15.45:0/1779743060 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323876d210 con 0x563238af5140 2024-04-25T19:02:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.568+0000 7f0707400640 1 --2- 172.21.15.45:0/1779743060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af5140 0x563238aff4c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f06f8000c00 tx=0x7f06f8005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de535eaeedb446f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.568+0000 7f06ffe00640 1 -- 172.21.15.45:0/3393210883 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f06f4010430 con 0x7f0708109490 2024-04-25T19:02:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.568+0000 7f06fcc00640 1 -- 172.21.15.45:0/1779743060 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8041020 con 0x563238af5140 2024-04-25T19:02:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.568+0000 7f06fcc00640 1 -- 172.21.15.45:0/1779743060 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f8030dd0 con 0x563238af5140 2024-04-25T19:02:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.568+0000 7f06fcc00640 1 -- 172.21.15.45:0/1779743060 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8037040 con 0x563238af5140 2024-04-25T19:02:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.569+0000 7f071157e740 1 -- 172.21.15.45:0/1779743060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af5140 msgr2=0x563238aff4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:53.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.569+0000 7f071157e740 1 --2- 172.21.15.45:0/1779743060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af5140 0x563238aff4c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f06f8000c00 tx=0x7f06f8005b20 comp rx=0 tx=0).stop 2024-04-25T19:02:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.571+0000 7f071157e740 1 -- 172.21.15.45:0/1779743060 shutdown_connections 2024-04-25T19:02:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.571+0000 7f071157e740 1 --2- 172.21.15.45:0/1779743060 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238affa00 0x563238b01df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.571+0000 7f071157e740 1 --2- 172.21.15.45:0/1779743060 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af5140 0x563238aff4c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.571+0000 7f071157e740 1 --2- 172.21.15.45:0/1779743060 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af4820 0x563238af4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.571+0000 7f071157e740 1 -- 172.21.15.45:0/1779743060 >> 172.21.15.45:0/1779743060 conn(0x563238af0d30 msgr2=0x563238af1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.571+0000 7f06ffe00640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f06ec0a3d10 0x7f06ec0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.571+0000 7f071157e740 1 -- 172.21.15.45:0/1779743060 shutdown_connections 2024-04-25T19:02:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.571+0000 7f06ffe00640 1 -- 172.21.15.45:0/3393210883 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f06f40d0a20 con 0x7f0708109490 2024-04-25T19:02:53.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.571+0000 7f071157e740 1 -- 172.21.15.45:0/1779743060 wait complete. 2024-04-25T19:02:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.571+0000 7f0707400640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f06ec0a3d10 0x7f06ec0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.572+0000 7f071157e740 1 -- start start 2024-04-25T19:02:53.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.572+0000 7f071157e740 1 Processor -- start 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.573+0000 7f0707400640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f06ec0a3d10 0x7f06ec0a61d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06f8031000 tx=0x7f06f802a340 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.573+0000 7f071157e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238afc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.573+0000 7f071157e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af5140 0x563238afcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.573+0000 7f071157e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238affa00 0x563238afd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.573+0000 7f071157e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563238b025d0 con 0x563238af4820 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.573+0000 7f071157e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563238a14310 con 0x563238af5140 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.573+0000 7f071157e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563238b02750 con 0x563238affa00 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.573+0000 7f071157e740 10 client.0 _ll_register_callbacks cb 0x5632389e2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0706a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af5140 0x563238afcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0706a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af5140 0x563238afcf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41004/0 (socket says 172.21.15.45:41004) 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0706a00640 1 -- 172.21.15.45:0/3692858265 learned_addr learned my addr 172.21.15.45:0/3692858265 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:53.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0707e00640 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238affa00 0x563238afd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0707400640 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238afc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0707e00640 1 -- 172.21.15.45:0/3692858265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af5140 msgr2=0x563238afcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0707e00640 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af5140 0x563238afcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0707e00640 1 -- 172.21.15.45:0/3692858265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 msgr2=0x563238afc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0707e00640 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238afc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0707e00640 1 -- 172.21.15.45:0/3692858265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563238839620 con 0x563238affa00 2024-04-25T19:02:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.574+0000 7f0707400640 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238afc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:02:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.575+0000 7f0707e00640 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238affa00 0x563238afd440 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f070000fe00 tx=0x7f0700012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25e8de4c1f11d027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.575+0000 7f06f1600640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:02:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.575+0000 7f06f1600640 1 -- 172.21.15.45:0/3692858265 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700004b20 con 0x563238affa00 2024-04-25T19:02:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.575+0000 7f06f1600640 1 -- 172.21.15.45:0/3692858265 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f0700004cc0 con 0x563238affa00 2024-04-25T19:02:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.575+0000 7f071157e740 1 -- 172.21.15.45:0/3692858265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f8030dd0 con 0x563238affa00 2024-04-25T19:02:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.576+0000 7f06f1600640 1 -- 172.21.15.45:0/3692858265 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f070001c470 con 0x563238affa00 2024-04-25T19:02:53.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.576+0000 7f071157e740 10 client.25618 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.576+0000 7f071157e740 1 -- 172.21.15.45:0/3692858265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563238b170a0 con 0x563238affa00 2024-04-25T19:02:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.576+0000 7f06f1600640 1 -- 172.21.15.45:0/3692858265 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f8030dd0 con 0x563238affa00 2024-04-25T19:02:53.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.576+0000 7f06f1600640 1 -- 172.21.15.45:0/3692858265 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f0700025430 con 0x563238affa00 2024-04-25T19:02:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.577+0000 7f071157e740 10 client.25618 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.577+0000 7f071157e740 10 client.25618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:02:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.577+0000 7f071157e740 20 client.25618 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.577+0000 7f071157e740 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563238b18560 0x563238b1a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.577+0000 7f071157e740 4 client.25618 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:02:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.577+0000 7f071157e740 1 -- 172.21.15.45:0/3692858265 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563238a6e8b0 con 0x563238b18560 2024-04-25T19:02:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.577+0000 7f0707400640 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563238b18560 0x563238b1a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.578+0000 7f0707400640 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563238b18560 0x563238b1a9e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c52f084810faf48 server_cookie=6bb7ce9eb1de8c0d in_seq=0 out_seq=0 2024-04-25T19:02:53.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.578+0000 7f06f1600640 10 client.25618 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.579+0000 7f06f1600640 1 -- 172.21.15.45:0/3692858265 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563238a6e8b0 con 0x563238b18560 2024-04-25T19:02:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.579+0000 7f06f1600640 10 client.25618 handle_command_reply: tid=0 2024-04-25T19:02:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.579+0000 7f071157e740 1 client.25618 shutdown 2024-04-25T19:02:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.579+0000 7f071157e740 1 -- 172.21.15.45:0/3692858265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238affa00 msgr2=0x563238afd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:53.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.579+0000 7f071157e740 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238affa00 0x563238afd440 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f070000fe00 tx=0x7f0700012820 comp rx=0 tx=0).stop 2024-04-25T19:02:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.580+0000 7f071157e740 1 -- 172.21.15.45:0/3692858265 shutdown_connections 2024-04-25T19:02:53.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.580+0000 7f071157e740 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563238b18560 0x563238b1a9e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.580+0000 7f071157e740 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238affa00 0x563238afd440 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.580+0000 7f071157e740 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238af5140 0x563238afcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.580+0000 7f06f1600640 0 client.25618 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.580+0000 7f071157e740 1 --2- 172.21.15.45:0/3692858265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238afc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.580+0000 7f071157e740 1 -- 172.21.15.45:0/3692858265 >> 172.21.15.45:0/3692858265 conn(0x563238af0d30 msgr2=0x563238af6ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.580+0000 7f071157e740 1 -- 172.21.15.45:0/3692858265 shutdown_connections 2024-04-25T19:02:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.580+0000 7f071157e740 1 -- 172.21.15.45:0/3692858265 wait complete. 2024-04-25T19:02:53.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.580+0000 7f071157e740 20 client.25618 trim_cache size 0 max 16384 2024-04-25T19:02:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.613+0000 7f071157e740 1 Processor -- start 2024-04-25T19:02:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.614+0000 7f071157e740 1 -- start start 2024-04-25T19:02:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.614+0000 7f071157e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238af00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.614+0000 7f071157e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238af5140 0x563238af7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.614+0000 7f071157e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 0x563238afc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.614+0000 7f071157e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f07081ae340 con 0x563238af4820 2024-04-25T19:02:53.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.614+0000 7f071157e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07081abc10 con 0x563238affa00 2024-04-25T19:02:53.617 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.614+0000 7f071157e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f07081a9a60 con 0x563238af5140 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.614+0000 7f0707400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238af00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.615+0000 7f0707400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238af00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54510/0 (socket says 172.21.15.45:54510) 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.615+0000 7f0707400640 1 -- 172.21.15.45:0/217054383 learned_addr learned my addr 172.21.15.45:0/217054383 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.615+0000 7f0706a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238af5140 0x563238af7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.615+0000 7f0707e00640 1 --2- 172.21.15.45:0/217054383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 0x563238afc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.615+0000 7f0707400640 1 -- 172.21.15.45:0/217054383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238af5140 msgr2=0x563238af7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.615+0000 7f0707400640 1 --2- 172.21.15.45:0/217054383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238af5140 0x563238af7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.615+0000 7f0707400640 1 -- 172.21.15.45:0/217054383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 msgr2=0x563238afc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.615+0000 7f0707400640 1 --2- 172.21.15.45:0/217054383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 0x563238afc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.615+0000 7f0707400640 1 -- 172.21.15.45:0/217054383 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563238af1ec0 con 0x563238af4820 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.615+0000 7f0707e00640 1 --2- 172.21.15.45:0/217054383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 0x563238afc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:02:53.618 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.616+0000 7f0707400640 1 --2- 172.21.15.45:0/217054383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238af00d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06f8030a80 tx=0x7f06f8030340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23896298932b3155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.616+0000 7f06f3e00640 1 -- 172.21.15.45:0/217054383 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8041050 con 0x563238af4820 2024-04-25T19:02:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.616+0000 7f06f3e00640 1 -- 172.21.15.45:0/217054383 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f8058020 con 0x563238af4820 2024-04-25T19:02:53.619 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.616+0000 7f06f3e00640 1 -- 172.21.15.45:0/217054383 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8037040 con 0x563238af4820 2024-04-25T19:02:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.617+0000 7f071157e740 1 -- 172.21.15.45:0/217054383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 msgr2=0x563238af00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.617+0000 7f071157e740 1 --2- 172.21.15.45:0/217054383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238af00d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06f8030a80 tx=0x7f06f8030340 comp rx=0 tx=0).stop 2024-04-25T19:02:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.617+0000 7f071157e740 1 -- 172.21.15.45:0/217054383 shutdown_connections 2024-04-25T19:02:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.617+0000 7f071157e740 1 --2- 172.21.15.45:0/217054383 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 0x563238afc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.617+0000 7f071157e740 1 --2- 172.21.15.45:0/217054383 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238af5140 0x563238af7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.617+0000 7f071157e740 1 --2- 172.21.15.45:0/217054383 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238af00d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.617+0000 7f071157e740 1 -- 172.21.15.45:0/217054383 >> 172.21.15.45:0/217054383 conn(0x563238af0d30 msgr2=0x563238afd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.617+0000 7f071157e740 1 -- 172.21.15.45:0/217054383 shutdown_connections 2024-04-25T19:02:53.620 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.618+0000 7f071157e740 1 -- 172.21.15.45:0/217054383 wait complete. 2024-04-25T19:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.618+0000 7f071157e740 1 -- start start 2024-04-25T19:02:53.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.618+0000 7f071157e740 1 Processor -- start 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.619+0000 7f071157e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238b105f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.619+0000 7f071157e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238af5140 0x563238b0d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.619+0000 7f071157e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 0x563238b0df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.619+0000 7f071157e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f07081aa0d0 con 0x563238af4820 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.619+0000 7f071157e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f07081ab540 con 0x563238affa00 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.619+0000 7f071157e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f07081aa7a0 con 0x563238af5140 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.619+0000 7f071157e740 10 client.0 _ll_register_callbacks cb 0x7f070000fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.619+0000 7f0707400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238b105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.620+0000 7f0707400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238b105f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54526/0 (socket says 172.21.15.45:54526) 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.620+0000 7f0707400640 1 -- 172.21.15.45:0/381586277 learned_addr learned my addr 172.21.15.45:0/381586277 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:02:53.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.620+0000 7f0706a00640 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238af5140 0x563238b0d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.620+0000 7f0707e00640 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 0x563238b0df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.620+0000 7f0707400640 1 -- 172.21.15.45:0/381586277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238af5140 msgr2=0x563238b0d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.620+0000 7f0707400640 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238af5140 0x563238b0d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.620+0000 7f0707400640 1 -- 172.21.15.45:0/381586277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 msgr2=0x563238b0df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.620+0000 7f0707400640 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 0x563238b0df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.620+0000 7f0707400640 1 -- 172.21.15.45:0/381586277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563238b0e740 con 0x563238af4820 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.620+0000 7f0707e00640 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 0x563238b0df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.621+0000 7f0707400640 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238b105f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f06f803d4a0 tx=0x7f06f80305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bea510cb935d584 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.621+0000 7f06f1600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.621+0000 7f06f1600640 1 -- 172.21.15.45:0/381586277 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f8041030 con 0x563238af4820 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.621+0000 7f06f1600640 1 -- 172.21.15.45:0/381586277 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f06f8005050 con 0x563238af4820 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.621+0000 7f071157e740 1 -- 172.21.15.45:0/381586277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06f8058020 con 0x563238af4820 2024-04-25T19:02:53.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.621+0000 7f06f1600640 1 -- 172.21.15.45:0/381586277 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f06f806a050 con 0x563238af4820 2024-04-25T19:02:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.621+0000 7f071157e740 10 client.15477 fetch_fsmap learned FSMap version 28 2024-04-25T19:02:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.621+0000 7f071157e740 1 -- 172.21.15.45:0/381586277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563238b7eba0 con 0x563238af4820 2024-04-25T19:02:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.622+0000 7f06f1600640 1 -- 172.21.15.45:0/381586277 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06f8058020 con 0x563238af4820 2024-04-25T19:02:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.622+0000 7f06f1600640 1 -- 172.21.15.45:0/381586277 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f06f80046c0 con 0x563238af4820 2024-04-25T19:02:53.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.623+0000 7f071157e740 10 client.15477 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:02:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.623+0000 7f071157e740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:02:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.623+0000 7f071157e740 20 client.15477 populate_metadata read hostname 'smithi045' 2024-04-25T19:02:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.623+0000 7f071157e740 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563238b18560 0x563238b80e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:02:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.623+0000 7f071157e740 4 client.15477 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:02:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.623+0000 7f071157e740 1 -- 172.21.15.45:0/381586277 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563238a68320 con 0x563238b18560 2024-04-25T19:02:53.626 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.623+0000 7f0706a00640 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563238b18560 0x563238b80e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:02:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.624+0000 7f0706a00640 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563238b18560 0x563238b80e30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe665cb9eb3c3e server_cookie=b41dbfabc9b131aa in_seq=0 out_seq=0 2024-04-25T19:02:53.627 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.624+0000 7f06f1600640 10 client.15477 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.729+0000 7f06f1600640 1 -- 172.21.15.45:0/381586277 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563238a68320 con 0x563238b18560 2024-04-25T19:02:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.729+0000 7f06f1600640 10 client.15477 handle_command_reply: tid=0 2024-04-25T19:02:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.730+0000 7f071157e740 1 client.15477 shutdown 2024-04-25T19:02:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f071157e740 1 -- 172.21.15.45:0/381586277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 msgr2=0x563238b105f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f071157e740 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238b105f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f06f803d4a0 tx=0x7f06f80305d0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f071157e740 1 -- 172.21.15.45:0/381586277 shutdown_connections 2024-04-25T19:02:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f071157e740 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563238b18560 0x563238b80e30 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f071157e740 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563238affa00 0x563238b0df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f071157e740 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563238af5140 0x563238b0d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f071157e740 1 --2- 172.21.15.45:0/381586277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563238af4820 0x563238b105f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f071157e740 1 -- 172.21.15.45:0/381586277 >> 172.21.15.45:0/381586277 conn(0x563238af0d30 msgr2=0x563238afd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f06f1600640 0 client.15477 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:02:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f071157e740 1 -- 172.21.15.45:0/381586277 shutdown_connections 2024-04-25T19:02:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.731+0000 7f071157e740 1 -- 172.21.15.45:0/381586277 wait complete. 2024-04-25T19:02:53.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.732+0000 7f071157e740 20 client.15477 trim_cache size 0 max 16384 2024-04-25T19:02:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:02:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:02:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:02:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:02:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.732+0000 7f070e800640 1 -- 172.21.15.45:0/3393210883 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f06ec0a3d10 msgr2=0x7f06ec0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.732+0000 7f070e800640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f06ec0a3d10 0x7f06ec0a61d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06f8031000 tx=0x7f06f802a340 comp rx=0 tx=0).stop 2024-04-25T19:02:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.732+0000 7f070e800640 1 -- 172.21.15.45:0/3393210883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0708109490 msgr2=0x7f070819afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:02:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.732+0000 7f070e800640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0708109490 0x7f070819afe0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f06f4005fd0 tx=0x7f06f400eb60 comp rx=0 tx=0).stop 2024-04-25T19:02:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.733+0000 7f070e800640 1 -- 172.21.15.45:0/3393210883 shutdown_connections 2024-04-25T19:02:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.733+0000 7f070e800640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f06ec0a3d10 0x7f06ec0a61d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.733+0000 7f070e800640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f070819f390 0x7f070819f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.733+0000 7f070e800640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0708109490 0x7f070819afe0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.733+0000 7f070e800640 1 --2- 172.21.15.45:0/3393210883 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07081043c0 0x7f070819aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:02:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.733+0000 7f070e800640 1 -- 172.21.15.45:0/3393210883 >> 172.21.15.45:0/3393210883 conn(0x7f07080ffc20 msgr2=0x7f0708100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:02:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.733+0000 7f070e800640 1 -- 172.21.15.45:0/3393210883 shutdown_connections 2024-04-25T19:02:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:02:53.733+0000 7f070e800640 1 -- 172.21.15.45:0/3393210883 wait complete. 2024-04-25T19:02:53.875 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] growfiles 2024-04-25T19:02:53.894 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] iogen 2024-04-25T19:02:54.434 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-stress 2024-04-25T19:02:54.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:54 smithi062 ceph-mon[32844]: pgmap v110: 97 pgs: 97 active+clean; 34 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.1 MiB/s wr, 324 op/s 2024-04-25T19:02:54.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:54 smithi079 ceph-mon[33002]: pgmap v110: 97 pgs: 97 active+clean; 34 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.1 MiB/s wr, 324 op/s 2024-04-25T19:02:54.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:54 smithi045 ceph-mon[27776]: pgmap v110: 97 pgs: 97 active+clean; 34 MiB data, 593 MiB used, 1.0 TiB / 1.0 TiB avail; 523 KiB/s rd, 1.1 MiB/s wr, 324 op/s 2024-04-25T19:02:56.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:56 smithi062 ceph-mon[32844]: pgmap v111: 97 pgs: 97 active+clean; 26 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.2 MiB/s wr, 473 op/s 2024-04-25T19:02:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:56 smithi079 ceph-mon[33002]: pgmap v111: 97 pgs: 97 active+clean; 26 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.2 MiB/s wr, 473 op/s 2024-04-25T19:02:56.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:56 smithi045 ceph-mon[27776]: pgmap v111: 97 pgs: 97 active+clean; 26 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.2 MiB/s wr, 473 op/s 2024-04-25T19:02:56.841 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-04-25T19:02:56.845 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-04-25T19:02:56.845 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-04-25T19:02:56.847 INFO:tasks.workunit.client.0.smithi045.stdout: [DEP] 2024-04-25T19:02:57.428 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] dirstress 2024-04-25T19:02:57.430 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fill 2024-04-25T19:02:57.434 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fill2 2024-04-25T19:02:57.434 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] getpagesize 2024-04-25T19:02:57.726 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] holes 2024-04-25T19:02:57.742 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] lstat64 2024-04-25T19:02:57.837 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] nametest 2024-04-25T19:02:57.871 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] permname 2024-04-25T19:02:58.084 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] randholes 2024-04-25T19:02:58.098 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] runas 2024-04-25T19:02:58.257 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] truncfile 2024-04-25T19:02:58.259 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] usemem 2024-04-25T19:02:58.419 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] mmapcat 2024-04-25T19:02:58.504 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] append_reader 2024-04-25T19:02:58.541 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] append_writer 2024-04-25T19:02:58.601 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] dirperf 2024-04-25T19:02:58.629 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:02:58 smithi062 ceph-mon[32844]: pgmap v112: 97 pgs: 97 active+clean; 26 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 243 KiB/s wr, 176 op/s 2024-04-25T19:02:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:02:58 smithi079 ceph-mon[33002]: pgmap v112: 97 pgs: 97 active+clean; 26 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 243 KiB/s wr, 176 op/s 2024-04-25T19:02:58.709 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] metaperf 2024-04-25T19:02:58.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:02:58 smithi045 ceph-mon[27776]: pgmap v112: 97 pgs: 97 active+clean; 26 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 243 KiB/s wr, 176 op/s 2024-04-25T19:02:58.799 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] devzero 2024-04-25T19:02:58.824 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] feature 2024-04-25T19:02:58.984 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] alloc 2024-04-25T19:02:59.107 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fault 2024-04-25T19:02:59.167 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fstest 2024-04-25T19:02:59.210 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_access_root 2024-04-25T19:02:59.364 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] godown 2024-04-25T19:02:59.411 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] resvtest 2024-04-25T19:02:59.502 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] writemod 2024-04-25T19:02:59.574 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] writev_on_pagefault 2024-04-25T19:02:59.679 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] makeextents 2024-04-25T19:02:59.729 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] itrash 2024-04-25T19:02:59.790 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] rename 2024-04-25T19:02:59.885 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] multi_open_unlink 2024-04-25T19:03:00.010 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] unwritten_sync 2024-04-25T19:03:00.034 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] genhashnames 2024-04-25T19:03:00.090 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_holes 2024-04-25T19:03:00.190 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_mmap_writev 2024-04-25T19:03:00.347 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_truncate_cmtime 2024-04-25T19:03:00.360 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] dirhash_collide 2024-04-25T19:03:00.362 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_rename_overwrite 2024-04-25T19:03:00.510 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] holetest 2024-04-25T19:03:00.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:00 smithi062 ceph-mon[32844]: pgmap v113: 97 pgs: 97 active+clean; 26 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 243 KiB/s wr, 176 op/s 2024-04-25T19:03:00.670 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_truncate_self 2024-04-25T19:03:00.672 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] af_unix 2024-04-25T19:03:00.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:00 smithi079 ceph-mon[33002]: pgmap v113: 97 pgs: 97 active+clean; 26 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 243 KiB/s wr, 176 op/s 2024-04-25T19:03:00.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:00 smithi045 ceph-mon[27776]: pgmap v113: 97 pgs: 97 active+clean; 26 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 243 KiB/s wr, 176 op/s 2024-04-25T19:03:00.716 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_mmap_stale_pmd 2024-04-25T19:03:00.893 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_mmap_cow_race 2024-04-25T19:03:00.945 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_mmap_fallocate 2024-04-25T19:03:00.960 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fsync-err 2024-04-25T19:03:01.013 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_mmap_write_ro 2024-04-25T19:03:01.205 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-25T19:03:01.230 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-25T19:03:01.288 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_ofd_locks 2024-04-25T19:03:01.294 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_mmap_collision 2024-04-25T19:03:01.509 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] mmap-write-concurrent 2024-04-25T19:03:01.534 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_get_file_time 2024-04-25T19:03:01.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:01 smithi062 ceph-mon[32844]: pgmap v114: 97 pgs: 97 active+clean; 27 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 251 KiB/s wr, 188 op/s 2024-04-25T19:03:01.623 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_create_short_dirs 2024-04-25T19:03:01.653 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_create_long_dirs 2024-04-25T19:03:01.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:01 smithi079 ceph-mon[33002]: pgmap v114: 97 pgs: 97 active+clean; 27 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 251 KiB/s wr, 188 op/s 2024-04-25T19:03:01.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:01 smithi045 ceph-mon[27776]: pgmap v114: 97 pgs: 97 active+clean; 27 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 251 KiB/s wr, 188 op/s 2024-04-25T19:03:01.809 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_enospc 2024-04-25T19:03:01.813 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_mmap_writev_overlap 2024-04-25T19:03:01.935 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] checkpoint_journal 2024-04-25T19:03:01.960 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] mmap-rw-fault 2024-04-25T19:03:02.148 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] allocstale 2024-04-25T19:03:02.167 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_mmap_cow_memory_failure 2024-04-25T19:03:02.217 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fake-dump-rootino 2024-04-25T19:03:02.314 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] dio-buf-fault 2024-04-25T19:03:02.458 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] rewinddir-test 2024-04-25T19:03:02.503 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] readdir-while-renames 2024-04-25T19:03:02.505 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] xfsctl 2024-04-25T19:03:02.622 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] bstat 2024-04-25T19:03:02.759 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_mtab 2024-04-25T19:03:02.813 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] getdevicesize 2024-04-25T19:03:02.844 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] preallo_rw_pattern_reader 2024-04-25T19:03:03.034 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] preallo_rw_pattern_writer 2024-04-25T19:03:03.093 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] ftrunc 2024-04-25T19:03:03.105 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] trunc 2024-04-25T19:03:03.126 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fs_perms 2024-04-25T19:03:03.338 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] testx 2024-04-25T19:03:03.375 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] looptest 2024-04-25T19:03:03.401 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] locktest 2024-04-25T19:03:03.430 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] unwritten_mmap 2024-04-25T19:03:03.613 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] bulkstat_unlink_test 2024-04-25T19:03:03.712 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] deduperace 2024-04-25T19:03:03.730 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] bulkstat_unlink_test_modified 2024-04-25T19:03:03.875 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_dir_offset 2024-04-25T19:03:03.965 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_futimens 2024-04-25T19:03:04.062 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_immutable 2024-04-25T19:03:04.067 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] stale_handle 2024-04-25T19:03:04.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:03 smithi079 ceph-mon[33002]: pgmap v115: 97 pgs: 97 active+clean; 27 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 123 KiB/s wr, 160 op/s 2024-04-25T19:03:04.201 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] pwrite_mmap_blocked 2024-04-25T19:03:04.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:03 smithi045 ceph-mon[27776]: pgmap v115: 97 pgs: 97 active+clean; 27 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 123 KiB/s wr, 160 op/s 2024-04-25T19:03:04.306 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_dir_offset2 2024-04-25T19:03:04.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:03 smithi062 ceph-mon[32844]: pgmap v115: 97 pgs: 97 active+clean; 27 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 123 KiB/s wr, 160 op/s 2024-04-25T19:03:04.442 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] seek_sanity_test 2024-04-25T19:03:04.519 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] seek_copy_test 2024-04-25T19:03:04.702 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_readdir_1 2024-04-25T19:03:04.869 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_readdir_2 2024-04-25T19:03:04.993 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fsync-tester 2024-04-25T19:03:05.033 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] nsexec 2024-04-25T19:03:05.128 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] cloner 2024-04-25T19:03:05.163 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] renameat2 2024-04-25T19:03:05.340 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_getcwd 2024-04-25T19:03:05.405 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] e4compact 2024-04-25T19:03:05.450 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] test-nextquota 2024-04-25T19:03:05.476 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] punch-alternating 2024-04-25T19:03:05.649 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-25T19:03:05.753 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] listxattr 2024-04-25T19:03:05.774 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] dio-interleaved 2024-04-25T19:03:05.787 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_dir_type 2024-04-25T19:03:05.973 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] dio-invalidate-cache 2024-04-25T19:03:06.042 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] stat_test 2024-04-25T19:03:06.057 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_encrypted_d_revalidate 2024-04-25T19:03:06.084 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] attr_replace_test 2024-04-25T19:03:06.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:05 smithi079 ceph-mon[33002]: pgmap v116: 97 pgs: 97 active+clean; 32 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 663 KiB/s wr, 210 op/s 2024-04-25T19:03:06.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:05 smithi045 ceph-mon[27776]: pgmap v116: 97 pgs: 97 active+clean; 32 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 663 KiB/s wr, 210 op/s 2024-04-25T19:03:06.344 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] swapon 2024-04-25T19:03:06.344 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] mkswap 2024-04-25T19:03:06.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:05 smithi062 ceph-mon[32844]: pgmap v116: 97 pgs: 97 active+clean; 32 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 663 KiB/s wr, 210 op/s 2024-04-25T19:03:06.408 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_attr_corruption 2024-04-25T19:03:06.501 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_open_tmpfiles 2024-04-25T19:03:06.648 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fscrypt-crypt-util 2024-04-25T19:03:06.657 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] bulkstat_null_ocount 2024-04-25T19:03:06.692 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] splice-test 2024-04-25T19:03:06.851 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] chprojid_fail 2024-04-25T19:03:06.888 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.886+0000 7fdc4fe00640 1 -- 172.21.15.45:0/537182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50101fd0 msgr2=0x7fdc501023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.886+0000 7fdc4fe00640 1 --2- 172.21.15.45:0/537182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50101fd0 0x7fdc501023b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdc40009900 tx=0x7fdc4002a3f0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.887+0000 7fdc4fe00640 1 -- 172.21.15.45:0/537182702 shutdown_connections 2024-04-25T19:03:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.887+0000 7fdc4fe00640 1 --2- 172.21.15.45:0/537182702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc501070a0 0x7fdc5010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.887+0000 7fdc4fe00640 1 --2- 172.21.15.45:0/537182702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc50102980 0x7fdc501069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.887+0000 7fdc4fe00640 1 --2- 172.21.15.45:0/537182702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50101fd0 0x7fdc501023b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.887+0000 7fdc4fe00640 1 -- 172.21.15.45:0/537182702 >> 172.21.15.45:0/537182702 conn(0x7fdc500fd890 msgr2=0x7fdc500ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.888+0000 7fdc4fe00640 1 -- 172.21.15.45:0/537182702 shutdown_connections 2024-04-25T19:03:06.890 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.888+0000 7fdc4fe00640 1 -- 172.21.15.45:0/537182702 wait complete. 2024-04-25T19:03:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.889+0000 7fdc4fe00640 1 Processor -- start 2024-04-25T19:03:06.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.889+0000 7fdc4fe00640 1 -- start start 2024-04-25T19:03:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.890+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50101fd0 0x7fdc501985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.890+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc501070a0 0x7fdc50198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.890+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5019ce50 0x7fdc5019d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.890+0000 7fdc4fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc50112060 con 0x7fdc501070a0 2024-04-25T19:03:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.890+0000 7fdc4fe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc50111ee0 con 0x7fdc50101fd0 2024-04-25T19:03:06.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.890+0000 7fdc4fe00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc501121e0 con 0x7fdc5019ce50 2024-04-25T19:03:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.890+0000 7fdc4ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50101fd0 0x7fdc501985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4ea00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50101fd0 0x7fdc501985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41972/0 (socket says 172.21.15.45:41972) 2024-04-25T19:03:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4ea00640 1 -- 172.21.15.45:0/2241616479 learned_addr learned my addr 172.21.15.45:0/2241616479 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4f400640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5019ce50 0x7fdc5019d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4e000640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc501070a0 0x7fdc50198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4ea00640 1 -- 172.21.15.45:0/2241616479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5019ce50 msgr2=0x7fdc5019d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5019ce50 0x7fdc5019d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4ea00640 1 -- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc501070a0 msgr2=0x7fdc50198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc501070a0 0x7fdc50198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4ea00640 1 -- 172.21.15.45:0/2241616479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5019d940 con 0x7fdc50101fd0 2024-04-25T19:03:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4e000640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc501070a0 0x7fdc50198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.891+0000 7fdc4f400640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5019ce50 0x7fdc5019d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.892+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50101fd0 0x7fdc501985e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdc40005ab0 tx=0x7fdc40003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.892+0000 7fdc47400640 1 -- 172.21.15.45:0/2241616479 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4003f070 con 0x7fdc50101fd0 2024-04-25T19:03:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.892+0000 7fdc47400640 1 -- 172.21.15.45:0/2241616479 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc40031b60 con 0x7fdc50101fd0 2024-04-25T19:03:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.892+0000 7fdc4fe00640 1 -- 172.21.15.45:0/2241616479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5019dbd0 con 0x7fdc50101fd0 2024-04-25T19:03:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.892+0000 7fdc47400640 1 -- 172.21.15.45:0/2241616479 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40035030 con 0x7fdc50101fd0 2024-04-25T19:03:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.892+0000 7fdc4fe00640 1 -- 172.21.15.45:0/2241616479 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc501a53f0 con 0x7fdc50101fd0 2024-04-25T19:03:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.897+0000 7fdc47400640 1 -- 172.21.15.45:0/2241616479 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdc40042070 con 0x7fdc50101fd0 2024-04-25T19:03:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.898+0000 7fdc58a7f740 1 Processor -- start 2024-04-25T19:03:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.898+0000 7fdc58a7f740 1 -- start start 2024-04-25T19:03:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.898+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca731a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.898+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.898+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca80390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.898+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ba4c9d4900 con 0x55ba4ca736e0 2024-04-25T19:03:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.898+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ba4c97aef0 con 0x55ba4ca72dc0 2024-04-25T19:03:06.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.898+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ba4c9d08c0 con 0x55ba4ca7dfa0 2024-04-25T19:03:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.898+0000 7fdc4f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca731a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.899+0000 7fdc4f400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca731a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41984/0 (socket says 172.21.15.45:41984) 2024-04-25T19:03:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.899+0000 7fdc4ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.899+0000 7fdc4f400640 1 -- 172.21.15.45:0/1401329222 learned_addr learned my addr 172.21.15.45:0/1401329222 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.899+0000 7fdc4e000640 1 --2- 172.21.15.45:0/1401329222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca80390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.899+0000 7fdc47400640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdc340a3d80 0x7fdc340a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.899+0000 7fdc4ea00640 1 -- 172.21.15.45:0/1401329222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 msgr2=0x55ba4ca80390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.899+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1401329222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca80390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.899+0000 7fdc4ea00640 1 -- 172.21.15.45:0/1401329222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 msgr2=0x55ba4ca731a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.899+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1401329222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca731a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.899+0000 7fdc4ea00640 1 -- 172.21.15.45:0/1401329222 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba4c6e7210 con 0x55ba4ca736e0 2024-04-25T19:03:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.900+0000 7fdc47400640 1 -- 172.21.15.45:0/2241616479 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fdc400f0b30 con 0x7fdc50101fd0 2024-04-25T19:03:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.900+0000 7fdc4f400640 1 --2- 172.21.15.45:0/1401329222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.900+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1401329222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7da60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdc400ee220 tx=0x7fdc4007d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5185a5607443899d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.900+0000 7fdc4f400640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdc340a3d80 0x7fdc340a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.900+0000 7fdc3be00640 1 -- 172.21.15.45:0/1401329222 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40049070 con 0x55ba4ca736e0 2024-04-25T19:03:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.900+0000 7fdc3be00640 1 -- 172.21.15.45:0/1401329222 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc400f4110 con 0x55ba4ca736e0 2024-04-25T19:03:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.900+0000 7fdc3be00640 1 -- 172.21.15.45:0/1401329222 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc400bcab0 con 0x55ba4ca736e0 2024-04-25T19:03:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.901+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1401329222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca736e0 msgr2=0x55ba4ca7da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.901+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1401329222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7da60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdc400ee220 tx=0x7fdc4007d5e0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.901+0000 7fdc4f400640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdc340a3d80 0x7fdc340a6240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009fd0 tx=0x7fdc48004a40 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.901+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1401329222 shutdown_connections 2024-04-25T19:03:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.901+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1401329222 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca80390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.901+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1401329222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7da60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.901+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1401329222 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.901+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1401329222 >> 172.21.15.45:0/1401329222 conn(0x55ba4ca6f2d0 msgr2=0x55ba4ca6f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.902+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1401329222 shutdown_connections 2024-04-25T19:03:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.902+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1401329222 wait complete. 2024-04-25T19:03:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.902+0000 7fdc58a7f740 1 -- start start 2024-04-25T19:03:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.903+0000 7fdc58a7f740 1 Processor -- start 2024-04-25T19:03:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.903+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca7ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.903+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.903+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca7b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.903+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ba4ca80990 con 0x55ba4ca72dc0 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.903+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ba4c98e310 con 0x55ba4ca736e0 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.903+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ba4ca80b10 con 0x55ba4ca7dfa0 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.903+0000 7fdc58a7f740 10 client.0 _ll_register_callbacks cb 0x55ba4c95ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.904+0000 7fdc4e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.904+0000 7fdc4e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41992/0 (socket says 172.21.15.45:41992) 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.904+0000 7fdc4e000640 1 -- 172.21.15.45:0/3773114410 learned_addr learned my addr 172.21.15.45:0/3773114410 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.904+0000 7fdc4f400640 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca7b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.904+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca7ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.904+0000 7fdc4e000640 1 -- 172.21.15.45:0/3773114410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 msgr2=0x55ba4ca7b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.904+0000 7fdc4e000640 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca7b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.904+0000 7fdc4e000640 1 -- 172.21.15.45:0/3773114410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 msgr2=0x55ba4ca7ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.904+0000 7fdc4e000640 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca7ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.904+0000 7fdc4e000640 1 -- 172.21.15.45:0/3773114410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba4c7b3620 con 0x55ba4ca736e0 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.905+0000 7fdc4e000640 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7b360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c012700 tx=0x7fdc3c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed17945d99eb128 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.905+0000 7fdc38c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.905+0000 7fdc38c00640 1 -- 172.21.15.45:0/3773114410 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c011bb0 con 0x55ba4ca736e0 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.905+0000 7fdc38c00640 1 -- 172.21.15.45:0/3773114410 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fdc3c011d50 con 0x55ba4ca736e0 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.905+0000 7fdc58a7f740 1 -- 172.21.15.45:0/3773114410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ba4c9e88b0 con 0x55ba4ca736e0 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.905+0000 7fdc38c00640 1 -- 172.21.15.45:0/3773114410 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc3c004020 con 0x55ba4ca736e0 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.905+0000 7fdc58a7f740 10 client.25616 fetch_fsmap learned FSMap version 28 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.905+0000 7fdc58a7f740 1 -- 172.21.15.45:0/3773114410 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba4ca9db30 con 0x55ba4ca736e0 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.906+0000 7fdc38c00640 1 -- 172.21.15.45:0/3773114410 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ba4c9e88b0 con 0x55ba4ca736e0 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.906+0000 7fdc38c00640 1 -- 172.21.15.45:0/3773114410 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fdc3c01c470 con 0x55ba4ca736e0 2024-04-25T19:03:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.906+0000 7fdc58a7f740 10 client.25616 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:03:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.906+0000 7fdc58a7f740 10 client.25616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.906+0000 7fdc58a7f740 20 client.25616 populate_metadata read hostname 'smithi045' 2024-04-25T19:03:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.906+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ba4ca9f0b0 0x55ba4caa14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.906+0000 7fdc58a7f740 4 client.25616 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.906+0000 7fdc58a7f740 1 -- 172.21.15.45:0/3773114410 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba4c9e2320 con 0x55ba4ca9f0b0 2024-04-25T19:03:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.907+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ba4ca9f0b0 0x55ba4caa14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.908+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ba4ca9f0b0 0x55ba4caa14e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c608f69bd4d21d0 server_cookie=b8ea81b0f429db08 in_seq=0 out_seq=0 2024-04-25T19:03:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.908+0000 7fdc38c00640 10 client.25616 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc38c00640 1 -- 172.21.15.45:0/3773114410 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba4c9e2320 con 0x55ba4ca9f0b0 2024-04-25T19:03:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc38c00640 10 client.25616 handle_command_reply: tid=0 2024-04-25T19:03:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc58a7f740 1 client.25616 shutdown 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc58a7f740 1 -- 172.21.15.45:0/3773114410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 msgr2=0x55ba4ca7b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7b360 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c012700 tx=0x7fdc3c012bc0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc58a7f740 1 -- 172.21.15.45:0/3773114410 shutdown_connections 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ba4ca9f0b0 0x55ba4caa14e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca7b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca7b360 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/3773114410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca7ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.909+0000 7fdc58a7f740 1 -- 172.21.15.45:0/3773114410 >> 172.21.15.45:0/3773114410 conn(0x55ba4ca6f2d0 msgr2=0x55ba4ca6a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.910+0000 7fdc38c00640 0 client.25616 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.910+0000 7fdc58a7f740 1 -- 172.21.15.45:0/3773114410 shutdown_connections 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.910+0000 7fdc58a7f740 1 -- 172.21.15.45:0/3773114410 wait complete. 2024-04-25T19:03:06.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.910+0000 7fdc58a7f740 20 client.25616 trim_cache size 0 max 16384 2024-04-25T19:03:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.936+0000 7fdc58a7f740 1 Processor -- start 2024-04-25T19:03:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc58a7f740 1 -- start start 2024-04-25T19:03:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca89730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca736e0 0x55ba4ca79fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca7cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc30011090 con 0x55ba4ca7dfa0 2024-04-25T19:03:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3000e960 con 0x55ba4ca72dc0 2024-04-25T19:03:06.939 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3000c7b0 con 0x55ba4ca736e0 2024-04-25T19:03:06.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc4e000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca736e0 0x55ba4ca79fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc4f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca7cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc4e000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca736e0 0x55ba4ca79fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:42030/0 (socket says 172.21.15.45:42030) 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc4e000640 1 -- 172.21.15.45:0/1960954875 learned_addr learned my addr 172.21.15.45:0/1960954875 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.937+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1960954875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca89730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.938+0000 7fdc4ea00640 1 -- 172.21.15.45:0/1960954875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca736e0 msgr2=0x55ba4ca79fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.938+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1960954875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca736e0 0x55ba4ca79fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.938+0000 7fdc4ea00640 1 -- 172.21.15.45:0/1960954875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca7dfa0 msgr2=0x55ba4ca7cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.938+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1960954875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca7cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.938+0000 7fdc4ea00640 1 -- 172.21.15.45:0/1960954875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba4ca73510 con 0x55ba4ca72dc0 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.938+0000 7fdc4e000640 1 --2- 172.21.15.45:0/1960954875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca736e0 0x55ba4ca79fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.938+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1960954875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca89730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdc400f0e20 tx=0x7fdc4007d5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c3dd2bb4cb641ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.939+0000 7fdc3b400640 1 -- 172.21.15.45:0/1960954875 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40049040 con 0x55ba4ca72dc0 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.939+0000 7fdc3b400640 1 -- 172.21.15.45:0/1960954875 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc400f6030 con 0x55ba4ca72dc0 2024-04-25T19:03:06.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.939+0000 7fdc3b400640 1 -- 172.21.15.45:0/1960954875 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc400c56e0 con 0x55ba4ca72dc0 2024-04-25T19:03:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.940+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1960954875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 msgr2=0x55ba4ca89730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.940+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1960954875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca89730 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fdc400f0e20 tx=0x7fdc4007d5e0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.940+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1960954875 shutdown_connections 2024-04-25T19:03:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.940+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1960954875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca7cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.940+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1960954875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca736e0 0x55ba4ca79fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.940+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1960954875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca89730 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.940+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1960954875 >> 172.21.15.45:0/1960954875 conn(0x55ba4ca6f2d0 msgr2=0x55ba4ca7c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.940+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1960954875 shutdown_connections 2024-04-25T19:03:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.940+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1960954875 wait complete. 2024-04-25T19:03:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.941+0000 7fdc58a7f740 1 -- start start 2024-04-25T19:03:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.941+0000 7fdc58a7f740 1 Processor -- start 2024-04-25T19:03:06.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca859b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca85ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc58a7f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca82ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3000ce20 con 0x55ba4ca72dc0 2024-04-25T19:03:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3000e290 con 0x55ba4ca736e0 2024-04-25T19:03:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc58a7f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc3000d4f0 con 0x55ba4ca7dfa0 2024-04-25T19:03:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc58a7f740 10 client.0 _ll_register_callbacks cb 0x7fdc3c009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc4e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca85ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc4e000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca85ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:42014/0 (socket says 172.21.15.45:42014) 2024-04-25T19:03:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc4e000640 1 -- 172.21.15.45:0/1252489795 learned_addr learned my addr 172.21.15.45:0/1252489795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.945 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.942+0000 7fdc4f400640 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca82ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.943+0000 7fdc4ea00640 1 -- 172.21.15.45:0/1252489795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 msgr2=0x55ba4ca82ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.943+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca82ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.943+0000 7fdc4ea00640 1 -- 172.21.15.45:0/1252489795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 msgr2=0x55ba4ca85ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.943+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca85ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.943+0000 7fdc4ea00640 1 -- 172.21.15.45:0/1252489795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba4ca837f0 con 0x55ba4ca72dc0 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.943+0000 7fdc4ea00640 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca859b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc400b8fb0 tx=0x7fdc4002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46579ad929cc17a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.943+0000 7fdc38c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.943+0000 7fdc38c00640 1 -- 172.21.15.45:0/1252489795 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40049040 con 0x55ba4ca72dc0 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.943+0000 7fdc38c00640 1 -- 172.21.15.45:0/1252489795 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fdc400d1150 con 0x55ba4ca72dc0 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.944+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1252489795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc400f6030 con 0x55ba4ca72dc0 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.944+0000 7fdc38c00640 1 -- 172.21.15.45:0/1252489795 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fdc400dd0f0 con 0x55ba4ca72dc0 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.944+0000 7fdc58a7f740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-04-25T19:03:06.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.944+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1252489795 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ba4ca837f0 con 0x55ba4ca72dc0 2024-04-25T19:03:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.944+0000 7fdc38c00640 1 -- 172.21.15.45:0/1252489795 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc400f6030 con 0x55ba4ca72dc0 2024-04-25T19:03:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.945+0000 7fdc38c00640 1 -- 172.21.15.45:0/1252489795 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7fdc4004c060 con 0x55ba4ca72dc0 2024-04-25T19:03:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.945+0000 7fdc58a7f740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:03:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.945+0000 7fdc58a7f740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.945+0000 7fdc58a7f740 20 client.15492 populate_metadata read hostname 'smithi045' 2024-04-25T19:03:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.945+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ba4cb05230 0x55ba4cb07620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.945+0000 7fdc58a7f740 4 client.15492 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:03:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.945+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1252489795 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba4c9dc190 con 0x55ba4cb05230 2024-04-25T19:03:06.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.945+0000 7fdc4e000640 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ba4cb05230 0x55ba4cb07620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.946+0000 7fdc4e000640 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ba4cb05230 0x55ba4cb07620 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e787c0bb64e16854 server_cookie=3209c8ae78860491 in_seq=0 out_seq=0 2024-04-25T19:03:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:06.946+0000 7fdc38c00640 10 client.15492 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:06.981 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] detached_mounts_propagation 2024-04-25T19:03:07.056 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] ext4_resize 2024-04-25T19:03:07.169 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_readdir_3 2024-04-25T19:03:07.315 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] splice2pipe 2024-04-25T19:03:07.349 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] uuid_ioctl 2024-04-25T19:03:07.523 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-25T19:03:07.579 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fiemap-fault 2024-04-25T19:03:07.618 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fiemap-tester 2024-04-25T19:03:07.673 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_stripealign 2024-04-25T19:03:07.843 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] open_by_handle 2024-04-25T19:03:07.874 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_mmap_dio 2024-04-25T19:03:07.978 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] xfsfind 2024-04-25T19:03:08.077 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] t_reflink_read_race 2024-04-25T19:03:08.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:07 smithi079 ceph-mon[33002]: pgmap v117: 97 pgs: 97 active+clean; 32 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 548 KiB/s wr, 61 op/s 2024-04-25T19:03:08.187 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] fssum 2024-04-25T19:03:08.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:07 smithi045 ceph-mon[27776]: pgmap v117: 97 pgs: 97 active+clean; 32 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 548 KiB/s wr, 61 op/s 2024-04-25T19:03:08.261 INFO:tasks.workunit.client.0.smithi045.stdout:Building log-writes 2024-04-25T19:03:08.279 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] replay-log 2024-04-25T19:03:08.310 INFO:tasks.workunit.client.0.smithi045.stdout:Building perf 2024-04-25T19:03:08.322 INFO:tasks.workunit.client.0.smithi045.stdout:Building aio-dio-regress 2024-04-25T19:03:08.337 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aiocp 2024-04-25T19:03:08.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:07 smithi062 ceph-mon[32844]: pgmap v117: 97 pgs: 97 active+clean; 32 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 548 KiB/s wr, 61 op/s 2024-04-25T19:03:08.461 INFO:tasks.workunit.client.0.smithi045.stdout:Building vfs 2024-04-25T19:03:08.473 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] vfstest 2024-04-25T19:03:08.713 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-25T19:03:08.781 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-append-write-read-race 2024-04-25T19:03:08.811 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] mount-idmapped 2024-04-25T19:03:09.022 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-cow-race 2024-04-25T19:03:09.120 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-cycle-write 2024-04-25T19:03:09.307 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-eof-race 2024-04-25T19:03:09.437 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-extend-stat 2024-04-25T19:03:09.454 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-fcntl-race 2024-04-25T19:03:09.652 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-hole-filling-race 2024-04-25T19:03:09.731 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-invalidate-failure 2024-04-25T19:03:09.774 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-invalidate-readahead 2024-04-25T19:03:09.951 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aiodio_sparse2 2024-04-25T19:03:10.038 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-subblock-eof-read 2024-04-25T19:03:10.084 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-dio-write-verify 2024-04-25T19:03:10.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:09 smithi079 ceph-mon[33002]: pgmap v118: 97 pgs: 97 active+clean; 32 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 548 KiB/s wr, 61 op/s 2024-04-25T19:03:10.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:09 smithi045 ceph-mon[27776]: pgmap v118: 97 pgs: 97 active+clean; 32 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 548 KiB/s wr, 61 op/s 2024-04-25T19:03:10.318 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-25T19:03:10.334 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-25T19:03:10.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:09 smithi062 ceph-mon[32844]: pgmap v118: 97 pgs: 97 active+clean; 32 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 548 KiB/s wr, 61 op/s 2024-04-25T19:03:10.415 INFO:tasks.workunit.client.0.smithi045.stdout: [CC] aio-last-ref-held-by-io 2024-04-25T19:03:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc38c00640 1 -- 172.21.15.45:0/1252489795 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba4c9dc190 con 0x55ba4cb05230 2024-04-25T19:03:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc38c00640 10 client.15492 handle_command_reply: tid=0 2024-04-25T19:03:10.510 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc58a7f740 1 client.15492 shutdown 2024-04-25T19:03:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1252489795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 msgr2=0x55ba4ca859b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca859b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc400b8fb0 tx=0x7fdc4002a3f0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1252489795 shutdown_connections 2024-04-25T19:03:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ba4cb05230 0x55ba4cb07620 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ba4ca7dfa0 0x55ba4ca82ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ba4ca736e0 0x55ba4ca85ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc58a7f740 1 --2- 172.21.15.45:0/1252489795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ba4ca72dc0 0x55ba4ca859b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1252489795 >> 172.21.15.45:0/1252489795 conn(0x55ba4ca6f2d0 msgr2=0x55ba4ca79e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.508+0000 7fdc38c00640 0 client.15492 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.509+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1252489795 shutdown_connections 2024-04-25T19:03:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.509+0000 7fdc58a7f740 1 -- 172.21.15.45:0/1252489795 wait complete. 2024-04-25T19:03:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.509+0000 7fdc58a7f740 20 client.15492 trim_cache size 0 max 16384 2024-04-25T19:03:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:03:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:03:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:03:10.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:03:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 -- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdc340a3d80 msgr2=0x7fdc340a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdc340a3d80 0x7fdc340a6240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc48009fd0 tx=0x7fdc48004a40 comp rx=0 tx=0).stop 2024-04-25T19:03:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 -- 172.21.15.45:0/2241616479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50101fd0 msgr2=0x7fdc501985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50101fd0 0x7fdc501985e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fdc40005ab0 tx=0x7fdc40003c10 comp rx=0 tx=0).stop 2024-04-25T19:03:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 -- 172.21.15.45:0/2241616479 shutdown_connections 2024-04-25T19:03:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdc340a3d80 0x7fdc340a6240 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5019ce50 0x7fdc5019d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdc501070a0 0x7fdc50198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 --2- 172.21.15.45:0/2241616479 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdc50101fd0 0x7fdc501985e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:10.513 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 -- 172.21.15.45:0/2241616479 >> 172.21.15.45:0/2241616479 conn(0x7fdc500fd890 msgr2=0x7fdc500fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 -- 172.21.15.45:0/2241616479 shutdown_connections 2024-04-25T19:03:10.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:10.510+0000 7fdc4fe00640 1 -- 172.21.15.45:0/2241616479 wait complete. 2024-04-25T19:03:11.143 INFO:tasks.workunit.client.0.smithi045.stderr:+ cd .. 2024-04-25T19:03:11.143 INFO:tasks.workunit.client.0.smithi045.stderr:+ cp xfstests-dev/ltp/fsx . 2024-04-25T19:03:11.152 INFO:tasks.workunit.client.0.smithi045.stderr:+ OPTIONS=-z 2024-04-25T19:03:11.152 INFO:tasks.workunit.client.0.smithi045.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-04-25T19:03:11.154 INFO:tasks.workunit.client.0.smithi045.stdout:Seed set to 1 2024-04-25T19:03:11.190 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T19:03:11.190 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T19:03:11.193 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T19:03:11.193 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T19:03:11.193 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T19:03:11.197 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T19:03:11.199 INFO:tasks.workunit.client.0.smithi045.stdout:skipping zero size read 2024-04-25T19:03:11.199 INFO:tasks.workunit.client.0.smithi045.stdout:skipping zero size read 2024-04-25T19:03:11.199 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T19:03:11.199 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0x81d54 2024-04-25T19:03:11.204 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0xb86d7 2024-04-25T19:03:11.214 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0xe627e 2024-04-25T19:03:11.259 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0xf3194 2024-04-25T19:03:11.284 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0xf3ceb 2024-04-25T19:03:11.294 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0xf47e4 2024-04-25T19:03:11.414 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0xfaccc 2024-04-25T19:03:11.949 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0xfe423 2024-04-25T19:03:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:11 smithi079 ceph-mon[33002]: pgmap v119: 97 pgs: 97 active+clean; 32 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 568 KiB/s wr, 66 op/s 2024-04-25T19:03:12.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:11 smithi045 ceph-mon[27776]: pgmap v119: 97 pgs: 97 active+clean; 32 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 568 KiB/s wr, 66 op/s 2024-04-25T19:03:12.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:11 smithi062 ceph-mon[32844]: pgmap v119: 97 pgs: 97 active+clean; 32 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 568 KiB/s wr, 66 op/s 2024-04-25T19:03:14.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:13 smithi079 ceph-mon[33002]: pgmap v120: 97 pgs: 97 active+clean; 32 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 560 KiB/s wr, 55 op/s 2024-04-25T19:03:14.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:13 smithi045 ceph-mon[27776]: pgmap v120: 97 pgs: 97 active+clean; 32 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 560 KiB/s wr, 55 op/s 2024-04-25T19:03:14.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:13 smithi062 ceph-mon[32844]: pgmap v120: 97 pgs: 97 active+clean; 32 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 560 KiB/s wr, 55 op/s 2024-04-25T19:03:14.913 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0xfe492 2024-04-25T19:03:16.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:15 smithi045 ceph-mon[27776]: pgmap v121: 97 pgs: 97 active+clean; 35 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 281 op/s 2024-04-25T19:03:16.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:15 smithi062 ceph-mon[32844]: pgmap v121: 97 pgs: 97 active+clean; 35 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 281 op/s 2024-04-25T19:03:16.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:15 smithi079 ceph-mon[33002]: pgmap v121: 97 pgs: 97 active+clean; 35 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 281 op/s 2024-04-25T19:03:17.450 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0xff038 2024-04-25T19:03:18.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:17 smithi045 ceph-mon[27776]: pgmap v122: 97 pgs: 97 active+clean; 35 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-25T19:03:18.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:17 smithi062 ceph-mon[32844]: pgmap v122: 97 pgs: 97 active+clean; 35 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-25T19:03:18.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:17 smithi079 ceph-mon[33002]: pgmap v122: 97 pgs: 97 active+clean; 35 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-25T19:03:18.572 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0xffcac 2024-04-25T19:03:18.721 INFO:tasks.workunit.client.0.smithi045.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-04-25T19:03:20.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:19 smithi045 ceph-mon[27776]: pgmap v123: 97 pgs: 97 active+clean; 35 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-25T19:03:20.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:19 smithi062 ceph-mon[32844]: pgmap v123: 97 pgs: 97 active+clean; 35 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-25T19:03:20.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:19 smithi079 ceph-mon[33002]: pgmap v123: 97 pgs: 97 active+clean; 35 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 231 op/s 2024-04-25T19:03:22.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:21 smithi045 ceph-mon[27776]: pgmap v124: 97 pgs: 97 active+clean; 39 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 480 op/s 2024-04-25T19:03:22.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:21 smithi062 ceph-mon[32844]: pgmap v124: 97 pgs: 97 active+clean; 39 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 480 op/s 2024-04-25T19:03:22.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:21 smithi079 ceph-mon[33002]: pgmap v124: 97 pgs: 97 active+clean; 39 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.0 MiB/s wr, 480 op/s 2024-04-25T19:03:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.668+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3666540947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ba4102f30 msgr2=0x7f2ba4103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.668+0000 7f2ba9e00640 1 --2- 172.21.15.45:0/3666540947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ba4102f30 0x7f2ba4103310 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009900 tx=0x7f2b9402a480 comp rx=0 tx=0).stop 2024-04-25T19:03:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.668+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3666540947 shutdown_connections 2024-04-25T19:03:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.668+0000 7f2ba9e00640 1 --2- 172.21.15.45:0/3666540947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba410c350 0x7f2ba410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.668+0000 7f2ba9e00640 1 --2- 172.21.15.45:0/3666540947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba4103850 0x7f2ba410bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.668+0000 7f2ba9e00640 1 --2- 172.21.15.45:0/3666540947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ba4102f30 0x7f2ba4103310 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.668+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3666540947 >> 172.21.15.45:0/3666540947 conn(0x7f2ba40fc730 msgr2=0x7f2ba40feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:23.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.669+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3666540947 shutdown_connections 2024-04-25T19:03:23.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.669+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3666540947 wait complete. 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.670+0000 7f2ba9e00640 1 Processor -- start 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.670+0000 7f2ba9e00640 1 -- start start 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.670+0000 7f2ba9e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba4102f30 0x7f2ba419a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.670+0000 7f2ba9e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ba4103850 0x7f2ba419aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.670+0000 7f2ba9e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba410c350 0x7f2ba419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.670+0000 7f2ba9e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba4112010 con 0x7f2ba410c350 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.670+0000 7f2ba9e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba4111e90 con 0x7f2ba4103850 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.670+0000 7f2ba9e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba4112190 con 0x7f2ba4102f30 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2ba8a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba4102f30 0x7f2ba419a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2ba8a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba4102f30 0x7f2ba419a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39864/0 (socket says 172.21.15.45:39864) 2024-04-25T19:03:23.674 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2ba8a00640 1 -- 172.21.15.45:0/3862425265 learned_addr learned my addr 172.21.15.45:0/3862425265 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2ba9400640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba410c350 0x7f2ba419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2b9be00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ba4103850 0x7f2ba419aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2b9be00640 1 -- 172.21.15.45:0/3862425265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba4102f30 msgr2=0x7f2ba419a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2b9be00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba4102f30 0x7f2ba419a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2b9be00640 1 -- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba410c350 msgr2=0x7f2ba419f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2b9be00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba410c350 0x7f2ba419f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2b9be00640 1 -- 172.21.15.45:0/3862425265 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba419f750 con 0x7f2ba4103850 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba4102f30 0x7f2ba419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2ba9400640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba410c350 0x7f2ba419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.671+0000 7f2b9be00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ba4103850 0x7f2ba419aca0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c005fd0 tx=0x7f2b8c00db60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.672+0000 7f2ba2000640 1 -- 172.21.15.45:0/3862425265 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c004230 con 0x7f2ba4103850 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.672+0000 7f2ba2000640 1 -- 172.21.15.45:0/3862425265 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b8c0043d0 con 0x7f2ba4103850 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.672+0000 7f2ba2000640 1 -- 172.21.15.45:0/3862425265 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c003da0 con 0x7f2ba4103850 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.672+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3862425265 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba419fa40 con 0x7f2ba4103850 2024-04-25T19:03:23.675 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.672+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3862425265 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba41a72d0 con 0x7f2ba4103850 2024-04-25T19:03:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.677+0000 7f2bae21c740 1 Processor -- start 2024-04-25T19:03:23.679 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.677+0000 7f2bae21c740 1 -- start start 2024-04-25T19:03:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.677+0000 7f2ba2000640 1 -- 172.21.15.45:0/3862425265 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2b8c009820 con 0x7f2ba4103850 2024-04-25T19:03:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853ea2d0 0x55ee853f4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853f4b90 0x55ee853f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2bae21c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ee8534f900 con 0x55ee853ea2d0 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2bae21c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ee852f5ef0 con 0x55ee853f4b90 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2bae21c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ee8534b8c0 con 0x55ee853e99b0 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2ba9400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2ba9400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853e9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39874/0 (socket says 172.21.15.45:39874) 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2ba9400640 1 -- 172.21.15.45:0/2539606913 learned_addr learned my addr 172.21.15.45:0/2539606913 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2b9be00640 1 --2- 172.21.15.45:0/2539606913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853f4b90 0x55ee853f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/2539606913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853ea2d0 0x55ee853f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2ba9400640 1 -- 172.21.15.45:0/2539606913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853f4b90 msgr2=0x55ee853f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2ba9400640 1 --2- 172.21.15.45:0/2539606913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853f4b90 0x55ee853f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.681 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2ba9400640 1 -- 172.21.15.45:0/2539606913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853ea2d0 msgr2=0x55ee853f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2ba9400640 1 --2- 172.21.15.45:0/2539606913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853ea2d0 0x55ee853f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.678+0000 7f2ba9400640 1 -- 172.21.15.45:0/2539606913 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee85062210 con 0x55ee853e99b0 2024-04-25T19:03:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.679+0000 7f2ba9400640 1 --2- 172.21.15.45:0/2539606913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853e9d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c011790 tx=0x7f2b9c011c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d45e03f87f38b9a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.679+0000 7f2b99c00640 1 -- 172.21.15.45:0/2539606913 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c003c30 con 0x55ee853e99b0 2024-04-25T19:03:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.679+0000 7f2b99c00640 1 -- 172.21.15.45:0/2539606913 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9c00fe50 con 0x55ee853e99b0 2024-04-25T19:03:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.679+0000 7f2b99c00640 1 -- 172.21.15.45:0/2539606913 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0193e0 con 0x55ee853e99b0 2024-04-25T19:03:23.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2ba2000640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b880a3d80 0x7f2b880a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2bae21c740 1 -- 172.21.15.45:0/2539606913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 msgr2=0x55ee853e9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2bae21c740 1 --2- 172.21.15.45:0/2539606913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853e9d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c011790 tx=0x7f2b9c011c50 comp rx=0 tx=0).stop 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2ba2000640 1 -- 172.21.15.45:0/3862425265 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2b8c0d0160 con 0x7f2ba4103850 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b880a3d80 0x7f2b880a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2bae21c740 1 -- 172.21.15.45:0/2539606913 shutdown_connections 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2bae21c740 1 --2- 172.21.15.45:0/2539606913 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853f4b90 0x55ee853f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2bae21c740 1 --2- 172.21.15.45:0/2539606913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853ea2d0 0x55ee853f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2bae21c740 1 --2- 172.21.15.45:0/2539606913 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853e9d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2bae21c740 1 -- 172.21.15.45:0/2539606913 >> 172.21.15.45:0/2539606913 conn(0x55ee853e5ec0 msgr2=0x55ee853e62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2bae21c740 1 -- 172.21.15.45:0/2539606913 shutdown_connections 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.680+0000 7f2bae21c740 1 -- 172.21.15.45:0/2539606913 wait complete. 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.681+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b880a3d80 0x7f2b880a6240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55ee853eaf50 tx=0x7f2b94005a20 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:23.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.681+0000 7f2bae21c740 1 -- start start 2024-04-25T19:03:23.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.681+0000 7f2bae21c740 1 Processor -- start 2024-04-25T19:03:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853f1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853ea2d0 0x55ee853f2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 0x55ee853f2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2bae21c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ee853f77c0 con 0x55ee853f4b90 2024-04-25T19:03:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2bae21c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ee85309310 con 0x55ee853ea2d0 2024-04-25T19:03:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2bae21c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ee853f7940 con 0x55ee853e99b0 2024-04-25T19:03:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2bae21c740 10 client.0 _ll_register_callbacks cb 0x55ee852d7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2ba8a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853f1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2ba8a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853f1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39882/0 (socket says 172.21.15.45:39882) 2024-04-25T19:03:23.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2ba8a00640 1 -- 172.21.15.45:0/2651901948 learned_addr learned my addr 172.21.15.45:0/2651901948 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2b9be00640 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853ea2d0 0x55ee853f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2ba9400640 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 0x55ee853f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2ba8a00640 1 -- 172.21.15.45:0/2651901948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853ea2d0 msgr2=0x55ee853f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.682+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853ea2d0 0x55ee853f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.683+0000 7f2ba8a00640 1 -- 172.21.15.45:0/2651901948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 msgr2=0x55ee853f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.683+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 0x55ee853f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.683+0000 7f2ba8a00640 1 -- 172.21.15.45:0/2651901948 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee8512e620 con 0x55ee853e99b0 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.683+0000 7f2ba9400640 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 0x55ee853f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.683+0000 7f2b9be00640 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853ea2d0 0x55ee853f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.683+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853f1b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2b94003bd0 tx=0x7f2b94009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7923d0ba5863715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.683+0000 7f2b92000640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.683+0000 7f2b92000640 1 -- 172.21.15.45:0/2651901948 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94035040 con 0x55ee853e99b0 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.683+0000 7f2bae21c740 1 -- 172.21.15.45:0/2651901948 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b9c00fe50 con 0x55ee853e99b0 2024-04-25T19:03:23.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.683+0000 7f2b92000640 1 -- 172.21.15.45:0/2651901948 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2b94031bb0 con 0x55ee853e99b0 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2b92000640 1 -- 172.21.15.45:0/2651901948 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b94031db0 con 0x55ee853e99b0 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2bae21c740 10 client.25654 fetch_fsmap learned FSMap version 28 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2bae21c740 1 -- 172.21.15.45:0/2651901948 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee85403a50 con 0x55ee853e99b0 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2b92000640 1 -- 172.21.15.45:0/2651901948 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9c00fe50 con 0x55ee853e99b0 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2b92000640 1 -- 172.21.15.45:0/2651901948 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f2b94044030 con 0x55ee853e99b0 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2bae21c740 10 client.25654 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2bae21c740 10 client.25654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2bae21c740 20 client.25654 populate_metadata read hostname 'smithi045' 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2bae21c740 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ee85404ec0 0x55ee854072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2bae21c740 4 client.25654 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.684+0000 7f2bae21c740 1 -- 172.21.15.45:0/2651901948 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee853638b0 con 0x55ee85404ec0 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.685+0000 7f2b9be00640 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ee85404ec0 0x55ee854072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.685+0000 7f2b9be00640 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ee85404ec0 0x55ee854072f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8df66e6223e2ae5 server_cookie=6864aad330850d51 in_seq=0 out_seq=0 2024-04-25T19:03:23.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.685+0000 7f2b92000640 10 client.25654 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2b92000640 1 -- 172.21.15.45:0/2651901948 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ee853638b0 con 0x55ee85404ec0 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2b92000640 10 client.25654 handle_command_reply: tid=0 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2bae21c740 1 client.25654 shutdown 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2bae21c740 1 -- 172.21.15.45:0/2651901948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 msgr2=0x55ee853f1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2bae21c740 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853f1b00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2b94003bd0 tx=0x7f2b94009260 comp rx=0 tx=0).stop 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2bae21c740 1 -- 172.21.15.45:0/2651901948 shutdown_connections 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2bae21c740 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ee85404ec0 0x55ee854072f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2bae21c740 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 0x55ee853f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2bae21c740 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853ea2d0 0x55ee853f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2b92000640 0 client.25654 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2bae21c740 1 --2- 172.21.15.45:0/2651901948 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853e99b0 0x55ee853f1b00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.686+0000 7f2bae21c740 1 -- 172.21.15.45:0/2651901948 >> 172.21.15.45:0/2651901948 conn(0x55ee853e5ec0 msgr2=0x55ee853ebc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.687+0000 7f2bae21c740 1 -- 172.21.15.45:0/2651901948 shutdown_connections 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.687+0000 7f2bae21c740 1 -- 172.21.15.45:0/2651901948 wait complete. 2024-04-25T19:03:23.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.687+0000 7f2bae21c740 20 client.25654 trim_cache size 0 max 16384 2024-04-25T19:03:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.704+0000 7f2bae21c740 1 Processor -- start 2024-04-25T19:03:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.704+0000 7f2bae21c740 1 -- start start 2024-04-25T19:03:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.704+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee85400680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.707 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853ea2d0 0x55ee8540cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 0x55ee8540d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2bae21c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba41b24c0 con 0x55ee853f4b90 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2bae21c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba41afd90 con 0x55ee853e99b0 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2bae21c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba41adbe0 con 0x55ee853ea2d0 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2ba8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee85400680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2ba8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee85400680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:42916/0 (socket says 172.21.15.45:42916) 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2ba8a00640 1 -- 172.21.15.45:0/1773295280 learned_addr learned my addr 172.21.15.45:0/1773295280 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2b9be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853ea2d0 0x55ee8540cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2ba9400640 1 --2- 172.21.15.45:0/1773295280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 0x55ee8540d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2ba8a00640 1 -- 172.21.15.45:0/1773295280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853ea2d0 msgr2=0x55ee8540cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/1773295280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853ea2d0 0x55ee8540cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2ba8a00640 1 -- 172.21.15.45:0/1773295280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 msgr2=0x55ee8540d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/1773295280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 0x55ee8540d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.705+0000 7f2ba8a00640 1 -- 172.21.15.45:0/1773295280 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee853e5970 con 0x55ee853e99b0 2024-04-25T19:03:23.708 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.706+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/1773295280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee85400680 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2b94042040 tx=0x7f2b9402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd9b7ed96117736 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.706+0000 7f2b99200640 1 -- 172.21.15.45:0/1773295280 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9405e070 con 0x55ee853e99b0 2024-04-25T19:03:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.706+0000 7f2b99200640 1 -- 172.21.15.45:0/1773295280 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9404c070 con 0x55ee853e99b0 2024-04-25T19:03:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.706+0000 7f2b99200640 1 -- 172.21.15.45:0/1773295280 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94051de0 con 0x55ee853e99b0 2024-04-25T19:03:23.710 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 -- 172.21.15.45:0/1773295280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 msgr2=0x55ee85400680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 --2- 172.21.15.45:0/1773295280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee85400680 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2b94042040 tx=0x7f2b9402a480 comp rx=0 tx=0).stop 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 -- 172.21.15.45:0/1773295280 shutdown_connections 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 --2- 172.21.15.45:0/1773295280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853f4b90 0x55ee8540d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 --2- 172.21.15.45:0/1773295280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853ea2d0 0x55ee8540cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 --2- 172.21.15.45:0/1773295280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee85400680 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 -- 172.21.15.45:0/1773295280 >> 172.21.15.45:0/1773295280 conn(0x55ee853e5ec0 msgr2=0x55ee853ecdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 -- 172.21.15.45:0/1773295280 shutdown_connections 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 -- 172.21.15.45:0/1773295280 wait complete. 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 -- start start 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.707+0000 7f2bae21c740 1 Processor -- start 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.708+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee853fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.708+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853ea2d0 0x55ee853fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.708+0000 7f2bae21c740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853f4b90 0x55ee853fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.708+0000 7f2bae21c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba41ae250 con 0x55ee853ea2d0 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.708+0000 7f2bae21c740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba41af6c0 con 0x55ee853e99b0 2024-04-25T19:03:23.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.708+0000 7f2bae21c740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba41ae920 con 0x55ee853f4b90 2024-04-25T19:03:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.708+0000 7f2bae21c740 10 client.0 _ll_register_callbacks cb 0x7f2b94003880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.708+0000 7f2ba8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee853fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2ba8a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee853fcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:42918/0 (socket says 172.21.15.45:42918) 2024-04-25T19:03:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2ba8a00640 1 -- 172.21.15.45:0/807678519 learned_addr learned my addr 172.21.15.45:0/807678519 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:23.712 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2b9be00640 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853ea2d0 0x55ee853fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2ba9400640 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853f4b90 0x55ee853fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2ba8a00640 1 -- 172.21.15.45:0/807678519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853f4b90 msgr2=0x55ee853fa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853f4b90 0x55ee853fa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2ba8a00640 1 -- 172.21.15.45:0/807678519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853ea2d0 msgr2=0x55ee853fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853ea2d0 0x55ee853fa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2ba8a00640 1 -- 172.21.15.45:0/807678519 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee853fac80 con 0x55ee853e99b0 2024-04-25T19:03:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2ba8a00640 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee853fcd70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b94064040 tx=0x7f2b94009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8012e56530db4d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.709+0000 7f2b92000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:03:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.710+0000 7f2b92000640 1 -- 172.21.15.45:0/807678519 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b94070060 con 0x55ee853e99b0 2024-04-25T19:03:23.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.710+0000 7f2b92000640 1 -- 172.21.15.45:0/807678519 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2b94051d60 con 0x55ee853e99b0 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.710+0000 7f2bae21c740 1 -- 172.21.15.45:0/807678519 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b9404c070 con 0x55ee853e99b0 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.710+0000 7f2b92000640 1 -- 172.21.15.45:0/807678519 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2b9405e040 con 0x55ee853e99b0 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.710+0000 7f2bae21c740 10 client.25640 fetch_fsmap learned FSMap version 28 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.710+0000 7f2bae21c740 1 -- 172.21.15.45:0/807678519 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ee8546b2a0 con 0x55ee853e99b0 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.710+0000 7f2b92000640 1 -- 172.21.15.45:0/807678519 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b9404c070 con 0x55ee853e99b0 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.710+0000 7f2b92000640 1 -- 172.21.15.45:0/807678519 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 28) ==== 8346+0+0 (secure 0 0 0) 0x7f2b94044040 con 0x55ee853e99b0 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.711+0000 7f2bae21c740 10 client.25640 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.711+0000 7f2bae21c740 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.711+0000 7f2bae21c740 20 client.25640 populate_metadata read hostname 'smithi045' 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.711+0000 7f2bae21c740 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ee8546b8e0 0x55ee8546dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.711+0000 7f2bae21c740 4 client.25640 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.711+0000 7f2bae21c740 1 -- 172.21.15.45:0/807678519 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee8535d320 con 0x55ee8546b8e0 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.711+0000 7f2b9be00640 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ee8546b8e0 0x55ee8546dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.711+0000 7f2b9be00640 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ee8546b8e0 0x55ee8546dd00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3531d4da60541086 server_cookie=1d9c0dfdea456572 in_seq=0 out_seq=0 2024-04-25T19:03:23.715 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.711+0000 7f2b92000640 10 client.25640 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.738+0000 7f2b92000640 1 -- 172.21.15.45:0/807678519 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ee8535d320 con 0x55ee8546b8e0 2024-04-25T19:03:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.738+0000 7f2b92000640 10 client.25640 handle_command_reply: tid=0 2024-04-25T19:03:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.738+0000 7f2bae21c740 1 client.25640 shutdown 2024-04-25T19:03:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 1 -- 172.21.15.45:0/807678519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 msgr2=0x55ee853fcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee853fcd70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b94064040 tx=0x7f2b94009260 comp rx=0 tx=0).stop 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 1 -- 172.21.15.45:0/807678519 shutdown_connections 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ee8546b8e0 0x55ee8546dd00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ee853f4b90 0x55ee853fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ee853ea2d0 0x55ee853fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2b92000640 0 client.25640 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 1 --2- 172.21.15.45:0/807678519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ee853e99b0 0x55ee853fcd70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 1 -- 172.21.15.45:0/807678519 >> 172.21.15.45:0/807678519 conn(0x55ee853e5ec0 msgr2=0x55ee853f1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 1 -- 172.21.15.45:0/807678519 shutdown_connections 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 1 -- 172.21.15.45:0/807678519 wait complete. 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.739+0000 7f2bae21c740 20 client.25640 trim_cache size 0 max 16384 2024-04-25T19:03:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:03:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:03:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:03:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:03:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b880a3d80 msgr2=0x7f2b880a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b880a3d80 0x7f2b880a6240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55ee853eaf50 tx=0x7f2b94005a20 comp rx=0 tx=0).stop 2024-04-25T19:03:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3862425265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ba4103850 msgr2=0x7f2ba419aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ba4103850 0x7f2ba419aca0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c005fd0 tx=0x7f2b8c00db60 comp rx=0 tx=0).stop 2024-04-25T19:03:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3862425265 shutdown_connections 2024-04-25T19:03:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b880a3d80 0x7f2b880a6240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ba410c350 0x7f2ba419f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2ba4103850 0x7f2ba419aca0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 --2- 172.21.15.45:0/3862425265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ba4102f30 0x7f2ba419a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3862425265 >> 172.21.15.45:0/3862425265 conn(0x7f2ba40fc730 msgr2=0x7f2ba4100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3862425265 shutdown_connections 2024-04-25T19:03:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:23.740+0000 7f2ba9e00640 1 -- 172.21.15.45:0/3862425265 wait complete. 2024-04-25T19:03:24.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:23 smithi045 ceph-mon[27776]: pgmap v125: 97 pgs: 97 active+clean; 39 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 475 op/s 2024-04-25T19:03:24.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:23 smithi062 ceph-mon[32844]: pgmap v125: 97 pgs: 97 active+clean; 39 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 475 op/s 2024-04-25T19:03:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:23 smithi079 ceph-mon[33002]: pgmap v125: 97 pgs: 97 active+clean; 39 MiB data, 656 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 475 op/s 2024-04-25T19:03:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:25 smithi045 ceph-mon[27776]: pgmap v126: 97 pgs: 97 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 675 op/s 2024-04-25T19:03:26.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:25 smithi062 ceph-mon[32844]: pgmap v126: 97 pgs: 97 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 675 op/s 2024-04-25T19:03:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:25 smithi079 ceph-mon[33002]: pgmap v126: 97 pgs: 97 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 675 op/s 2024-04-25T19:03:28.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:27 smithi062 ceph-mon[32844]: pgmap v127: 97 pgs: 97 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 449 op/s 2024-04-25T19:03:28.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:27 smithi079 ceph-mon[33002]: pgmap v127: 97 pgs: 97 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 449 op/s 2024-04-25T19:03:28.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:27 smithi045 ceph-mon[27776]: pgmap v127: 97 pgs: 97 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 449 op/s 2024-04-25T19:03:30.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:29 smithi062 ceph-mon[32844]: pgmap v128: 97 pgs: 97 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 449 op/s 2024-04-25T19:03:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:29 smithi079 ceph-mon[33002]: pgmap v128: 97 pgs: 97 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 449 op/s 2024-04-25T19:03:30.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:29 smithi045 ceph-mon[27776]: pgmap v128: 97 pgs: 97 active+clean; 33 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 449 op/s 2024-04-25T19:03:32.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:31 smithi062 ceph-mon[32844]: pgmap v129: 97 pgs: 97 active+clean; 35 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 618 op/s 2024-04-25T19:03:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:31 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:03:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:31 smithi079 ceph-mon[33002]: pgmap v129: 97 pgs: 97 active+clean; 35 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 618 op/s 2024-04-25T19:03:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:31 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:03:32.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:31 smithi045 ceph-mon[27776]: pgmap v129: 97 pgs: 97 active+clean; 35 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.0 MiB/s wr, 618 op/s 2024-04-25T19:03:32.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:31 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:03:33.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:32 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:03:33.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:32 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:03:33.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:32 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:03:34.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:33 smithi062 ceph-mon[32844]: pgmap v130: 97 pgs: 97 active+clean; 35 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 368 op/s 2024-04-25T19:03:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:33 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:03:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:33 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:03:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:33 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:03:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:33 smithi079 ceph-mon[33002]: pgmap v130: 97 pgs: 97 active+clean; 35 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 368 op/s 2024-04-25T19:03:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:33 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:03:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:33 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:03:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:33 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:03:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:33 smithi045 ceph-mon[27776]: pgmap v130: 97 pgs: 97 active+clean; 35 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 368 op/s 2024-04-25T19:03:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:33 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:03:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:33 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:03:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:33 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:03:36.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:36 smithi062 ceph-mon[32844]: pgmap v131: 97 pgs: 97 active+clean; 38 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 541 op/s 2024-04-25T19:03:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:36 smithi079 ceph-mon[33002]: pgmap v131: 97 pgs: 97 active+clean; 38 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 541 op/s 2024-04-25T19:03:36.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:36 smithi045 ceph-mon[27776]: pgmap v131: 97 pgs: 97 active+clean; 38 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 541 op/s 2024-04-25T19:03:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.897+0000 7fb32da00640 1 -- 172.21.15.45:0/1555147289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32810c350 msgr2=0x7fb32810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.897+0000 7fb32da00640 1 --2- 172.21.15.45:0/1555147289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32810c350 0x7fb32810e7f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb320009ca0 tx=0x7fb32002a950 comp rx=0 tx=0).stop 2024-04-25T19:03:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.897+0000 7fb32da00640 1 -- 172.21.15.45:0/1555147289 shutdown_connections 2024-04-25T19:03:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.897+0000 7fb32da00640 1 --2- 172.21.15.45:0/1555147289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32810c350 0x7fb32810e7f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.897+0000 7fb32da00640 1 --2- 172.21.15.45:0/1555147289 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb328103850 0x7fb32810bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.897+0000 7fb32da00640 1 --2- 172.21.15.45:0/1555147289 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb328102f30 0x7fb328103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.897+0000 7fb32da00640 1 -- 172.21.15.45:0/1555147289 >> 172.21.15.45:0/1555147289 conn(0x7fb3280fc730 msgr2=0x7fb3280feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.897+0000 7fb32da00640 1 -- 172.21.15.45:0/1555147289 shutdown_connections 2024-04-25T19:03:36.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.897+0000 7fb32da00640 1 -- 172.21.15.45:0/1555147289 wait complete. 2024-04-25T19:03:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.898+0000 7fb32da00640 1 Processor -- start 2024-04-25T19:03:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb32da00640 1 -- start start 2024-04-25T19:03:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb32da00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb328102f30 0x7fb3281a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb32da00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb328103850 0x7fb3281a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb32da00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32810c350 0x7fb32819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb32da00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb328111fe0 con 0x7fb328103850 2024-04-25T19:03:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb32da00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb328111e60 con 0x7fb328102f30 2024-04-25T19:03:36.901 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb32da00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb328112160 con 0x7fb32810c350 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb327e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb328102f30 0x7fb3281a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb32d000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32810c350 0x7fb32819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb327e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb328102f30 0x7fb3281a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:47184/0 (socket says 172.21.15.45:47184) 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb327e00640 1 -- 172.21.15.45:0/1738440049 learned_addr learned my addr 172.21.15.45:0/1738440049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.899+0000 7fb327400640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb328103850 0x7fb3281a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.900+0000 7fb327e00640 1 -- 172.21.15.45:0/1738440049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32810c350 msgr2=0x7fb32819a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.900+0000 7fb327e00640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32810c350 0x7fb32819a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.900+0000 7fb327e00640 1 -- 172.21.15.45:0/1738440049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb328103850 msgr2=0x7fb3281a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.900+0000 7fb327e00640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb328103850 0x7fb3281a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.900+0000 7fb327e00640 1 -- 172.21.15.45:0/1738440049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32819af60 con 0x7fb328102f30 2024-04-25T19:03:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.900+0000 7fb327e00640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb328102f30 0x7fb3281a04f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb31800d950 tx=0x7fb31800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.900+0000 7fb324c00640 1 -- 172.21.15.45:0/1738440049 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318017070 con 0x7fb328102f30 2024-04-25T19:03:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.900+0000 7fb324c00640 1 -- 172.21.15.45:0/1738440049 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb318009bb0 con 0x7fb328102f30 2024-04-25T19:03:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.900+0000 7fb32da00640 1 -- 172.21.15.45:0/1738440049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb32819b250 con 0x7fb328102f30 2024-04-25T19:03:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.901+0000 7fb324c00640 1 -- 172.21.15.45:0/1738440049 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31800f620 con 0x7fb328102f30 2024-04-25T19:03:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.901+0000 7fb32da00640 1 -- 172.21.15.45:0/1738440049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3281a7470 con 0x7fb328102f30 2024-04-25T19:03:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.905+0000 7fb331e15740 1 Processor -- start 2024-04-25T19:03:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.905+0000 7fb331e15740 1 -- start start 2024-04-25T19:03:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.905+0000 7fb324c00640 1 -- 172.21.15.45:0/1738440049 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb318010460 con 0x7fb328102f30 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb331e15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119039b0 0x560211903d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb331e15740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x56021190e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb331e15740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 0x560211910f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb331e15740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560211869900 con 0x5602119042d0 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb331e15740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56021180fef0 con 0x56021190eb90 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb331e15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5602118658c0 con 0x5602119039b0 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb327e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x56021190e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb327400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 0x560211910f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb32d000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119039b0 0x560211903d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb327400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 0x560211910f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:47196/0 (socket says 172.21.15.45:47196) 2024-04-25T19:03:36.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb327400640 1 -- 172.21.15.45:0/1948176297 learned_addr learned my addr 172.21.15.45:0/1948176297 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.906+0000 7fb327e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x56021190e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:46252/0 (socket says 172.21.15.45:46252) 2024-04-25T19:03:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.907+0000 7fb327400640 1 -- 172.21.15.45:0/1948176297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119039b0 msgr2=0x560211903d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.907+0000 7fb327400640 1 --2- 172.21.15.45:0/1948176297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119039b0 0x560211903d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.907+0000 7fb327400640 1 -- 172.21.15.45:0/1948176297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 msgr2=0x56021190e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.907+0000 7fb327400640 1 --2- 172.21.15.45:0/1948176297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x56021190e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.907+0000 7fb327400640 1 -- 172.21.15.45:0/1948176297 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56021157c210 con 0x56021190eb90 2024-04-25T19:03:36.909 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.907+0000 7fb327400640 1 --2- 172.21.15.45:0/1948176297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 0x560211910f80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb314009c10 tx=0x7fb314011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f880d063c008690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.907+0000 7fb31d600640 1 -- 172.21.15.45:0/1948176297 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314004050 con 0x56021190eb90 2024-04-25T19:03:36.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.907+0000 7fb31d600640 1 -- 172.21.15.45:0/1948176297 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3140054d0 con 0x56021190eb90 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.907+0000 7fb31d600640 1 -- 172.21.15.45:0/1948176297 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb314004d00 con 0x56021190eb90 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.907+0000 7fb324c00640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb30c0a3d60 0x7fb30c0a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.908+0000 7fb324c00640 1 -- 172.21.15.45:0/1738440049 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fb3180d0300 con 0x7fb328102f30 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.908+0000 7fb32d000640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb30c0a3d60 0x7fb30c0a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.908+0000 7fb331e15740 1 -- 172.21.15.45:0/1948176297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 msgr2=0x560211910f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.908+0000 7fb331e15740 1 --2- 172.21.15.45:0/1948176297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 0x560211910f80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb314009c10 tx=0x7fb314011810 comp rx=0 tx=0).stop 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.909+0000 7fb331e15740 1 -- 172.21.15.45:0/1948176297 shutdown_connections 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.909+0000 7fb331e15740 1 --2- 172.21.15.45:0/1948176297 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 0x560211910f80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.909+0000 7fb331e15740 1 --2- 172.21.15.45:0/1948176297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x56021190e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.909+0000 7fb331e15740 1 --2- 172.21.15.45:0/1948176297 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119039b0 0x560211903d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.909+0000 7fb331e15740 1 -- 172.21.15.45:0/1948176297 >> 172.21.15.45:0/1948176297 conn(0x5602118ffec0 msgr2=0x5602119002a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.909+0000 7fb331e15740 1 -- 172.21.15.45:0/1948176297 shutdown_connections 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.909+0000 7fb32d000640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb30c0a3d60 0x7fb30c0a6220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb320005fe0 tx=0x7fb320005ef0 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.909+0000 7fb331e15740 1 -- 172.21.15.45:0/1948176297 wait complete. 2024-04-25T19:03:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.909+0000 7fb331e15740 1 -- start start 2024-04-25T19:03:36.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.909+0000 7fb331e15740 1 Processor -- start 2024-04-25T19:03:36.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb331e15740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x56021190bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb331e15740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x56021190c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb331e15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56021190eb90 0x56021190c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb331e15740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5602119117c0 con 0x5602119042d0 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb331e15740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560211823310 con 0x5602119039b0 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb331e15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560211911940 con 0x56021190eb90 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb331e15740 10 client.0 _ll_register_callbacks cb 0x5602117f1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb327e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x56021190bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb32d000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56021190eb90 0x56021190c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb327e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x56021190bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:47208/0 (socket says 172.21.15.45:47208) 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb327e00640 1 -- 172.21.15.45:0/67341919 learned_addr learned my addr 172.21.15.45:0/67341919 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.910+0000 7fb327400640 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x56021190c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.911+0000 7fb327e00640 1 -- 172.21.15.45:0/67341919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56021190eb90 msgr2=0x56021190c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.911+0000 7fb327e00640 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56021190eb90 0x56021190c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.915 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.911+0000 7fb327e00640 1 -- 172.21.15.45:0/67341919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 msgr2=0x56021190c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.911+0000 7fb327e00640 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x56021190c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.911+0000 7fb327e00640 1 -- 172.21.15.45:0/67341919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560211648620 con 0x5602119039b0 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.911+0000 7fb327e00640 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x56021190bb00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb31809c050 tx=0x7fb3180d3a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe6c1340591b6941 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.911+0000 7fb312000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.911+0000 7fb312000640 1 -- 172.21.15.45:0/67341919 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318020070 con 0x5602119039b0 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.911+0000 7fb312000640 1 -- 172.21.15.45:0/67341919 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fb31809b7a0 con 0x5602119039b0 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.911+0000 7fb331e15740 1 -- 172.21.15.45:0/67341919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3140054d0 con 0x5602119039b0 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb312000640 1 -- 172.21.15.45:0/67341919 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb31809bcb0 con 0x5602119039b0 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb331e15740 10 client.25652 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb331e15740 1 -- 172.21.15.45:0/67341919 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56021191da60 con 0x5602119039b0 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb312000640 1 -- 172.21.15.45:0/67341919 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3140054d0 con 0x5602119039b0 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb312000640 1 -- 172.21.15.45:0/67341919 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7fb318010750 con 0x5602119039b0 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb331e15740 10 client.25652 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb331e15740 10 client.25652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb331e15740 20 client.25652 populate_metadata read hostname 'smithi045' 2024-04-25T19:03:36.916 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb331e15740 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56021191ef70 0x5602119213c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb331e15740 4 client.25652 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.912+0000 7fb331e15740 1 -- 172.21.15.45:0/67341919 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56021187d8b0 con 0x56021191ef70 2024-04-25T19:03:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.913+0000 7fb327400640 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56021191ef70 0x5602119213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.913+0000 7fb327400640 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56021191ef70 0x5602119213c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc6bb2ced3ce173 server_cookie=b9fbe24ba79f19ef in_seq=0 out_seq=0 2024-04-25T19:03:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.913+0000 7fb312000640 10 client.25652 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb312000640 1 -- 172.21.15.45:0/67341919 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56021187d8b0 con 0x56021191ef70 2024-04-25T19:03:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb312000640 10 client.25652 handle_command_reply: tid=0 2024-04-25T19:03:36.917 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 client.25652 shutdown 2024-04-25T19:03:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 -- 172.21.15.45:0/67341919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 msgr2=0x56021190bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x56021190bb00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fb31809c050 tx=0x7fb3180d3a40 comp rx=0 tx=0).stop 2024-04-25T19:03:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 -- 172.21.15.45:0/67341919 shutdown_connections 2024-04-25T19:03:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x56021191ef70 0x5602119213c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56021190eb90 0x56021190c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x56021190c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 --2- 172.21.15.45:0/67341919 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x56021190bb00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb312000640 0 client.25652 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 -- 172.21.15.45:0/67341919 >> 172.21.15.45:0/67341919 conn(0x5602118ffec0 msgr2=0x560211905c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 -- 172.21.15.45:0/67341919 shutdown_connections 2024-04-25T19:03:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.914+0000 7fb331e15740 1 -- 172.21.15.45:0/67341919 wait complete. 2024-04-25T19:03:36.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.915+0000 7fb331e15740 20 client.25652 trim_cache size 0 max 16384 2024-04-25T19:03:36.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.932+0000 7fb331e15740 1 Processor -- start 2024-04-25T19:03:36.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb331e15740 1 -- start start 2024-04-25T19:03:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb331e15740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119039b0 0x56021191a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb331e15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119042d0 0x560211906370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb331e15740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 0x560211926cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb331e15740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281b2570 con 0x5602119039b0 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb331e15740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281afe40 con 0x56021190eb90 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb331e15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281adc90 con 0x5602119042d0 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb327e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119039b0 0x56021191a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb327400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119042d0 0x560211906370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb327400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119042d0 0x560211906370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39780/0 (socket says 172.21.15.45:39780) 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb327400640 1 -- 172.21.15.45:0/3490090904 learned_addr learned my addr 172.21.15.45:0/3490090904 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.933+0000 7fb32d000640 1 --2- 172.21.15.45:0/3490090904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 0x560211926cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.934+0000 7fb327e00640 1 -- 172.21.15.45:0/3490090904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119042d0 msgr2=0x560211906370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.934+0000 7fb327e00640 1 --2- 172.21.15.45:0/3490090904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119042d0 0x560211906370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.934+0000 7fb327e00640 1 -- 172.21.15.45:0/3490090904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 msgr2=0x560211926cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.934+0000 7fb327e00640 1 --2- 172.21.15.45:0/3490090904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 0x560211926cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.936 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.934+0000 7fb327e00640 1 -- 172.21.15.45:0/3490090904 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56021190ca60 con 0x5602119039b0 2024-04-25T19:03:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.934+0000 7fb327400640 1 --2- 172.21.15.45:0/3490090904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119042d0 0x560211906370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.934+0000 7fb327e00640 1 --2- 172.21.15.45:0/3490090904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119039b0 0x56021191a680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb318012040 tx=0x7fb3180d0290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2031eed55ba6ace4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.934+0000 7fb31cc00640 1 -- 172.21.15.45:0/3490090904 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318020070 con 0x5602119039b0 2024-04-25T19:03:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.934+0000 7fb31cc00640 1 -- 172.21.15.45:0/3490090904 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb318009d60 con 0x5602119039b0 2024-04-25T19:03:36.937 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.934+0000 7fb31cc00640 1 -- 172.21.15.45:0/3490090904 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31809b7a0 con 0x5602119039b0 2024-04-25T19:03:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.935+0000 7fb331e15740 1 -- 172.21.15.45:0/3490090904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119039b0 msgr2=0x56021191a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.935+0000 7fb331e15740 1 --2- 172.21.15.45:0/3490090904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119039b0 0x56021191a680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb318012040 tx=0x7fb3180d0290 comp rx=0 tx=0).stop 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.935+0000 7fb331e15740 1 -- 172.21.15.45:0/3490090904 shutdown_connections 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.935+0000 7fb331e15740 1 --2- 172.21.15.45:0/3490090904 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x56021190eb90 0x560211926cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.935+0000 7fb331e15740 1 --2- 172.21.15.45:0/3490090904 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5602119042d0 0x560211906370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.935+0000 7fb331e15740 1 --2- 172.21.15.45:0/3490090904 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119039b0 0x56021191a680 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.935+0000 7fb331e15740 1 -- 172.21.15.45:0/3490090904 >> 172.21.15.45:0/3490090904 conn(0x5602118ffec0 msgr2=0x56021190c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.935+0000 7fb331e15740 1 -- 172.21.15.45:0/3490090904 shutdown_connections 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.936+0000 7fb331e15740 1 -- 172.21.15.45:0/3490090904 wait complete. 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.936+0000 7fb331e15740 1 -- start start 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.936+0000 7fb331e15740 1 Processor -- start 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb331e15740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x560211917010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb331e15740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x5602119143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb331e15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56021190eb90 0x560211914940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb331e15740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281ae300 con 0x5602119042d0 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb331e15740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281af770 con 0x5602119039b0 2024-04-25T19:03:36.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb331e15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb3281ae9d0 con 0x56021190eb90 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb331e15740 10 client.0 _ll_register_callbacks cb 0x7fb318010f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb327e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x560211917010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb327e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x560211917010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:47236/0 (socket says 172.21.15.45:47236) 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb327e00640 1 -- 172.21.15.45:0/1492965501 learned_addr learned my addr 172.21.15.45:0/1492965501 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb32d000640 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56021190eb90 0x560211914940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb327400640 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x5602119143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb327e00640 1 -- 172.21.15.45:0/1492965501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56021190eb90 msgr2=0x560211914940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb327e00640 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56021190eb90 0x560211914940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb327e00640 1 -- 172.21.15.45:0/1492965501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 msgr2=0x5602119143a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb327e00640 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x5602119143a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb327e00640 1 -- 172.21.15.45:0/1492965501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560211915170 con 0x5602119039b0 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.937+0000 7fb327400640 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x5602119143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.938+0000 7fb327e00640 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x560211917010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb318010b60 tx=0x7fb318009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f47578c5c6c60820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:36.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.938+0000 7fb312000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.938+0000 7fb312000640 1 -- 172.21.15.45:0/1492965501 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318020050 con 0x5602119039b0 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.938+0000 7fb312000640 1 -- 172.21.15.45:0/1492965501 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fb3180a0140 con 0x5602119039b0 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.938+0000 7fb331e15740 1 -- 172.21.15.45:0/1492965501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb318009d60 con 0x5602119039b0 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.938+0000 7fb312000640 1 -- 172.21.15.45:0/1492965501 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb31807c020 con 0x5602119039b0 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.938+0000 7fb331e15740 10 client.25658 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.938+0000 7fb331e15740 1 -- 172.21.15.45:0/1492965501 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560211985630 con 0x5602119039b0 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.938+0000 7fb312000640 1 -- 172.21.15.45:0/1492965501 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb318009d60 con 0x5602119039b0 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.939+0000 7fb312000640 1 -- 172.21.15.45:0/1492965501 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7fb31809b310 con 0x5602119039b0 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.939+0000 7fb331e15740 10 client.25658 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.939+0000 7fb331e15740 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.939+0000 7fb331e15740 20 client.25658 populate_metadata read hostname 'smithi045' 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.939+0000 7fb331e15740 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560211985b00 0x560211987ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.939+0000 7fb331e15740 4 client.25658 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.939+0000 7fb331e15740 1 -- 172.21.15.45:0/1492965501 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560211877320 con 0x560211985b00 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.939+0000 7fb327400640 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560211985b00 0x560211987ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:36.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.940+0000 7fb327400640 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560211985b00 0x560211987ef0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23dcbf90d3d6f95a server_cookie=f64860c8ba76a3dd in_seq=0 out_seq=0 2024-04-25T19:03:36.944 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.940+0000 7fb312000640 10 client.25658 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.962+0000 7fb312000640 1 -- 172.21.15.45:0/1492965501 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560211877320 con 0x560211985b00 2024-04-25T19:03:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.962+0000 7fb312000640 10 client.25658 handle_command_reply: tid=0 2024-04-25T19:03:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.962+0000 7fb331e15740 1 client.25658 shutdown 2024-04-25T19:03:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.963+0000 7fb331e15740 1 -- 172.21.15.45:0/1492965501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 msgr2=0x560211917010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.963+0000 7fb331e15740 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x560211917010 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb318010b60 tx=0x7fb318009730 comp rx=0 tx=0).stop 2024-04-25T19:03:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.963+0000 7fb331e15740 1 -- 172.21.15.45:0/1492965501 shutdown_connections 2024-04-25T19:03:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.963+0000 7fb331e15740 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560211985b00 0x560211987ef0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.963+0000 7fb331e15740 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x56021190eb90 0x560211914940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.963+0000 7fb331e15740 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5602119042d0 0x5602119143a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.963+0000 7fb331e15740 1 --2- 172.21.15.45:0/1492965501 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5602119039b0 0x560211917010 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.963+0000 7fb312000640 0 client.25658 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.963+0000 7fb331e15740 1 -- 172.21.15.45:0/1492965501 >> 172.21.15.45:0/1492965501 conn(0x5602118ffec0 msgr2=0x56021190c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.963+0000 7fb331e15740 1 -- 172.21.15.45:0/1492965501 shutdown_connections 2024-04-25T19:03:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.964+0000 7fb331e15740 1 -- 172.21.15.45:0/1492965501 wait complete. 2024-04-25T19:03:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.964+0000 7fb331e15740 20 client.25658 trim_cache size 0 max 16384 2024-04-25T19:03:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:03:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:03:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:03:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 -- 172.21.15.45:0/1738440049 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb30c0a3d60 msgr2=0x7fb30c0a6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb30c0a3d60 0x7fb30c0a6220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb320005fe0 tx=0x7fb320005ef0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 -- 172.21.15.45:0/1738440049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb328102f30 msgr2=0x7fb3281a04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb328102f30 0x7fb3281a04f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb31800d950 tx=0x7fb31800de10 comp rx=0 tx=0).stop 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 -- 172.21.15.45:0/1738440049 shutdown_connections 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb30c0a3d60 0x7fb30c0a6220 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32810c350 0x7fb32819a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb328103850 0x7fb3281a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 --2- 172.21.15.45:0/1738440049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb328102f30 0x7fb3281a04f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 -- 172.21.15.45:0/1738440049 >> 172.21.15.45:0/1738440049 conn(0x7fb3280fc730 msgr2=0x7fb328101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 -- 172.21.15.45:0/1738440049 shutdown_connections 2024-04-25T19:03:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:36.965+0000 7fb32da00640 1 -- 172.21.15.45:0/1738440049 wait complete. 2024-04-25T19:03:38.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:38 smithi062 ceph-mon[32844]: pgmap v132: 97 pgs: 97 active+clean; 38 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 341 op/s 2024-04-25T19:03:38.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:38 smithi079 ceph-mon[33002]: pgmap v132: 97 pgs: 97 active+clean; 38 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 341 op/s 2024-04-25T19:03:38.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:38 smithi045 ceph-mon[27776]: pgmap v132: 97 pgs: 97 active+clean; 38 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 341 op/s 2024-04-25T19:03:40.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:40 smithi062 ceph-mon[32844]: pgmap v133: 97 pgs: 97 active+clean; 38 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 341 op/s 2024-04-25T19:03:40.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:40 smithi079 ceph-mon[33002]: pgmap v133: 97 pgs: 97 active+clean; 38 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 341 op/s 2024-04-25T19:03:40.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:40 smithi045 ceph-mon[27776]: pgmap v133: 97 pgs: 97 active+clean; 38 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 341 op/s 2024-04-25T19:03:41.188 INFO:tasks.workunit.client.0.smithi045.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-04-25T19:03:41.673 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0xffcdd 2024-04-25T19:03:42.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:42 smithi062 ceph-mon[32844]: pgmap v134: 97 pgs: 97 active+clean; 36 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 526 op/s 2024-04-25T19:03:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:42 smithi079 ceph-mon[33002]: pgmap v134: 97 pgs: 97 active+clean; 36 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 526 op/s 2024-04-25T19:03:42.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:42 smithi045 ceph-mon[27776]: pgmap v134: 97 pgs: 97 active+clean; 36 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 526 op/s 2024-04-25T19:03:43.996 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0xffd97 2024-04-25T19:03:44.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:44 smithi062 ceph-mon[32844]: pgmap v135: 97 pgs: 97 active+clean; 36 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 356 op/s 2024-04-25T19:03:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:44 smithi079 ceph-mon[33002]: pgmap v135: 97 pgs: 97 active+clean; 36 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 356 op/s 2024-04-25T19:03:44.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:44 smithi045 ceph-mon[27776]: pgmap v135: 97 pgs: 97 active+clean; 36 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 356 op/s 2024-04-25T19:03:46.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:46 smithi062 ceph-mon[32844]: pgmap v136: 97 pgs: 97 active+clean; 40 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 535 op/s 2024-04-25T19:03:46.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:46 smithi079 ceph-mon[33002]: pgmap v136: 97 pgs: 97 active+clean; 40 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 535 op/s 2024-04-25T19:03:46.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:46 smithi045 ceph-mon[27776]: pgmap v136: 97 pgs: 97 active+clean; 40 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 535 op/s 2024-04-25T19:03:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:48 smithi062 ceph-mon[32844]: pgmap v137: 97 pgs: 97 active+clean; 40 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-04-25T19:03:48.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:48 smithi079 ceph-mon[33002]: pgmap v137: 97 pgs: 97 active+clean; 40 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-04-25T19:03:48.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:48 smithi045 ceph-mon[27776]: pgmap v137: 97 pgs: 97 active+clean; 40 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-04-25T19:03:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.116+0000 7f09a5a00640 1 -- 172.21.15.45:0/2104869418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a01038f0 msgr2=0x7f09a010bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.116+0000 7f09a5a00640 1 --2- 172.21.15.45:0/2104869418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a01038f0 0x7f09a010bdd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0988009900 tx=0x7f098802a490 comp rx=0 tx=0).stop 2024-04-25T19:03:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.116+0000 7f09a5a00640 1 -- 172.21.15.45:0/2104869418 shutdown_connections 2024-04-25T19:03:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.116+0000 7f09a5a00640 1 --2- 172.21.15.45:0/2104869418 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09a010c3a0 0x7f09a010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.116+0000 7f09a5a00640 1 --2- 172.21.15.45:0/2104869418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a01038f0 0x7f09a010bdd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.116+0000 7f09a5a00640 1 --2- 172.21.15.45:0/2104869418 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a0102fd0 0x7f09a01033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.116+0000 7f09a5a00640 1 -- 172.21.15.45:0/2104869418 >> 172.21.15.45:0/2104869418 conn(0x7f09a00fc730 msgr2=0x7f09a00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.116+0000 7f09a5a00640 1 -- 172.21.15.45:0/2104869418 shutdown_connections 2024-04-25T19:03:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.116+0000 7f09a5a00640 1 -- 172.21.15.45:0/2104869418 wait complete. 2024-04-25T19:03:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.117+0000 7f09a5a00640 1 Processor -- start 2024-04-25T19:03:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.118+0000 7f09a5a00640 1 -- start start 2024-04-25T19:03:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.118+0000 7f09a5a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a0102fd0 0x7f09a01a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.118+0000 7f09a5a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09a01038f0 0x7f09a01a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.118+0000 7f09a5a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a010c3a0 0x7f09a019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.118+0000 7f09a5a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09a0112010 con 0x7f09a01038f0 2024-04-25T19:03:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.118+0000 7f09a5a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09a0111e90 con 0x7f09a0102fd0 2024-04-25T19:03:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.118+0000 7f09a5a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f09a0112190 con 0x7f09a010c3a0 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f099fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a0102fd0 0x7f09a01a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f099fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a0102fd0 0x7f09a01a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:60786/0 (socket says 172.21.15.45:60786) 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f099fe00640 1 -- 172.21.15.45:0/1024617750 learned_addr learned my addr 172.21.15.45:0/1024617750 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f09a5000640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a010c3a0 0x7f09a019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f0997e00640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09a01038f0 0x7f09a01a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f099fe00640 1 -- 172.21.15.45:0/1024617750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a010c3a0 msgr2=0x7f09a019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f099fe00640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a010c3a0 0x7f09a019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f099fe00640 1 -- 172.21.15.45:0/1024617750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09a01038f0 msgr2=0x7f09a01a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f099fe00640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09a01038f0 0x7f09a01a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f099fe00640 1 -- 172.21.15.45:0/1024617750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a019ad90 con 0x7f09a0102fd0 2024-04-25T19:03:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.119+0000 7f099fe00640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a0102fd0 0x7f09a01a0410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f099000e9f0 tx=0x7f099000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.120+0000 7f099d600640 1 -- 172.21.15.45:0/1024617750 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099000ccc0 con 0x7f09a0102fd0 2024-04-25T19:03:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.120+0000 7f099d600640 1 -- 172.21.15.45:0/1024617750 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09900108e0 con 0x7f09a0102fd0 2024-04-25T19:03:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.120+0000 7f09a5a00640 1 -- 172.21.15.45:0/1024617750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a019b080 con 0x7f09a0102fd0 2024-04-25T19:03:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.120+0000 7f099d600640 1 -- 172.21.15.45:0/1024617750 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990018880 con 0x7f09a0102fd0 2024-04-25T19:03:50.122 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.120+0000 7f09a5a00640 1 -- 172.21.15.45:0/1024617750 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a01a72d0 con 0x7f09a0102fd0 2024-04-25T19:03:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.123+0000 7f099d600640 1 -- 172.21.15.45:0/1024617750 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0990010430 con 0x7f09a0102fd0 2024-04-25T19:03:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.123+0000 7f09a9d66740 1 Processor -- start 2024-04-25T19:03:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.123+0000 7f09a9d66740 1 -- start start 2024-04-25T19:03:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 0x5598f8eebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 0x5598f8ef6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8ef8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f09a9d66740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5598f8e51900 con 0x5598f8eeb9b0 2024-04-25T19:03:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f09a9d66740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5598f8df7ef0 con 0x5598f8ef6b90 2024-04-25T19:03:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f09a9d66740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5598f8e4d8c0 con 0x5598f8eec2d0 2024-04-25T19:03:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f0997e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8ef8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f099fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 0x5598f8ef6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f09a5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 0x5598f8eebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f0997e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8ef8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:60788/0 (socket says 172.21.15.45:60788) 2024-04-25T19:03:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f09a5000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 0x5598f8eebd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56394/0 (socket says 172.21.15.45:56394) 2024-04-25T19:03:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.124+0000 7f0997e00640 1 -- 172.21.15.45:0/916074873 learned_addr learned my addr 172.21.15.45:0/916074873 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.125+0000 7f0997e00640 1 -- 172.21.15.45:0/916074873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 msgr2=0x5598f8ef6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.125+0000 7f0997e00640 1 --2- 172.21.15.45:0/916074873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 0x5598f8ef6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.125+0000 7f0997e00640 1 -- 172.21.15.45:0/916074873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 msgr2=0x5598f8eebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.125+0000 7f0997e00640 1 --2- 172.21.15.45:0/916074873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 0x5598f8eebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.125+0000 7f0997e00640 1 -- 172.21.15.45:0/916074873 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f8b64210 con 0x5598f8ef6b90 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.125+0000 7f0997e00640 1 --2- 172.21.15.45:0/916074873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8ef8f80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0988009900 tx=0x7f09880040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ddac33a1324c548 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.125+0000 7f099d600640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f09840a3d00 0x7f09840a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.126+0000 7f0995600640 1 -- 172.21.15.45:0/916074873 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988035030 con 0x5598f8ef6b90 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.126+0000 7f0995600640 1 -- 172.21.15.45:0/916074873 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09880044b0 con 0x5598f8ef6b90 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.126+0000 7f099d600640 1 -- 172.21.15.45:0/1024617750 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f09900d0cc0 con 0x7f09a0102fd0 2024-04-25T19:03:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.126+0000 7f0995600640 1 -- 172.21.15.45:0/916074873 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09880035f0 con 0x5598f8ef6b90 2024-04-25T19:03:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.126+0000 7f09a5000640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f09840a3d00 0x7f09840a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.126+0000 7f09a9d66740 1 -- 172.21.15.45:0/916074873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 msgr2=0x5598f8ef8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.126+0000 7f09a9d66740 1 --2- 172.21.15.45:0/916074873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8ef8f80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0988009900 tx=0x7f09880040c0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.127+0000 7f09a5000640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f09840a3d00 0x7f09840a61c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0998005fd0 tx=0x7f0998009500 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.127+0000 7f09a9d66740 1 -- 172.21.15.45:0/916074873 shutdown_connections 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.127+0000 7f09a9d66740 1 --2- 172.21.15.45:0/916074873 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8ef8f80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.127+0000 7f09a9d66740 1 --2- 172.21.15.45:0/916074873 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 0x5598f8ef6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.127+0000 7f09a9d66740 1 --2- 172.21.15.45:0/916074873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 0x5598f8eebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.127+0000 7f09a9d66740 1 -- 172.21.15.45:0/916074873 >> 172.21.15.45:0/916074873 conn(0x5598f8ee7ec0 msgr2=0x5598f8ee82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.127+0000 7f09a9d66740 1 -- 172.21.15.45:0/916074873 shutdown_connections 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.127+0000 7f09a9d66740 1 -- 172.21.15.45:0/916074873 wait complete. 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.127+0000 7f09a9d66740 1 -- start start 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.127+0000 7f09a9d66740 1 Processor -- start 2024-04-25T19:03:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 0x5598f8ef3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 0x5598f8ef4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8ef45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f09a9d66740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5598f8ef9760 con 0x5598f8eeb9b0 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f09a9d66740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5598f8e0b310 con 0x5598f8ef6b90 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f09a9d66740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5598f8ef98e0 con 0x5598f8eec2d0 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f09a9d66740 10 client.0 _ll_register_callbacks cb 0x5598f8dd9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f0997e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 0x5598f8ef4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f0997e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 0x5598f8ef4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:36046/0 (socket says 172.21.15.45:36046) 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f0997e00640 1 -- 172.21.15.45:0/2355227762 learned_addr learned my addr 172.21.15.45:0/2355227762 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f099fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 0x5598f8ef3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.128+0000 7f09a5000640 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8ef45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f0997e00640 1 -- 172.21.15.45:0/2355227762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 msgr2=0x5598f8ef45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f0997e00640 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8ef45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f0997e00640 1 -- 172.21.15.45:0/2355227762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 msgr2=0x5598f8ef3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f0997e00640 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 0x5598f8ef3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f0997e00640 1 -- 172.21.15.45:0/2355227762 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f8c30620 con 0x5598f8eec2d0 2024-04-25T19:03:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f099fe00640 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 0x5598f8ef3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:03:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f0997e00640 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 0x5598f8ef4080 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f098802a9b0 tx=0x7f0988003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c042689f8ef54f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f098e000640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:03:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f098e000640 1 -- 172.21.15.45:0/2355227762 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0988035070 con 0x5598f8eec2d0 2024-04-25T19:03:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f09a9d66740 1 -- 172.21.15.45:0/2355227762 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09880044b0 con 0x5598f8eec2d0 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.129+0000 7f098e000640 1 -- 172.21.15.45:0/2355227762 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f0988003c20 con 0x5598f8eec2d0 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.130+0000 7f098e000640 1 -- 172.21.15.45:0/2355227762 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0988048070 con 0x5598f8eec2d0 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.130+0000 7f09a9d66740 10 client.25690 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.130+0000 7f09a9d66740 1 -- 172.21.15.45:0/2355227762 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5598f8ef9760 con 0x5598f8eec2d0 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.130+0000 7f098e000640 1 -- 172.21.15.45:0/2355227762 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09880044b0 con 0x5598f8eec2d0 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.131+0000 7f098e000640 1 -- 172.21.15.45:0/2355227762 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f09880314a0 con 0x5598f8eec2d0 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.131+0000 7f09a9d66740 10 client.25690 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.131+0000 7f09a9d66740 10 client.25690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.131+0000 7f09a9d66740 20 client.25690 populate_metadata read hostname 'smithi045' 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.131+0000 7f09a9d66740 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5598f8f06b60 0x5598f8f08f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.131+0000 7f09a9d66740 4 client.25690 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.131+0000 7f09a9d66740 1 -- 172.21.15.45:0/2355227762 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598f8e658b0 con 0x5598f8f06b60 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.131+0000 7f099fe00640 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5598f8f06b60 0x5598f8f08f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.132+0000 7f098e000640 10 client.25690 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.132+0000 7f099fe00640 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5598f8f06b60 0x5598f8f08f50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=943f9abee694dbd2 server_cookie=b962cf6ba6b70492 in_seq=0 out_seq=0 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f098e000640 1 -- 172.21.15.45:0/2355227762 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598f8e658b0 con 0x5598f8f06b60 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f098e000640 10 client.25690 handle_command_reply: tid=0 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 client.25690 shutdown 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 -- 172.21.15.45:0/2355227762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 msgr2=0x5598f8ef4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 0x5598f8ef4080 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f098802a9b0 tx=0x7f0988003820 comp rx=0 tx=0).stop 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 -- 172.21.15.45:0/2355227762 shutdown_connections 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5598f8f06b60 0x5598f8f08f50 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8ef45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eec2d0 0x5598f8ef4080 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 --2- 172.21.15.45:0/2355227762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eeb9b0 0x5598f8ef3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 -- 172.21.15.45:0/2355227762 >> 172.21.15.45:0/2355227762 conn(0x5598f8ee7ec0 msgr2=0x5598f8eedc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f098e000640 0 client.25690 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 -- 172.21.15.45:0/2355227762 shutdown_connections 2024-04-25T19:03:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.133+0000 7f09a9d66740 1 -- 172.21.15.45:0/2355227762 wait complete. 2024-04-25T19:03:50.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.134+0000 7f09a9d66740 20 client.25690 trim_cache size 0 max 16384 2024-04-25T19:03:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.150+0000 7f09a9d66740 1 Processor -- start 2024-04-25T19:03:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.150+0000 7f09a9d66740 1 -- start start 2024-04-25T19:03:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.150+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8f02620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.150+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8ee6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.150+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8f0e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.150+0000 7f09a9d66740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01b2460 con 0x5598f8eec2d0 2024-04-25T19:03:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.150+0000 7f09a9d66740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01afd30 con 0x5598f8ef6b90 2024-04-25T19:03:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.150+0000 7f09a9d66740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01adb80 con 0x5598f8eeb9b0 2024-04-25T19:03:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.150+0000 7f099fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8f02620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f0997e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8ee6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f09a5000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8f0e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f099fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8f02620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:36050/0 (socket says 172.21.15.45:36050) 2024-04-25T19:03:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f099fe00640 1 -- 172.21.15.45:0/891345529 learned_addr learned my addr 172.21.15.45:0/891345529 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f099fe00640 1 -- 172.21.15.45:0/891345529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 msgr2=0x5598f8f0e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f099fe00640 1 --2- 172.21.15.45:0/891345529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8f0e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f099fe00640 1 -- 172.21.15.45:0/891345529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 msgr2=0x5598f8ee6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f099fe00640 1 --2- 172.21.15.45:0/891345529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8ee6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f099fe00640 1 -- 172.21.15.45:0/891345529 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f8ee7b10 con 0x5598f8eeb9b0 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f0997e00640 1 --2- 172.21.15.45:0/891345529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8ee6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f099fe00640 1 --2- 172.21.15.45:0/891345529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8f02620 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f099009c900 tx=0x7f09900d4770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=427f632be89337e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.151+0000 7f0994c00640 1 -- 172.21.15.45:0/891345529 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990010720 con 0x5598f8eeb9b0 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f0994c00640 1 -- 172.21.15.45:0/891345529 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099009d300 con 0x5598f8eeb9b0 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f0994c00640 1 -- 172.21.15.45:0/891345529 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0990010a90 con 0x5598f8eeb9b0 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f09a9d66740 1 -- 172.21.15.45:0/891345529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 msgr2=0x5598f8f02620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f09a9d66740 1 --2- 172.21.15.45:0/891345529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8f02620 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f099009c900 tx=0x7f09900d4770 comp rx=0 tx=0).stop 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f09a9d66740 1 -- 172.21.15.45:0/891345529 shutdown_connections 2024-04-25T19:03:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f09a9d66740 1 --2- 172.21.15.45:0/891345529 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8f0e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f09a9d66740 1 --2- 172.21.15.45:0/891345529 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8ee6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f09a9d66740 1 --2- 172.21.15.45:0/891345529 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8f02620 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f09a9d66740 1 -- 172.21.15.45:0/891345529 >> 172.21.15.45:0/891345529 conn(0x5598f8ee7ec0 msgr2=0x5598f8ee6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f09a9d66740 1 -- 172.21.15.45:0/891345529 shutdown_connections 2024-04-25T19:03:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.152+0000 7f09a9d66740 1 -- 172.21.15.45:0/891345529 wait complete. 2024-04-25T19:03:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.153+0000 7f09a9d66740 1 -- start start 2024-04-25T19:03:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.153+0000 7f09a9d66740 1 Processor -- start 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8efc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8efcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f09a9d66740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8efd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f09a9d66740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01ae1f0 con 0x5598f8eec2d0 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f09a9d66740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01af660 con 0x5598f8ef6b90 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f09a9d66740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f09a01ae8c0 con 0x5598f8eeb9b0 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f09a9d66740 10 client.0 _ll_register_callbacks cb 0x7f0990005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f099fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8efc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f0997e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8efcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f099fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8efc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:36066/0 (socket says 172.21.15.45:36066) 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f0997e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8efcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:56430/0 (socket says 172.21.15.45:56430) 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f099fe00640 1 -- 172.21.15.45:0/152673452 learned_addr learned my addr 172.21.15.45:0/152673452 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f09a5000640 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8efd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f099fe00640 1 -- 172.21.15.45:0/152673452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 msgr2=0x5598f8efd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f099fe00640 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8efd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f099fe00640 1 -- 172.21.15.45:0/152673452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 msgr2=0x5598f8efcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f099fe00640 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8efcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f099fe00640 1 -- 172.21.15.45:0/152673452 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f8f6bfe0 con 0x5598f8eeb9b0 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.154+0000 7f0997e00640 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8efcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.155+0000 7f099fe00640 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8efc5f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0990000c00 tx=0x7f099000c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22b9145f3eed2760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.155+0000 7f098e000640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.155+0000 7f098e000640 1 -- 172.21.15.45:0/152673452 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09900b6070 con 0x5598f8eeb9b0 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.155+0000 7f098e000640 1 -- 172.21.15.45:0/152673452 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f0990099270 con 0x5598f8eeb9b0 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.155+0000 7f09a9d66740 1 -- 172.21.15.45:0/152673452 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099009d300 con 0x5598f8eeb9b0 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.155+0000 7f098e000640 1 -- 172.21.15.45:0/152673452 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f099009cc00 con 0x5598f8eeb9b0 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.155+0000 7f09a9d66740 10 client.25696 fetch_fsmap learned FSMap version 29 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.155+0000 7f09a9d66740 1 -- 172.21.15.45:0/152673452 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5598f8f6d2d0 con 0x5598f8eeb9b0 2024-04-25T19:03:50.159 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.155+0000 7f098e000640 1 -- 172.21.15.45:0/152673452 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099009d300 con 0x5598f8eeb9b0 2024-04-25T19:03:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.156+0000 7f098e000640 1 -- 172.21.15.45:0/152673452 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f0990063ce0 con 0x5598f8eeb9b0 2024-04-25T19:03:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.156+0000 7f09a9d66740 10 client.25696 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:03:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.156+0000 7f09a9d66740 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:03:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.156+0000 7f09a9d66740 20 client.25696 populate_metadata read hostname 'smithi045' 2024-04-25T19:03:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.156+0000 7f09a9d66740 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5598f8f6da10 0x5598f8f6fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:03:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.156+0000 7f09a9d66740 4 client.25696 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:03:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.156+0000 7f09a9d66740 1 -- 172.21.15.45:0/152673452 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598f8e5f320 con 0x5598f8f6da10 2024-04-25T19:03:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.156+0000 7f0997e00640 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5598f8f6da10 0x5598f8f6fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:03:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.156+0000 7f0997e00640 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5598f8f6da10 0x5598f8f6fe30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2658d35e89fe5bb server_cookie=dd453719e137aef9 in_seq=0 out_seq=0 2024-04-25T19:03:50.160 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.156+0000 7f098e000640 10 client.25696 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f098e000640 1 -- 172.21.15.45:0/152673452 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5598f8e5f320 con 0x5598f8f6da10 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f098e000640 10 client.25696 handle_command_reply: tid=0 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f09a9d66740 1 client.25696 shutdown 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f09a9d66740 1 -- 172.21.15.45:0/152673452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 msgr2=0x5598f8efc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f09a9d66740 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8efc5f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0990000c00 tx=0x7f099000c840 comp rx=0 tx=0).stop 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f09a9d66740 1 -- 172.21.15.45:0/152673452 shutdown_connections 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f09a9d66740 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5598f8f6da10 0x5598f8f6fe30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f09a9d66740 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5598f8ef6b90 0x5598f8efd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f09a9d66740 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5598f8eec2d0 0x5598f8efcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f098e000640 0 client.25696 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f09a9d66740 1 --2- 172.21.15.45:0/152673452 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5598f8eeb9b0 0x5598f8efc5f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.182+0000 7f09a9d66740 1 -- 172.21.15.45:0/152673452 >> 172.21.15.45:0/152673452 conn(0x5598f8ee7ec0 msgr2=0x5598f8ee7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.183+0000 7f09a9d66740 1 -- 172.21.15.45:0/152673452 shutdown_connections 2024-04-25T19:03:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.183+0000 7f09a9d66740 1 -- 172.21.15.45:0/152673452 wait complete. 2024-04-25T19:03:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.183+0000 7f09a9d66740 20 client.25696 trim_cache size 0 max 16384 2024-04-25T19:03:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:03:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:03:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:03:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:03:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 -- 172.21.15.45:0/1024617750 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f09840a3d00 msgr2=0x7f09840a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f09840a3d00 0x7f09840a61c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0998005fd0 tx=0x7f0998009500 comp rx=0 tx=0).stop 2024-04-25T19:03:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 -- 172.21.15.45:0/1024617750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a0102fd0 msgr2=0x7f09a01a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:03:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a0102fd0 0x7f09a01a0410 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f099000e9f0 tx=0x7f099000edf0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 -- 172.21.15.45:0/1024617750 shutdown_connections 2024-04-25T19:03:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f09840a3d00 0x7f09840a61c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f09a010c3a0 0x7f09a019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f09a01038f0 0x7f09a01a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 --2- 172.21.15.45:0/1024617750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f09a0102fd0 0x7f09a01a0410 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:03:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 -- 172.21.15.45:0/1024617750 >> 172.21.15.45:0/1024617750 conn(0x7f09a00fc730 msgr2=0x7f09a010d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:03:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 -- 172.21.15.45:0/1024617750 shutdown_connections 2024-04-25T19:03:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:03:50.184+0000 7f09a5a00640 1 -- 172.21.15.45:0/1024617750 wait complete. 2024-04-25T19:03:50.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:50 smithi062 ceph-mon[32844]: pgmap v138: 97 pgs: 97 active+clean; 40 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-04-25T19:03:50.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:50 smithi079 ceph-mon[33002]: pgmap v138: 97 pgs: 97 active+clean; 40 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-04-25T19:03:50.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:50 smithi045 ceph-mon[27776]: pgmap v138: 97 pgs: 97 active+clean; 40 MiB data, 725 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 363 op/s 2024-04-25T19:03:52.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:52 smithi062 ceph-mon[32844]: pgmap v139: 97 pgs: 97 active+clean; 43 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 503 op/s 2024-04-25T19:03:52.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:52 smithi079 ceph-mon[33002]: pgmap v139: 97 pgs: 97 active+clean; 43 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 503 op/s 2024-04-25T19:03:52.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:52 smithi045 ceph-mon[27776]: pgmap v139: 97 pgs: 97 active+clean; 43 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 503 op/s 2024-04-25T19:03:54.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:54 smithi062 ceph-mon[32844]: pgmap v140: 97 pgs: 97 active+clean; 43 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 319 op/s 2024-04-25T19:03:54.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:54 smithi079 ceph-mon[33002]: pgmap v140: 97 pgs: 97 active+clean; 43 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 319 op/s 2024-04-25T19:03:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:54 smithi045 ceph-mon[27776]: pgmap v140: 97 pgs: 97 active+clean; 43 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.4 MiB/s wr, 319 op/s 2024-04-25T19:03:56.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:56 smithi062 ceph-mon[32844]: pgmap v141: 97 pgs: 97 active+clean; 37 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 458 op/s 2024-04-25T19:03:56.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:56 smithi079 ceph-mon[33002]: pgmap v141: 97 pgs: 97 active+clean; 37 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 458 op/s 2024-04-25T19:03:56.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:56 smithi045 ceph-mon[27776]: pgmap v141: 97 pgs: 97 active+clean; 37 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 458 op/s 2024-04-25T19:03:57.318 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0xffdb8 2024-04-25T19:03:58.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:03:58 smithi062 ceph-mon[32844]: pgmap v142: 97 pgs: 97 active+clean; 37 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-04-25T19:03:58.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:03:58 smithi079 ceph-mon[33002]: pgmap v142: 97 pgs: 97 active+clean; 37 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-04-25T19:03:58.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:03:58 smithi045 ceph-mon[27776]: pgmap v142: 97 pgs: 97 active+clean; 37 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-04-25T19:04:00.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:00 smithi062 ceph-mon[32844]: pgmap v143: 97 pgs: 97 active+clean; 37 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-04-25T19:04:00.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:00 smithi079 ceph-mon[33002]: pgmap v143: 97 pgs: 97 active+clean; 37 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-04-25T19:04:00.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:00 smithi045 ceph-mon[27776]: pgmap v143: 97 pgs: 97 active+clean; 37 MiB data, 732 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 279 op/s 2024-04-25T19:04:02.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:02 smithi062 ceph-mon[32844]: pgmap v144: 97 pgs: 97 active+clean; 38 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 426 op/s 2024-04-25T19:04:02.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:02 smithi079 ceph-mon[33002]: pgmap v144: 97 pgs: 97 active+clean; 38 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 426 op/s 2024-04-25T19:04:02.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:02 smithi045 ceph-mon[27776]: pgmap v144: 97 pgs: 97 active+clean; 38 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 426 op/s 2024-04-25T19:04:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.335+0000 7f2183200640 1 -- 172.21.15.45:0/2094205750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c1041d0 msgr2=0x7f217c1045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.335+0000 7f2183200640 1 --2- 172.21.15.45:0/2094205750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c1041d0 0x7f217c1045b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2170009900 tx=0x7f217002a3f0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.335+0000 7f2183200640 1 -- 172.21.15.45:0/2094205750 shutdown_connections 2024-04-25T19:04:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.335+0000 7f2183200640 1 --2- 172.21.15.45:0/2094205750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217c1092a0 0x7f217c110e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.335+0000 7f2183200640 1 --2- 172.21.15.45:0/2094205750 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f217c104b80 0x7f217c108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.335+0000 7f2183200640 1 --2- 172.21.15.45:0/2094205750 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c1041d0 0x7f217c1045b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.335+0000 7f2183200640 1 -- 172.21.15.45:0/2094205750 >> 172.21.15.45:0/2094205750 conn(0x7f217c0ffa30 msgr2=0x7f217c101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.336+0000 7f2183200640 1 -- 172.21.15.45:0/2094205750 shutdown_connections 2024-04-25T19:04:03.338 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.336+0000 7f2183200640 1 -- 172.21.15.45:0/2094205750 wait complete. 2024-04-25T19:04:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.337+0000 7f2183200640 1 Processor -- start 2024-04-25T19:04:03.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.337+0000 7f2183200640 1 -- start start 2024-04-25T19:04:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2183200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c1041d0 0x7f217c1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2183200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217c1092a0 0x7f217c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2183200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f217c19a610 0x7f217c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2183200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f217c114250 con 0x7f217c1092a0 2024-04-25T19:04:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2183200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1140d0 con 0x7f217c1041d0 2024-04-25T19:04:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2183200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1143d0 con 0x7f217c19a610 2024-04-25T19:04:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2181e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c1041d0 0x7f217c1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2181e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c1041d0 0x7f217c1a0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:38214/0 (socket says 172.21.15.45:38214) 2024-04-25T19:04:03.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2181e00640 1 -- 172.21.15.45:0/2538763666 learned_addr learned my addr 172.21.15.45:0/2538763666 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2181400640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217c1092a0 0x7f217c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2182800640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f217c19a610 0x7f217c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2181e00640 1 -- 172.21.15.45:0/2538763666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f217c19a610 msgr2=0x7f217c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2181e00640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f217c19a610 0x7f217c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2181e00640 1 -- 172.21.15.45:0/2538763666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217c1092a0 msgr2=0x7f217c1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2181e00640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217c1092a0 0x7f217c1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.338+0000 7f2181e00640 1 -- 172.21.15.45:0/2538763666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c19b130 con 0x7f217c1041d0 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.339+0000 7f2181e00640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c1041d0 0x7f217c1a0460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2170005ab0 tx=0x7f2170003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.339+0000 7f2176a00640 1 -- 172.21.15.45:0/2538763666 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217003f070 con 0x7f217c1041d0 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.339+0000 7f2176a00640 1 -- 172.21.15.45:0/2538763666 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2170031d60 con 0x7f217c1041d0 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.339+0000 7f2183200640 1 -- 172.21.15.45:0/2538763666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217c19b3c0 con 0x7f217c1041d0 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.339+0000 7f2176a00640 1 -- 172.21.15.45:0/2538763666 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2170035030 con 0x7f217c1041d0 2024-04-25T19:04:03.342 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.339+0000 7f2183200640 1 -- 172.21.15.45:0/2538763666 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217c1a7610 con 0x7f217c1041d0 2024-04-25T19:04:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f218756e740 1 Processor -- start 2024-04-25T19:04:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f2176a00640 1 -- 172.21.15.45:0/2538763666 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2170042070 con 0x7f217c1041d0 2024-04-25T19:04:03.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f218756e740 1 -- start start 2024-04-25T19:04:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f218756e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d43208d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f218756e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d432092d0 0x563d43213650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f218756e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d43213b90 0x563d43215f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f218756e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563d4316e900 con 0x563d432092d0 2024-04-25T19:04:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f218756e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563d43114ef0 con 0x563d43213b90 2024-04-25T19:04:03.346 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f218756e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563d4316a8c0 con 0x563d432089b0 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f2182800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d43208d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f2182800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d43208d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:36368/0 (socket says 172.21.15.45:36368) 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.343+0000 7f2182800640 1 -- 172.21.15.45:0/2613984371 learned_addr learned my addr 172.21.15.45:0/2613984371 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.344+0000 7f2181e00640 1 --2- 172.21.15.45:0/2613984371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d432092d0 0x563d43213650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.344+0000 7f2181400640 1 --2- 172.21.15.45:0/2613984371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d43213b90 0x563d43215f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.344+0000 7f2182800640 1 -- 172.21.15.45:0/2613984371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d43213b90 msgr2=0x563d43215f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.344+0000 7f2182800640 1 --2- 172.21.15.45:0/2613984371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d43213b90 0x563d43215f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.344+0000 7f2182800640 1 -- 172.21.15.45:0/2613984371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d432092d0 msgr2=0x563d43213650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.344+0000 7f2182800640 1 --2- 172.21.15.45:0/2613984371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d432092d0 0x563d43213650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.344+0000 7f2182800640 1 -- 172.21.15.45:0/2613984371 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d42e81210 con 0x563d432089b0 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.344+0000 7f2181400640 1 --2- 172.21.15.45:0/2613984371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d43213b90 0x563d43215f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:03.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.344+0000 7f2182800640 1 --2- 172.21.15.45:0/2613984371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d43208d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2178009f90 tx=0x7f2178010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb3b21be415fbe58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.345+0000 7f216b000640 1 -- 172.21.15.45:0/2613984371 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178011700 con 0x563d432089b0 2024-04-25T19:04:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.345+0000 7f216b000640 1 -- 172.21.15.45:0/2613984371 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2178011d50 con 0x563d432089b0 2024-04-25T19:04:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.345+0000 7f216b000640 1 -- 172.21.15.45:0/2613984371 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178019d40 con 0x563d432089b0 2024-04-25T19:04:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.345+0000 7f2176a00640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21640a3d80 0x7f21640a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.345+0000 7f2181e00640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21640a3d80 0x7f21640a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.345+0000 7f2176a00640 1 -- 172.21.15.45:0/2538763666 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f21700f0bb0 con 0x7f217c1041d0 2024-04-25T19:04:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.345+0000 7f218756e740 1 -- 172.21.15.45:0/2613984371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 msgr2=0x563d43208d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.345+0000 7f218756e740 1 --2- 172.21.15.45:0/2613984371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d43208d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2178009f90 tx=0x7f2178010860 comp rx=0 tx=0).stop 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.346+0000 7f218756e740 1 -- 172.21.15.45:0/2613984371 shutdown_connections 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.346+0000 7f218756e740 1 --2- 172.21.15.45:0/2613984371 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d43213b90 0x563d43215f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.346+0000 7f218756e740 1 --2- 172.21.15.45:0/2613984371 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d432092d0 0x563d43213650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.346+0000 7f218756e740 1 --2- 172.21.15.45:0/2613984371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d43208d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.346+0000 7f218756e740 1 -- 172.21.15.45:0/2613984371 >> 172.21.15.45:0/2613984371 conn(0x563d43204ec0 msgr2=0x563d432052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.346+0000 7f218756e740 1 -- 172.21.15.45:0/2613984371 shutdown_connections 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.346+0000 7f218756e740 1 -- 172.21.15.45:0/2613984371 wait complete. 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.346+0000 7f2181e00640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21640a3d80 0x7f21640a6240 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x563d43209f50 tx=0x7f21700f4660 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.346+0000 7f218756e740 1 -- start start 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.347+0000 7f218756e740 1 Processor -- start 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.347+0000 7f218756e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d43210ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.347+0000 7f218756e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43211010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.347+0000 7f218756e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d43211550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.347+0000 7f218756e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563d432167c0 con 0x563d43213b90 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.347+0000 7f218756e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x563d43128310 con 0x563d432092d0 2024-04-25T19:04:03.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.347+0000 7f218756e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x563d43216940 con 0x563d432089b0 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.347+0000 7f218756e740 10 client.0 _ll_register_callbacks cb 0x563d430f6cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2182800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d43210ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2182800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d43211550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2181400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43211010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2181400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43211010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:38242/0 (socket says 172.21.15.45:38242) 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2181400640 1 -- 172.21.15.45:0/3377989663 learned_addr learned my addr 172.21.15.45:0/3377989663 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2181400640 1 -- 172.21.15.45:0/3377989663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 msgr2=0x563d43210ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2181400640 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d43210ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2181400640 1 -- 172.21.15.45:0/3377989663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 msgr2=0x563d43211550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2181400640 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d43211550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2181400640 1 -- 172.21.15.45:0/3377989663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d42f4d620 con 0x563d432092d0 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f2181400640 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43211010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f216c0126d0 tx=0x7f216c012ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df59649b4e6644f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.348+0000 7f215f400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.349+0000 7f215f400640 1 -- 172.21.15.45:0/3377989663 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c004050 con 0x563d432092d0 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.349+0000 7f215f400640 1 -- 172.21.15.45:0/3377989663 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f216c0041f0 con 0x563d432092d0 2024-04-25T19:04:03.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.349+0000 7f218756e740 1 -- 172.21.15.45:0/3377989663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2178011d50 con 0x563d432092d0 2024-04-25T19:04:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.349+0000 7f215f400640 1 -- 172.21.15.45:0/3377989663 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f216c0043f0 con 0x563d432092d0 2024-04-25T19:04:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.349+0000 7f218756e740 10 client.25688 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.349+0000 7f218756e740 1 -- 172.21.15.45:0/3377989663 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d43222a70 con 0x563d432092d0 2024-04-25T19:04:03.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.349+0000 7f215f400640 1 -- 172.21.15.45:0/3377989663 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2178011d50 con 0x563d432092d0 2024-04-25T19:04:03.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.349+0000 7f215f400640 1 -- 172.21.15.45:0/3377989663 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f216c00f870 con 0x563d432092d0 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.350+0000 7f218756e740 10 client.25688 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.350+0000 7f218756e740 10 client.25688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.350+0000 7f218756e740 20 client.25688 populate_metadata read hostname 'smithi045' 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.350+0000 7f218756e740 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563d43223f80 0x563d432263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.350+0000 7f218756e740 4 client.25688 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.350+0000 7f218756e740 1 -- 172.21.15.45:0/3377989663 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d431828b0 con 0x563d43223f80 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.350+0000 7f2182800640 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563d43223f80 0x563d432263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.350+0000 7f2182800640 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563d43223f80 0x563d432263b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810963f09add44ec server_cookie=bf0efa5ae473c079 in_seq=0 out_seq=0 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.350+0000 7f215f400640 10 client.25688 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f215f400640 1 -- 172.21.15.45:0/3377989663 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d431828b0 con 0x563d43223f80 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f215f400640 10 client.25688 handle_command_reply: tid=0 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 client.25688 shutdown 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 -- 172.21.15.45:0/3377989663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 msgr2=0x563d43211010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43211010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f216c0126d0 tx=0x7f216c012ad0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 -- 172.21.15.45:0/3377989663 shutdown_connections 2024-04-25T19:04:03.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563d43223f80 0x563d432263b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d43211550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43211010 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 --2- 172.21.15.45:0/3377989663 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d43210ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 -- 172.21.15.45:0/3377989663 >> 172.21.15.45:0/3377989663 conn(0x563d43204ec0 msgr2=0x563d4320ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f215f400640 0 client.25688 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 -- 172.21.15.45:0/3377989663 shutdown_connections 2024-04-25T19:04:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 1 -- 172.21.15.45:0/3377989663 wait complete. 2024-04-25T19:04:03.357 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.352+0000 7f218756e740 20 client.25688 trim_cache size 0 max 16384 2024-04-25T19:04:03.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.369+0000 7f218756e740 1 Processor -- start 2024-04-25T19:04:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.369+0000 7f218756e740 1 -- start start 2024-04-25T19:04:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f218756e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432089b0 0x563d4321f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f218756e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432092d0 0x563d4320b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f218756e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d4322bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f218756e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1b2580 con 0x563d43213b90 2024-04-25T19:04:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f218756e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1afe50 con 0x563d432089b0 2024-04-25T19:04:03.372 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f218756e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1adca0 con 0x563d432092d0 2024-04-25T19:04:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2182800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d4322bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2181400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432092d0 0x563d4320b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2182800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d4322bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:39834/0 (socket says 172.21.15.45:39834) 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2182800640 1 -- 172.21.15.45:0/281755172 learned_addr learned my addr 172.21.15.45:0/281755172 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2182800640 1 --2- 172.21.15.45:0/281755172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432089b0 0x563d4321f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2182800640 1 -- 172.21.15.45:0/281755172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432092d0 msgr2=0x563d4320b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2182800640 1 --2- 172.21.15.45:0/281755172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432092d0 0x563d4320b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2182800640 1 -- 172.21.15.45:0/281755172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432089b0 msgr2=0x563d4321f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2182800640 1 --2- 172.21.15.45:0/281755172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432089b0 0x563d4321f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2182800640 1 -- 172.21.15.45:0/281755172 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4320b8b0 con 0x563d43213b90 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.370+0000 7f2181400640 1 --2- 172.21.15.45:0/281755172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432092d0 0x563d4320b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.371+0000 7f2182800640 1 --2- 172.21.15.45:0/281755172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d4322bc80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f21780196b0 tx=0x7f21780096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad6098900c0088a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.371+0000 7f216a600640 1 -- 172.21.15.45:0/281755172 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217801f040 con 0x563d43213b90 2024-04-25T19:04:03.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.371+0000 7f216a600640 1 -- 172.21.15.45:0/281755172 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2178019da0 con 0x563d43213b90 2024-04-25T19:04:03.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.371+0000 7f216a600640 1 -- 172.21.15.45:0/281755172 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21780118f0 con 0x563d43213b90 2024-04-25T19:04:03.377 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.372+0000 7f218756e740 1 -- 172.21.15.45:0/281755172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 msgr2=0x563d4322bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.372+0000 7f218756e740 1 --2- 172.21.15.45:0/281755172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d4322bc80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f21780196b0 tx=0x7f21780096a0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.372+0000 7f218756e740 1 -- 172.21.15.45:0/281755172 shutdown_connections 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.372+0000 7f218756e740 1 --2- 172.21.15.45:0/281755172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d4322bc80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.372+0000 7f218756e740 1 --2- 172.21.15.45:0/281755172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432092d0 0x563d4320b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.372+0000 7f218756e740 1 --2- 172.21.15.45:0/281755172 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432089b0 0x563d4321f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.372+0000 7f218756e740 1 -- 172.21.15.45:0/281755172 >> 172.21.15.45:0/281755172 conn(0x563d43204ec0 msgr2=0x563d43211320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.372+0000 7f218756e740 1 -- 172.21.15.45:0/281755172 shutdown_connections 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.372+0000 7f218756e740 1 -- 172.21.15.45:0/281755172 wait complete. 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.372+0000 7f218756e740 1 -- start start 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f218756e740 1 Processor -- start 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f218756e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d4321bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f218756e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43219180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f218756e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d432196c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f218756e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1ae310 con 0x563d43213b90 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f218756e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1af780 con 0x563d432092d0 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f218756e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1ae9e0 con 0x563d432089b0 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f218756e740 10 client.0 _ll_register_callbacks cb 0x7f216c009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f2182800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d4321bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f2182800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d4321bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:36394/0 (socket says 172.21.15.45:36394) 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f2182800640 1 -- 172.21.15.45:0/1639757702 learned_addr learned my addr 172.21.15.45:0/1639757702 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.373+0000 7f2182800640 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d432196c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.374+0000 7f2181400640 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43219180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.374+0000 7f2182800640 1 -- 172.21.15.45:0/1639757702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 msgr2=0x563d43219180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.374+0000 7f2182800640 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43219180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.374+0000 7f2182800640 1 -- 172.21.15.45:0/1639757702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 msgr2=0x563d432196c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.374+0000 7f2182800640 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d432196c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.374+0000 7f2182800640 1 -- 172.21.15.45:0/1639757702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d43219d60 con 0x563d432089b0 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.374+0000 7f2181400640 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43219180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.374+0000 7f2182800640 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d4321bdf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2178002230 tx=0x7f2178019f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f280d904086a7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.374+0000 7f215f400640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.374+0000 7f215f400640 1 -- 172.21.15.45:0/1639757702 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178019450 con 0x563d432089b0 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f218756e740 1 -- 172.21.15.45:0/1639757702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2178019da0 con 0x563d432089b0 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f215f400640 1 -- 172.21.15.45:0/1639757702 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f2178003970 con 0x563d432089b0 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f215f400640 1 -- 172.21.15.45:0/1639757702 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2178003b70 con 0x563d432089b0 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f218756e740 10 client.25714 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f218756e740 1 -- 172.21.15.45:0/1639757702 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d4328a3a0 con 0x563d432089b0 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f215f400640 1 -- 172.21.15.45:0/1639757702 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2178019da0 con 0x563d432089b0 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f215f400640 1 -- 172.21.15.45:0/1639757702 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f217802d140 con 0x563d432089b0 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f218756e740 10 client.25714 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f218756e740 10 client.25714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f218756e740 20 client.25714 populate_metadata read hostname 'smithi045' 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f218756e740 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563d4328a9c0 0x563d4328cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:03.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f218756e740 4 client.25714 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:04:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.375+0000 7f218756e740 1 -- 172.21.15.45:0/1639757702 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d4317c320 con 0x563d4328a9c0 2024-04-25T19:04:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.376+0000 7f2181400640 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563d4328a9c0 0x563d4328cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.376+0000 7f2181400640 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563d4328a9c0 0x563d4328cde0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6f0d77c5fe66c74 server_cookie=6e3073b625394de1 in_seq=0 out_seq=0 2024-04-25T19:04:03.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.376+0000 7f215f400640 10 client.25714 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f215f400640 1 -- 172.21.15.45:0/1639757702 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d4317c320 con 0x563d4328a9c0 2024-04-25T19:04:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f215f400640 10 client.25714 handle_command_reply: tid=0 2024-04-25T19:04:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 client.25714 shutdown 2024-04-25T19:04:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 -- 172.21.15.45:0/1639757702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 msgr2=0x563d4321bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d4321bdf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2178002230 tx=0x7f2178019f50 comp rx=0 tx=0).stop 2024-04-25T19:04:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 -- 172.21.15.45:0/1639757702 shutdown_connections 2024-04-25T19:04:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563d4328a9c0 0x563d4328cde0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d43213b90 0x563d432196c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f215f400640 0 client.25714 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x563d432092d0 0x563d43219180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 --2- 172.21.15.45:0/1639757702 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x563d432089b0 0x563d4321bdf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 -- 172.21.15.45:0/1639757702 >> 172.21.15.45:0/1639757702 conn(0x563d43204ec0 msgr2=0x563d43203790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 -- 172.21.15.45:0/1639757702 shutdown_connections 2024-04-25T19:04:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.394+0000 7f218756e740 1 -- 172.21.15.45:0/1639757702 wait complete. 2024-04-25T19:04:03.397 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.395+0000 7f218756e740 20 client.25714 trim_cache size 0 max 16384 2024-04-25T19:04:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:04:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:04:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:04:03.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 -- 172.21.15.45:0/2538763666 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21640a3d80 msgr2=0x7f21640a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21640a3d80 0x7f21640a6240 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x563d43209f50 tx=0x7f21700f4660 comp rx=0 tx=0).stop 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 -- 172.21.15.45:0/2538763666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c1041d0 msgr2=0x7f217c1a0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c1041d0 0x7f217c1a0460 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2170005ab0 tx=0x7f2170003c80 comp rx=0 tx=0).stop 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 -- 172.21.15.45:0/2538763666 shutdown_connections 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21640a3d80 0x7f21640a6240 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f217c19a610 0x7f217c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f217c1092a0 0x7f217c1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 --2- 172.21.15.45:0/2538763666 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f217c1041d0 0x7f217c1a0460 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 -- 172.21.15.45:0/2538763666 >> 172.21.15.45:0/2538763666 conn(0x7f217c0ffa30 msgr2=0x7f217c100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 -- 172.21.15.45:0/2538763666 shutdown_connections 2024-04-25T19:04:03.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:03.396+0000 7f2183200640 1 -- 172.21.15.45:0/2538763666 wait complete. 2024-04-25T19:04:04.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:04 smithi062 ceph-mon[32844]: pgmap v145: 97 pgs: 97 active+clean; 38 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-04-25T19:04:04.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:04 smithi079 ceph-mon[33002]: pgmap v145: 97 pgs: 97 active+clean; 38 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-04-25T19:04:04.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:04 smithi045 ceph-mon[27776]: pgmap v145: 97 pgs: 97 active+clean; 38 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.8 MiB/s wr, 286 op/s 2024-04-25T19:04:06.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:06 smithi062 ceph-mon[32844]: pgmap v146: 97 pgs: 97 active+clean; 37 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 457 op/s 2024-04-25T19:04:06.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:06 smithi079 ceph-mon[33002]: pgmap v146: 97 pgs: 97 active+clean; 37 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 457 op/s 2024-04-25T19:04:06.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:06 smithi045 ceph-mon[27776]: pgmap v146: 97 pgs: 97 active+clean; 37 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 457 op/s 2024-04-25T19:04:08.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:08 smithi062 ceph-mon[32844]: pgmap v147: 97 pgs: 97 active+clean; 37 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-25T19:04:08.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:08 smithi079 ceph-mon[33002]: pgmap v147: 97 pgs: 97 active+clean; 37 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-25T19:04:08.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:08 smithi045 ceph-mon[27776]: pgmap v147: 97 pgs: 97 active+clean; 37 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-25T19:04:08.775 INFO:tasks.workunit.client.0.smithi045.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-04-25T19:04:08.776 INFO:tasks.workunit.client.0.smithi045.stdout:All 50000 operations completed A-OK! 2024-04-25T19:04:08.776 INFO:tasks.workunit.client.0.smithi045.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-04-25T19:04:08.779 INFO:tasks.workunit.client.0.smithi045.stdout:Seed set to 1 2024-04-25T19:04:08.942 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T19:04:08.943 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T19:04:08.946 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T19:04:08.946 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T19:04:08.946 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T19:04:08.950 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T19:04:08.954 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T19:04:08.955 INFO:tasks.workunit.client.0.smithi045.stdout:skipping zero length punch hole 2024-04-25T19:04:08.955 INFO:tasks.workunit.client.0.smithi045.stdout:skipping zero length punch hole 2024-04-25T19:04:08.955 INFO:tasks.workunit.client.0.smithi045.stdout:skipping zero length punch hole 2024-04-25T19:04:08.955 INFO:tasks.workunit.client.0.smithi045.stdout:skipping zero size read 2024-04-25T19:04:08.955 INFO:tasks.workunit.client.0.smithi045.stdout:skipping zero length punch hole 2024-04-25T19:04:08.955 INFO:tasks.workunit.client.0.smithi045.stdout:skipping zero size read 2024-04-25T19:04:08.955 INFO:tasks.workunit.client.0.smithi045.stdout:skipping zero length punch hole 2024-04-25T19:04:08.955 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0x4310e9 2024-04-25T19:04:08.969 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0x954a45 2024-04-25T19:04:09.033 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0x9bc582 2024-04-25T19:04:09.088 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0x9eecbc 2024-04-25T19:04:10.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:10 smithi062 ceph-mon[32844]: pgmap v148: 97 pgs: 97 active+clean; 37 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-25T19:04:10.433 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:10 smithi079 ceph-mon[33002]: pgmap v148: 97 pgs: 97 active+clean; 37 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-25T19:04:10.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:10 smithi045 ceph-mon[27776]: pgmap v148: 97 pgs: 97 active+clean; 37 MiB data, 780 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-25T19:04:12.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:12 smithi062 ceph-mon[32844]: pgmap v149: 97 pgs: 97 active+clean; 43 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 452 op/s 2024-04-25T19:04:12.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:12 smithi079 ceph-mon[33002]: pgmap v149: 97 pgs: 97 active+clean; 43 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 452 op/s 2024-04-25T19:04:12.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:12 smithi045 ceph-mon[27776]: pgmap v149: 97 pgs: 97 active+clean; 43 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.7 MiB/s wr, 452 op/s 2024-04-25T19:04:13.525 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0x9fdd84 2024-04-25T19:04:14.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:14 smithi062 ceph-mon[32844]: pgmap v150: 97 pgs: 97 active+clean; 43 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 305 op/s 2024-04-25T19:04:14.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:14 smithi079 ceph-mon[33002]: pgmap v150: 97 pgs: 97 active+clean; 43 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 305 op/s 2024-04-25T19:04:14.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:14 smithi045 ceph-mon[27776]: pgmap v150: 97 pgs: 97 active+clean; 43 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 305 op/s 2024-04-25T19:04:16.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:16 smithi062 ceph-mon[32844]: pgmap v151: 97 pgs: 97 active+clean; 43 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.6 MiB/s wr, 461 op/s 2024-04-25T19:04:16.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:16 smithi079 ceph-mon[33002]: pgmap v151: 97 pgs: 97 active+clean; 43 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.6 MiB/s wr, 461 op/s 2024-04-25T19:04:16.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:16 smithi045 ceph-mon[27776]: pgmap v151: 97 pgs: 97 active+clean; 43 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.6 MiB/s wr, 461 op/s 2024-04-25T19:04:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.552+0000 7f3867200640 1 -- 172.21.15.45:0/2557566358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3860107280 msgr2=0x7f386010ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.552+0000 7f3867200640 1 --2- 172.21.15.45:0/2557566358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3860107280 0x7f386010ee50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3858009900 tx=0x7f385802a3f0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.553+0000 7f3867200640 1 -- 172.21.15.45:0/2557566358 shutdown_connections 2024-04-25T19:04:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.553+0000 7f3867200640 1 --2- 172.21.15.45:0/2557566358 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3860107280 0x7f386010ee50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.553+0000 7f3867200640 1 --2- 172.21.15.45:0/2557566358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3860102b60 0x7f3860106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.553+0000 7f3867200640 1 --2- 172.21.15.45:0/2557566358 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f38601021b0 0x7f3860102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.553+0000 7f3867200640 1 -- 172.21.15.45:0/2557566358 >> 172.21.15.45:0/2557566358 conn(0x7f38600fda10 msgr2=0x7f38600ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.553+0000 7f3867200640 1 -- 172.21.15.45:0/2557566358 shutdown_connections 2024-04-25T19:04:16.555 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.553+0000 7f3867200640 1 -- 172.21.15.45:0/2557566358 wait complete. 2024-04-25T19:04:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.554+0000 7f3867200640 1 Processor -- start 2024-04-25T19:04:16.556 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.554+0000 7f3867200640 1 -- start start 2024-04-25T19:04:16.557 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.554+0000 7f3867200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38601021b0 0x7f38601a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f3867200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3860107280 0x7f38601a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f3867200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386019ac00 0x7f386019b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f3867200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38600748e0 con 0x7f3860107280 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f3867200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3860074760 con 0x7f386019ac00 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f3867200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3860074a60 con 0x7f38601021b0 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f385fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38601021b0 0x7f38601a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f385fe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38601021b0 0x7f38601a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41652/0 (socket says 172.21.15.45:41652) 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f385fe00640 1 -- 172.21.15.45:0/1990556546 learned_addr learned my addr 172.21.15.45:0/1990556546 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f3865200640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386019ac00 0x7f386019b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f3857400640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3860107280 0x7f38601a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f385fe00640 1 -- 172.21.15.45:0/1990556546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386019ac00 msgr2=0x7f386019b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f385fe00640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386019ac00 0x7f386019b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.558 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f385fe00640 1 -- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3860107280 msgr2=0x7f38601a0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f385fe00640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3860107280 0x7f38601a0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f385fe00640 1 -- 172.21.15.45:0/1990556546 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386019b7b0 con 0x7f38601021b0 2024-04-25T19:04:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.555+0000 7f3857400640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3860107280 0x7f38601a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:16.559 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.556+0000 7f385fe00640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38601021b0 0x7f38601a0670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3850009d50 tx=0x7f385000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.556+0000 7f385d600640 1 -- 172.21.15.45:0/1990556546 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3850003b70 con 0x7f38601021b0 2024-04-25T19:04:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.556+0000 7f3867200640 1 -- 172.21.15.45:0/1990556546 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38601a7430 con 0x7f38601021b0 2024-04-25T19:04:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.556+0000 7f385d600640 1 -- 172.21.15.45:0/1990556546 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3850003d10 con 0x7f38601021b0 2024-04-25T19:04:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.556+0000 7f385d600640 1 -- 172.21.15.45:0/1990556546 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38500108b0 con 0x7f38601021b0 2024-04-25T19:04:16.560 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.556+0000 7f3867200640 1 -- 172.21.15.45:0/1990556546 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38601a7830 con 0x7f38601021b0 2024-04-25T19:04:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.559+0000 7f385d600640 1 -- 172.21.15.45:0/1990556546 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3850004000 con 0x7f38601021b0 2024-04-25T19:04:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.560+0000 7f3869f91740 1 Processor -- start 2024-04-25T19:04:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.560+0000 7f3869f91740 1 -- start start 2024-04-25T19:04:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.560+0000 7f3869f91740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.560+0000 7f3869f91740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9bb140 0x5646af9c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.560+0000 7f3869f91740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9c5a00 0x5646af9c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.560+0000 7f3869f91740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5646af920900 con 0x5646af9c5a00 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.560+0000 7f3869f91740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5646af8c6ef0 con 0x5646af9bb140 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.560+0000 7f3869f91740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5646af91c8c0 con 0x5646af9ba820 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f385fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9bb140 0x5646af9c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f3865200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f3865200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9bac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41666/0 (socket says 172.21.15.45:41666) 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f385fe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9bb140 0x5646af9c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:51318/0 (socket says 172.21.15.45:51318) 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f3865200640 1 -- 172.21.15.45:0/4237622318 learned_addr learned my addr 172.21.15.45:0/4237622318 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f3857400640 1 --2- 172.21.15.45:0/4237622318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9c5a00 0x5646af9c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f3865200640 1 -- 172.21.15.45:0/4237622318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9bb140 msgr2=0x5646af9c54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f3865200640 1 --2- 172.21.15.45:0/4237622318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9bb140 0x5646af9c54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f3865200640 1 -- 172.21.15.45:0/4237622318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9c5a00 msgr2=0x5646af9c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f3865200640 1 --2- 172.21.15.45:0/4237622318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9c5a00 0x5646af9c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f3865200640 1 -- 172.21.15.45:0/4237622318 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646af633210 con 0x5646af9ba820 2024-04-25T19:04:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.561+0000 7f385fe00640 1 --2- 172.21.15.45:0/4237622318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9bb140 0x5646af9c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.562+0000 7f3865200640 1 --2- 172.21.15.45:0/4237622318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9bac00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f385802a9e0 tx=0x7f3858005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=314f8f04772f5aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.562+0000 7f385d600640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f38440a3d10 0x7f38440a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.562+0000 7f3855600640 1 -- 172.21.15.45:0/4237622318 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858040070 con 0x5646af9ba820 2024-04-25T19:04:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.562+0000 7f3855600640 1 -- 172.21.15.45:0/4237622318 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38580044c0 con 0x5646af9ba820 2024-04-25T19:04:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.562+0000 7f3855600640 1 -- 172.21.15.45:0/4237622318 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3858035040 con 0x5646af9ba820 2024-04-25T19:04:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.562+0000 7f385d600640 1 -- 172.21.15.45:0/1990556546 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f38500d0f00 con 0x7f38601021b0 2024-04-25T19:04:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.562+0000 7f385fe00640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f38440a3d10 0x7f38440a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.563+0000 7f3869f91740 1 -- 172.21.15.45:0/4237622318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 msgr2=0x5646af9bac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.563+0000 7f3869f91740 1 --2- 172.21.15.45:0/4237622318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9bac00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f385802a9e0 tx=0x7f3858005be0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.563+0000 7f385fe00640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f38440a3d10 0x7f38440a61d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5646af9bbdc0 tx=0x7f38500d49b0 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.563+0000 7f3869f91740 1 -- 172.21.15.45:0/4237622318 shutdown_connections 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.563+0000 7f3869f91740 1 --2- 172.21.15.45:0/4237622318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9c5a00 0x5646af9c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.563+0000 7f3869f91740 1 --2- 172.21.15.45:0/4237622318 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9bb140 0x5646af9c54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.563+0000 7f3869f91740 1 --2- 172.21.15.45:0/4237622318 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9bac00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.563+0000 7f3869f91740 1 -- 172.21.15.45:0/4237622318 >> 172.21.15.45:0/4237622318 conn(0x5646af9b6d30 msgr2=0x5646af9b7110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.563+0000 7f3869f91740 1 -- 172.21.15.45:0/4237622318 shutdown_connections 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.563+0000 7f3869f91740 1 -- 172.21.15.45:0/4237622318 wait complete. 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.564+0000 7f3869f91740 1 -- start start 2024-04-25T19:04:16.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.564+0000 7f3869f91740 1 Processor -- start 2024-04-25T19:04:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.564+0000 7f3869f91740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9ba820 0x5646af9c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.564+0000 7f3869f91740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.565+0000 7f3869f91740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9c5a00 0x5646af9c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.565+0000 7f3869f91740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5646af9c85d0 con 0x5646af9bb140 2024-04-25T19:04:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.565+0000 7f3869f91740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5646af8da310 con 0x5646af9ba820 2024-04-25T19:04:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.565+0000 7f3869f91740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5646af9c8750 con 0x5646af9c5a00 2024-04-25T19:04:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.565+0000 7f3869f91740 10 client.0 _ll_register_callbacks cb 0x5646af8a8cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.565+0000 7f3857400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.565+0000 7f3857400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9c2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42060/0 (socket says 172.21.15.45:42060) 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.565+0000 7f3857400640 1 -- 172.21.15.45:0/3530498120 learned_addr learned my addr 172.21.15.45:0/3530498120 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.565+0000 7f3865200640 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9ba820 0x5646af9c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.565+0000 7f3865200640 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9c5a00 0x5646af9c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f3857400640 1 -- 172.21.15.45:0/3530498120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9c5a00 msgr2=0x5646af9c34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f3857400640 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9c5a00 0x5646af9c34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f3857400640 1 -- 172.21.15.45:0/3530498120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9ba820 msgr2=0x5646af9c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f3857400640 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9ba820 0x5646af9c2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f3857400640 1 -- 172.21.15.45:0/3530498120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646af6ff620 con 0x5646af9bb140 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f3865200640 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9c5a00 0x5646af9c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f3857400640 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9c2f70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3848013510 tx=0x7f38480139d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08cefbba76e0794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f384e000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f384e000640 1 -- 172.21.15.45:0/3530498120 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848011990 con 0x5646af9bb140 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f384e000640 1 -- 172.21.15.45:0/3530498120 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f3848011b30 con 0x5646af9bb140 2024-04-25T19:04:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.566+0000 7f3869f91740 1 -- 172.21.15.45:0/3530498120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646af9348b0 con 0x5646af9bb140 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.567+0000 7f384e000640 1 -- 172.21.15.45:0/3530498120 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3848011d70 con 0x5646af9bb140 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.567+0000 7f3869f91740 10 client.15570 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.567+0000 7f3869f91740 1 -- 172.21.15.45:0/3530498120 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646af9d4920 con 0x5646af9bb140 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.567+0000 7f384e000640 1 -- 172.21.15.45:0/3530498120 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5646af9348b0 con 0x5646af9bb140 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.568+0000 7f384e000640 1 -- 172.21.15.45:0/3530498120 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f384801d480 con 0x5646af9bb140 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.568+0000 7f3869f91740 10 client.15570 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.568+0000 7f3869f91740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.568+0000 7f3869f91740 20 client.15570 populate_metadata read hostname 'smithi045' 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.568+0000 7f3869f91740 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5646af9d5e30 0x5646af9d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.568+0000 7f3869f91740 4 client.15570 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.568+0000 7f3869f91740 1 -- 172.21.15.45:0/3530498120 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646af92e320 con 0x5646af9d5e30 2024-04-25T19:04:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.568+0000 7f3865200640 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5646af9d5e30 0x5646af9d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.569+0000 7f3865200640 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5646af9d5e30 0x5646af9d82a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b8db3d41a29ccb server_cookie=30f8023f90d7da62 in_seq=0 out_seq=0 2024-04-25T19:04:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.569+0000 7f384e000640 10 client.15570 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f384e000640 1 -- 172.21.15.45:0/3530498120 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646af92e320 con 0x5646af9d5e30 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f384e000640 10 client.15570 handle_command_reply: tid=0 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f3869f91740 1 client.15570 shutdown 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f3869f91740 1 -- 172.21.15.45:0/3530498120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 msgr2=0x5646af9c2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f3869f91740 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9c2f70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3848013510 tx=0x7f38480139d0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f3869f91740 1 -- 172.21.15.45:0/3530498120 shutdown_connections 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f3869f91740 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5646af9d5e30 0x5646af9d82a0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f3869f91740 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9c5a00 0x5646af9c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f3869f91740 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9c2f70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f3869f91740 1 --2- 172.21.15.45:0/3530498120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9ba820 0x5646af9c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f384e000640 0 client.15570 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f3869f91740 1 -- 172.21.15.45:0/3530498120 >> 172.21.15.45:0/3530498120 conn(0x5646af9b6d30 msgr2=0x5646af9bca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.570+0000 7f3869f91740 1 -- 172.21.15.45:0/3530498120 shutdown_connections 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.571+0000 7f3869f91740 1 -- 172.21.15.45:0/3530498120 wait complete. 2024-04-25T19:04:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.571+0000 7f3869f91740 20 client.15570 trim_cache size 0 max 16384 2024-04-25T19:04:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.588+0000 7f3869f91740 1 Processor -- start 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.588+0000 7f3869f91740 1 -- start start 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.588+0000 7f3869f91740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9ba820 0x5646af9d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.588+0000 7f3869f91740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9bb140 0x5646af9ddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.588+0000 7f3869f91740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 0x5646af9de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.588+0000 7f3869f91740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38601b2730 con 0x5646af9ba820 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.588+0000 7f3869f91740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f38601b0000 con 0x5646af9c5a00 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.588+0000 7f3869f91740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f38601ade50 con 0x5646af9bb140 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3857400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9bb140 0x5646af9ddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3857400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9bb140 0x5646af9ddb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:41686/0 (socket says 172.21.15.45:41686) 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3857400640 1 -- 172.21.15.45:0/391533147 learned_addr learned my addr 172.21.15.45:0/391533147 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3865200640 1 --2- 172.21.15.45:0/391533147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9ba820 0x5646af9d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3865200640 1 --2- 172.21.15.45:0/391533147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 0x5646af9de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3857400640 1 -- 172.21.15.45:0/391533147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 msgr2=0x5646af9de0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3857400640 1 --2- 172.21.15.45:0/391533147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 0x5646af9de0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3857400640 1 -- 172.21.15.45:0/391533147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9ba820 msgr2=0x5646af9d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3857400640 1 --2- 172.21.15.45:0/391533147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9ba820 0x5646af9d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3857400640 1 -- 172.21.15.45:0/391533147 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646af9b6610 con 0x5646af9bb140 2024-04-25T19:04:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3865200640 1 --2- 172.21.15.45:0/391533147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 0x5646af9de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.589+0000 7f3857400640 1 --2- 172.21.15.45:0/391533147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9bb140 0x5646af9ddb70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3848000c00 tx=0x7f3848011f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9e6496dd452857 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.590+0000 7f3854c00640 1 -- 172.21.15.45:0/391533147 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384801d3f0 con 0x5646af9bb140 2024-04-25T19:04:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.590+0000 7f3854c00640 1 -- 172.21.15.45:0/391533147 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f384801d590 con 0x5646af9bb140 2024-04-25T19:04:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.590+0000 7f3854c00640 1 -- 172.21.15.45:0/391533147 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3848005360 con 0x5646af9bb140 2024-04-25T19:04:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 -- 172.21.15.45:0/391533147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9bb140 msgr2=0x5646af9ddb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 --2- 172.21.15.45:0/391533147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9bb140 0x5646af9ddb70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3848000c00 tx=0x7f3848011f10 comp rx=0 tx=0).stop 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 -- 172.21.15.45:0/391533147 shutdown_connections 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 --2- 172.21.15.45:0/391533147 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 0x5646af9de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 --2- 172.21.15.45:0/391533147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9bb140 0x5646af9ddb70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 --2- 172.21.15.45:0/391533147 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9ba820 0x5646af9d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 -- 172.21.15.45:0/391533147 >> 172.21.15.45:0/391533147 conn(0x5646af9b6d30 msgr2=0x5646af9bdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 -- 172.21.15.45:0/391533147 shutdown_connections 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 -- 172.21.15.45:0/391533147 wait complete. 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 -- start start 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.591+0000 7f3869f91740 1 Processor -- start 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3869f91740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3869f91740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3869f91740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 0x5646af9cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3869f91740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38601ae4c0 con 0x5646af9bb140 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3869f91740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f38601af930 con 0x5646af9c5a00 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3869f91740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f38601aeb90 con 0x5646af9ba820 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3869f91740 10 client.0 _ll_register_callbacks cb 0x7f3848009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3857400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3865200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3857400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:42064/0 (socket says 172.21.15.45:42064) 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3857400640 1 -- 172.21.15.45:0/333586627 learned_addr learned my addr 172.21.15.45:0/333586627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.592+0000 7f3865200640 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 0x5646af9cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f3857400640 1 -- 172.21.15.45:0/333586627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 msgr2=0x5646af9cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f3857400640 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f3857400640 1 -- 172.21.15.45:0/333586627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 msgr2=0x5646af9cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f3857400640 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 0x5646af9cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f3857400640 1 -- 172.21.15.45:0/333586627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646af9cb650 con 0x5646af9bb140 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f3865200640 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f3857400640 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9cde40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f384801d8a0 tx=0x7f3848011450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e91649d7c857819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f384e000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f384e000640 1 -- 172.21.15.45:0/333586627 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384801d3f0 con 0x5646af9bb140 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f384e000640 1 -- 172.21.15.45:0/333586627 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f38480050e0 con 0x5646af9bb140 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.593+0000 7f3869f91740 1 -- 172.21.15.45:0/333586627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f384801d590 con 0x5646af9bb140 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f384e000640 1 -- 172.21.15.45:0/333586627 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38480052e0 con 0x5646af9bb140 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f3869f91740 10 client.15576 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f3869f91740 1 -- 172.21.15.45:0/333586627 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646afa3bfc0 con 0x5646af9bb140 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f384e000640 1 -- 172.21.15.45:0/333586627 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f384801d590 con 0x5646af9bb140 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f384e000640 1 -- 172.21.15.45:0/333586627 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7f3848026460 con 0x5646af9bb140 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f3869f91740 10 client.15576 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f3869f91740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f3869f91740 20 client.15576 populate_metadata read hostname 'smithi045' 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f3869f91740 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5646afa3c620 0x5646afa3ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f3869f91740 4 client.15576 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:04:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.594+0000 7f3869f91740 1 -- 172.21.15.45:0/333586627 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646af928190 con 0x5646afa3c620 2024-04-25T19:04:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.595+0000 7f3865200640 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5646afa3c620 0x5646afa3ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.595+0000 7f3865200640 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5646afa3c620 0x5646afa3ea40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e52ef1eb4565cb8 server_cookie=b311edd76fe6cbaa in_seq=0 out_seq=0 2024-04-25T19:04:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.595+0000 7f384e000640 10 client.15576 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.632+0000 7f384e000640 1 -- 172.21.15.45:0/333586627 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646af928190 con 0x5646afa3c620 2024-04-25T19:04:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.632+0000 7f384e000640 10 client.15576 handle_command_reply: tid=0 2024-04-25T19:04:16.634 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.632+0000 7f3869f91740 1 client.15576 shutdown 2024-04-25T19:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.632+0000 7f3869f91740 1 -- 172.21.15.45:0/333586627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 msgr2=0x5646af9cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.632+0000 7f3869f91740 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9cde40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f384801d8a0 tx=0x7f3848011450 comp rx=0 tx=0).stop 2024-04-25T19:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.632+0000 7f3869f91740 1 -- 172.21.15.45:0/333586627 shutdown_connections 2024-04-25T19:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.632+0000 7f3869f91740 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5646afa3c620 0x5646afa3ea40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.632+0000 7f3869f91740 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5646af9c5a00 0x5646af9cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.632+0000 7f3869f91740 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5646af9bb140 0x5646af9cde40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.633+0000 7f3869f91740 1 --2- 172.21.15.45:0/333586627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5646af9ba820 0x5646af9cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.633+0000 7f384e000640 0 client.15576 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.633+0000 7f3869f91740 1 -- 172.21.15.45:0/333586627 >> 172.21.15.45:0/333586627 conn(0x5646af9b6d30 msgr2=0x5646af9b5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.633+0000 7f3869f91740 1 -- 172.21.15.45:0/333586627 shutdown_connections 2024-04-25T19:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.633+0000 7f3869f91740 1 -- 172.21.15.45:0/333586627 wait complete. 2024-04-25T19:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.633+0000 7f3869f91740 20 client.15576 trim_cache size 0 max 16384 2024-04-25T19:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:04:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 -- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f38440a3d10 msgr2=0x7f38440a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f38440a3d10 0x7f38440a61d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x5646af9bbdc0 tx=0x7f38500d49b0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 -- 172.21.15.45:0/1990556546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38601021b0 msgr2=0x7f38601a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38601021b0 0x7f38601a0670 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3850009d50 tx=0x7f385000c6a0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 -- 172.21.15.45:0/1990556546 shutdown_connections 2024-04-25T19:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f38440a3d10 0x7f38440a61d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f386019ac00 0x7f386019b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3860107280 0x7f38601a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 --2- 172.21.15.45:0/1990556546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38601021b0 0x7f38601a0670 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 -- 172.21.15.45:0/1990556546 >> 172.21.15.45:0/1990556546 conn(0x7f38600fda10 msgr2=0x7f3860105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 -- 172.21.15.45:0/1990556546 shutdown_connections 2024-04-25T19:04:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:16.634+0000 7f3867200640 1 -- 172.21.15.45:0/1990556546 wait complete. 2024-04-25T19:04:18.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:18 smithi062 ceph-mon[32844]: pgmap v152: 97 pgs: 97 active+clean; 43 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-25T19:04:18.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:18 smithi079 ceph-mon[33002]: pgmap v152: 97 pgs: 97 active+clean; 43 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-25T19:04:18.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:18 smithi045 ceph-mon[27776]: pgmap v152: 97 pgs: 97 active+clean; 43 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-25T19:04:20.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:20 smithi062 ceph-mon[32844]: pgmap v153: 97 pgs: 97 active+clean; 43 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-25T19:04:20.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:20 smithi079 ceph-mon[33002]: pgmap v153: 97 pgs: 97 active+clean; 43 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-25T19:04:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:20 smithi045 ceph-mon[27776]: pgmap v153: 97 pgs: 97 active+clean; 43 MiB data, 828 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.6 MiB/s wr, 290 op/s 2024-04-25T19:04:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:22 smithi079 ceph-mon[33002]: pgmap v154: 97 pgs: 97 active+clean; 43 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.6 MiB/s wr, 464 op/s 2024-04-25T19:04:22.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:22 smithi045 ceph-mon[27776]: pgmap v154: 97 pgs: 97 active+clean; 43 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.6 MiB/s wr, 464 op/s 2024-04-25T19:04:22.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:22 smithi062 ceph-mon[32844]: pgmap v154: 97 pgs: 97 active+clean; 43 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.6 MiB/s wr, 464 op/s 2024-04-25T19:04:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:24 smithi079 ceph-mon[33002]: pgmap v155: 97 pgs: 97 active+clean; 43 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 330 op/s 2024-04-25T19:04:24.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:24 smithi045 ceph-mon[27776]: pgmap v155: 97 pgs: 97 active+clean; 43 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 330 op/s 2024-04-25T19:04:24.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:24 smithi062 ceph-mon[32844]: pgmap v155: 97 pgs: 97 active+clean; 43 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 330 op/s 2024-04-25T19:04:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:26 smithi079 ceph-mon[33002]: pgmap v156: 97 pgs: 97 active+clean; 41 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.6 MiB/s wr, 474 op/s 2024-04-25T19:04:26.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:26 smithi045 ceph-mon[27776]: pgmap v156: 97 pgs: 97 active+clean; 41 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.6 MiB/s wr, 474 op/s 2024-04-25T19:04:26.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:26 smithi062 ceph-mon[32844]: pgmap v156: 97 pgs: 97 active+clean; 41 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.6 MiB/s wr, 474 op/s 2024-04-25T19:04:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:28 smithi079 ceph-mon[33002]: pgmap v157: 97 pgs: 97 active+clean; 41 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 318 op/s 2024-04-25T19:04:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:28 smithi045 ceph-mon[27776]: pgmap v157: 97 pgs: 97 active+clean; 41 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 318 op/s 2024-04-25T19:04:28.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:28 smithi062 ceph-mon[32844]: pgmap v157: 97 pgs: 97 active+clean; 41 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 318 op/s 2024-04-25T19:04:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.791+0000 7fcc32e00640 1 -- 172.21.15.45:0/457198595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2c100d60 msgr2=0x7fcc2c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.791+0000 7fcc32e00640 1 --2- 172.21.15.45:0/457198595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2c100d60 0x7fcc2c101140 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009900 tx=0x7fcc1c02a410 comp rx=0 tx=0).stop 2024-04-25T19:04:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.791+0000 7fcc32e00640 1 -- 172.21.15.45:0/457198595 shutdown_connections 2024-04-25T19:04:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.791+0000 7fcc32e00640 1 --2- 172.21.15.45:0/457198595 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c10e910 0x7fcc2c110d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.791+0000 7fcc32e00640 1 --2- 172.21.15.45:0/457198595 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2c101680 0x7fcc2c10e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.791+0000 7fcc32e00640 1 --2- 172.21.15.45:0/457198595 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2c100d60 0x7fcc2c101140 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.791+0000 7fcc32e00640 1 -- 172.21.15.45:0/457198595 >> 172.21.15.45:0/457198595 conn(0x7fcc2c077fd0 msgr2=0x7fcc2c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.791+0000 7fcc32e00640 1 -- 172.21.15.45:0/457198595 shutdown_connections 2024-04-25T19:04:29.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.791+0000 7fcc32e00640 1 -- 172.21.15.45:0/457198595 wait complete. 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.793+0000 7fcc32e00640 1 Processor -- start 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.793+0000 7fcc32e00640 1 -- start start 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.793+0000 7fcc32e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c100d60 0x7fcc2c1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.793+0000 7fcc32e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2c101680 0x7fcc2c1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.793+0000 7fcc32e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2c10e910 0x7fcc2c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.793+0000 7fcc32e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c114480 con 0x7fcc2c100d60 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.793+0000 7fcc32e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c114300 con 0x7fcc2c10e910 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.793+0000 7fcc32e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c114600 con 0x7fcc2c101680 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.793+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2c101680 0x7fcc2c1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.794+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2c101680 0x7fcc2c1a0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60146/0 (socket says 172.21.15.45:60146) 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.794+0000 7fcc2b400640 1 -- 172.21.15.45:0/1595664150 learned_addr learned my addr 172.21.15.45:0/1595664150 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:29.796 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.794+0000 7fcc30e00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2c10e910 0x7fcc2c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.794+0000 7fcc2be00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c100d60 0x7fcc2c1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.797 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.794+0000 7fcc2b400640 1 -- 172.21.15.45:0/1595664150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2c10e910 msgr2=0x7fcc2c19a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.794+0000 7fcc2b400640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2c10e910 0x7fcc2c19a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.794+0000 7fcc2b400640 1 -- 172.21.15.45:0/1595664150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c100d60 msgr2=0x7fcc2c1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.794+0000 7fcc2b400640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c100d60 0x7fcc2c1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.794+0000 7fcc2b400640 1 -- 172.21.15.45:0/1595664150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c19af70 con 0x7fcc2c101680 2024-04-25T19:04:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.794+0000 7fcc2b400640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2c101680 0x7fcc2c1a0b70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800e9f0 tx=0x7fcc1800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.795+0000 7fcc28c00640 1 -- 172.21.15.45:0/1595664150 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1800cd80 con 0x7fcc2c101680 2024-04-25T19:04:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.795+0000 7fcc28c00640 1 -- 172.21.15.45:0/1595664150 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc180054d0 con 0x7fcc2c101680 2024-04-25T19:04:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.795+0000 7fcc28c00640 1 -- 172.21.15.45:0/1595664150 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18010690 con 0x7fcc2c101680 2024-04-25T19:04:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.795+0000 7fcc32e00640 1 -- 172.21.15.45:0/1595664150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2c19b260 con 0x7fcc2c101680 2024-04-25T19:04:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.795+0000 7fcc32e00640 1 -- 172.21.15.45:0/1595664150 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2c1a7480 con 0x7fcc2c101680 2024-04-25T19:04:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.798+0000 7fcc35c07740 1 Processor -- start 2024-04-25T19:04:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc35c07740 1 -- start start 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc28c00640 1 -- 172.21.15.45:0/1595664150 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fcc18004e90 con 0x7fcc2c101680 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eaca1820 0x5621eaca1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 0x5621eacac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eacaca00 0x5621eacaedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc35c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5621eac07900 con 0x5621eaca1820 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc35c07740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5621eabadef0 con 0x5621eaca2140 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc35c07740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5621eac038c0 con 0x5621eacaca00 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 0x5621eacac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eacaca00 0x5621eacaedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 0x5621eacac4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:49214/0 (socket says 172.21.15.45:49214) 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc2be00640 1 -- 172.21.15.45:0/2497255465 learned_addr learned my addr 172.21.15.45:0/2497255465 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:29.802 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.799+0000 7fcc30e00640 1 --2- 172.21.15.45:0/2497255465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eaca1820 0x5621eaca1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.800+0000 7fcc2be00640 1 -- 172.21.15.45:0/2497255465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eacaca00 msgr2=0x5621eacaedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.800+0000 7fcc2be00640 1 --2- 172.21.15.45:0/2497255465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eacaca00 0x5621eacaedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.800+0000 7fcc2be00640 1 -- 172.21.15.45:0/2497255465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eaca1820 msgr2=0x5621eaca1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.800+0000 7fcc2be00640 1 --2- 172.21.15.45:0/2497255465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eaca1820 0x5621eaca1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.803 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.800+0000 7fcc2be00640 1 -- 172.21.15.45:0/2497255465 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ea91a210 con 0x5621eaca2140 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.800+0000 7fcc2be00640 1 --2- 172.21.15.45:0/2497255465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 0x5621eacac4c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000c00 tx=0x7fcc1c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd2f9c43ccbd6b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.800+0000 7fcc28c00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fcc100a3d90 0x7fcc100a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.801+0000 7fcc21600640 1 -- 172.21.15.45:0/2497255465 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c035040 con 0x5621eaca2140 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.801+0000 7fcc21600640 1 -- 172.21.15.45:0/2497255465 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c0054f0 con 0x5621eaca2140 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.801+0000 7fcc21600640 1 -- 172.21.15.45:0/2497255465 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c0035f0 con 0x5621eaca2140 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.801+0000 7fcc28c00640 1 -- 172.21.15.45:0/1595664150 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fcc180d0c00 con 0x7fcc2c101680 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.801+0000 7fcc30e00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fcc100a3d90 0x7fcc100a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.801+0000 7fcc35c07740 1 -- 172.21.15.45:0/2497255465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 msgr2=0x5621eacac4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.802+0000 7fcc35c07740 1 --2- 172.21.15.45:0/2497255465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 0x5621eacac4c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c000c00 tx=0x7fcc1c004040 comp rx=0 tx=0).stop 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.802+0000 7fcc30e00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fcc100a3d90 0x7fcc100a6250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c06a3a0 tx=0x7fcc24009500 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.802+0000 7fcc35c07740 1 -- 172.21.15.45:0/2497255465 shutdown_connections 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.802+0000 7fcc35c07740 1 --2- 172.21.15.45:0/2497255465 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eacaca00 0x5621eacaedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.805 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.802+0000 7fcc35c07740 1 --2- 172.21.15.45:0/2497255465 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 0x5621eacac4c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.802+0000 7fcc35c07740 1 --2- 172.21.15.45:0/2497255465 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eaca1820 0x5621eaca1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.802+0000 7fcc35c07740 1 -- 172.21.15.45:0/2497255465 >> 172.21.15.45:0/2497255465 conn(0x5621eac9dd30 msgr2=0x5621eac9e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.802+0000 7fcc35c07740 1 -- 172.21.15.45:0/2497255465 shutdown_connections 2024-04-25T19:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.802+0000 7fcc35c07740 1 -- 172.21.15.45:0/2497255465 wait complete. 2024-04-25T19:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.802+0000 7fcc35c07740 1 -- start start 2024-04-25T19:04:29.806 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc35c07740 1 Processor -- start 2024-04-25T19:04:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca1820 0x5621eaca9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eaca9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacaa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc35c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5621eacaf5d0 con 0x5621eacaca00 2024-04-25T19:04:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc35c07740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5621eabc1310 con 0x5621eaca1820 2024-04-25T19:04:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc35c07740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5621eacaf750 con 0x5621eaca2140 2024-04-25T19:04:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc35c07740 10 client.0 _ll_register_callbacks cb 0x5621eab8fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:29.808 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eaca9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eaca9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60172/0 (socket says 172.21.15.45:60172) 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.803+0000 7fcc2b400640 1 -- 172.21.15.45:0/2506905115 learned_addr learned my addr 172.21.15.45:0/2506905115 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.804+0000 7fcc2be00640 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca1820 0x5621eaca9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.804+0000 7fcc30e00640 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacaa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.804+0000 7fcc2b400640 1 -- 172.21.15.45:0/2506905115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca1820 msgr2=0x5621eaca9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.804+0000 7fcc2b400640 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca1820 0x5621eaca9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.804+0000 7fcc2b400640 1 -- 172.21.15.45:0/2506905115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 msgr2=0x5621eacaa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.804+0000 7fcc2b400640 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacaa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.804+0000 7fcc2b400640 1 -- 172.21.15.45:0/2506905115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ea9e6620 con 0x5621eaca2140 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.805+0000 7fcc2b400640 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eaca9ea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc180ce350 tx=0x7fcc18098ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2afae8c351a3c0c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.805+0000 7fcc16000640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.805+0000 7fcc16000640 1 -- 172.21.15.45:0/2506905115 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18005180 con 0x5621eaca2140 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.805+0000 7fcc16000640 1 -- 172.21.15.45:0/2506905115 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fcc180991b0 con 0x5621eaca2140 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.805+0000 7fcc35c07740 1 -- 172.21.15.45:0/2506905115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc1c0054f0 con 0x5621eaca2140 2024-04-25T19:04:29.809 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.805+0000 7fcc16000640 1 -- 172.21.15.45:0/2506905115 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc18005680 con 0x5621eaca2140 2024-04-25T19:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.805+0000 7fcc35c07740 10 client.25741 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.805+0000 7fcc35c07740 1 -- 172.21.15.45:0/2506905115 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5621eacbb850 con 0x5621eaca2140 2024-04-25T19:04:29.810 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.806+0000 7fcc16000640 1 -- 172.21.15.45:0/2506905115 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1c0054f0 con 0x5621eaca2140 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.807+0000 7fcc16000640 1 -- 172.21.15.45:0/2506905115 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7fcc180d7030 con 0x5621eaca2140 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.807+0000 7fcc35c07740 10 client.25741 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.807+0000 7fcc35c07740 10 client.25741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.807+0000 7fcc35c07740 20 client.25741 populate_metadata read hostname 'smithi045' 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.807+0000 7fcc35c07740 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5621eacbccc0 0x5621eacbf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.807+0000 7fcc35c07740 4 client.25741 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.807+0000 7fcc35c07740 1 -- 172.21.15.45:0/2506905115 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621eac1b8b0 con 0x5621eacbccc0 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.807+0000 7fcc2be00640 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5621eacbccc0 0x5621eacbf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.808+0000 7fcc2be00640 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5621eacbccc0 0x5621eacbf110 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae6c857cf3b04d3d server_cookie=353900e58df06cbb in_seq=0 out_seq=0 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.808+0000 7fcc16000640 10 client.25741 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc16000640 1 -- 172.21.15.45:0/2506905115 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621eac1b8b0 con 0x5621eacbccc0 2024-04-25T19:04:29.811 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc16000640 10 client.25741 handle_command_reply: tid=0 2024-04-25T19:04:29.812 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc35c07740 1 client.25741 shutdown 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc35c07740 1 -- 172.21.15.45:0/2506905115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 msgr2=0x5621eaca9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc35c07740 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eaca9ea0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcc180ce350 tx=0x7fcc18098ea0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc35c07740 1 -- 172.21.15.45:0/2506905115 shutdown_connections 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc35c07740 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5621eacbccc0 0x5621eacbf110 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc35c07740 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacaa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc16000640 0 client.25741 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc35c07740 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eaca9ea0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc35c07740 1 --2- 172.21.15.45:0/2506905115 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca1820 0x5621eaca9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.809+0000 7fcc35c07740 1 -- 172.21.15.45:0/2506905115 >> 172.21.15.45:0/2506905115 conn(0x5621eac9dd30 msgr2=0x5621eaca3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.810+0000 7fcc35c07740 1 -- 172.21.15.45:0/2506905115 shutdown_connections 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.810+0000 7fcc35c07740 1 -- 172.21.15.45:0/2506905115 wait complete. 2024-04-25T19:04:29.813 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.810+0000 7fcc35c07740 20 client.25741 trim_cache size 0 max 16384 2024-04-25T19:04:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.828+0000 7fcc35c07740 1 Processor -- start 2024-04-25T19:04:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc35c07740 1 -- start start 2024-04-25T19:04:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca1820 0x5621eacb8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eac9cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacc4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc35c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1b2690 con 0x5621eacaca00 2024-04-25T19:04:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc35c07740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1aff60 con 0x5621eaca1820 2024-04-25T19:04:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc35c07740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1addb0 con 0x5621eaca2140 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eac9cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eac9cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60176/0 (socket says 172.21.15.45:60176) 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc2b400640 1 -- 172.21.15.45:0/90766983 learned_addr learned my addr 172.21.15.45:0/90766983 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc2be00640 1 --2- 172.21.15.45:0/90766983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca1820 0x5621eacb8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.829+0000 7fcc30e00640 1 --2- 172.21.15.45:0/90766983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacc4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.830+0000 7fcc2b400640 1 -- 172.21.15.45:0/90766983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca1820 msgr2=0x5621eacb8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.830+0000 7fcc2b400640 1 --2- 172.21.15.45:0/90766983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca1820 0x5621eacb8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.830+0000 7fcc2b400640 1 -- 172.21.15.45:0/90766983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 msgr2=0x5621eacc4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.830+0000 7fcc2b400640 1 --2- 172.21.15.45:0/90766983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacc4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.830+0000 7fcc2b400640 1 -- 172.21.15.45:0/90766983 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621eac9ef60 con 0x5621eaca2140 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.830+0000 7fcc2b400640 1 --2- 172.21.15.45:0/90766983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eac9cd10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcc180a5850 tx=0x7fcc18019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac3df13efc5db714 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.830+0000 7fcc20c00640 1 -- 172.21.15.45:0/90766983 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18005180 con 0x5621eaca2140 2024-04-25T19:04:29.833 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.830+0000 7fcc20c00640 1 -- 172.21.15.45:0/90766983 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1809d240 con 0x5621eaca2140 2024-04-25T19:04:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.830+0000 7fcc20c00640 1 -- 172.21.15.45:0/90766983 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc180d46b0 con 0x5621eaca2140 2024-04-25T19:04:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.831+0000 7fcc35c07740 1 -- 172.21.15.45:0/90766983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 msgr2=0x5621eac9cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.831+0000 7fcc35c07740 1 --2- 172.21.15.45:0/90766983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eac9cd10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcc180a5850 tx=0x7fcc18019000 comp rx=0 tx=0).stop 2024-04-25T19:04:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.831+0000 7fcc35c07740 1 -- 172.21.15.45:0/90766983 shutdown_connections 2024-04-25T19:04:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.831+0000 7fcc35c07740 1 --2- 172.21.15.45:0/90766983 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacc4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.831+0000 7fcc35c07740 1 --2- 172.21.15.45:0/90766983 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca2140 0x5621eac9cd10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.831+0000 7fcc35c07740 1 --2- 172.21.15.45:0/90766983 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca1820 0x5621eacb8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.831+0000 7fcc35c07740 1 -- 172.21.15.45:0/90766983 >> 172.21.15.45:0/90766983 conn(0x5621eac9dd30 msgr2=0x5621eac9c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.831+0000 7fcc35c07740 1 -- 172.21.15.45:0/90766983 shutdown_connections 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.831+0000 7fcc35c07740 1 -- 172.21.15.45:0/90766983 wait complete. 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.832+0000 7fcc35c07740 1 -- start start 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.832+0000 7fcc35c07740 1 Processor -- start 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca1820 0x5621eacb23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 0x5621eacb28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc35c07740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacb2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc35c07740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1ae420 con 0x5621eacaca00 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc35c07740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1af890 con 0x5621eaca2140 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc35c07740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1aeaf0 con 0x5621eaca1820 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc35c07740 10 client.0 _ll_register_callbacks cb 0x7fcc18087040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca1820 0x5621eacb23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.837 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc2be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca1820 0x5621eacb23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60186/0 (socket says 172.21.15.45:60186) 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc2be00640 1 -- 172.21.15.45:0/1457670630 learned_addr learned my addr 172.21.15.45:0/1457670630 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc2b400640 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 0x5621eacb28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc30e00640 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacb2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc2be00640 1 -- 172.21.15.45:0/1457670630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 msgr2=0x5621eacb28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc2be00640 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 0x5621eacb28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc2be00640 1 -- 172.21.15.45:0/1457670630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 msgr2=0x5621eacb2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc2be00640 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacb2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.833+0000 7fcc2be00640 1 -- 172.21.15.45:0/1457670630 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ead222a0 con 0x5621eaca1820 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.834+0000 7fcc2be00640 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca1820 0x5621eacb23a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009900 tx=0x7fcc1c004950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58d8ebe02f697221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.834+0000 7fcc16000640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.834+0000 7fcc16000640 1 -- 172.21.15.45:0/1457670630 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c035040 con 0x5621eaca1820 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.834+0000 7fcc16000640 1 -- 172.21.15.45:0/1457670630 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7fcc1c053370 con 0x5621eaca1820 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.834+0000 7fcc35c07740 1 -- 172.21.15.45:0/1457670630 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc1809d240 con 0x5621eaca1820 2024-04-25T19:04:29.838 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.834+0000 7fcc16000640 1 -- 172.21.15.45:0/1457670630 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc1c060050 con 0x5621eaca1820 2024-04-25T19:04:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.834+0000 7fcc35c07740 10 client.25747 fetch_fsmap learned FSMap version 29 2024-04-25T19:04:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.834+0000 7fcc35c07740 1 -- 172.21.15.45:0/1457670630 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5621ead235c0 con 0x5621eaca1820 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.835+0000 7fcc16000640 1 -- 172.21.15.45:0/1457670630 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc1809d240 con 0x5621eaca1820 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.835+0000 7fcc16000640 1 -- 172.21.15.45:0/1457670630 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 29) ==== 8342+0+0 (secure 0 0 0) 0x7fcc1c05b030 con 0x5621eaca1820 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.835+0000 7fcc35c07740 10 client.25747 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.835+0000 7fcc35c07740 10 client.25747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.835+0000 7fcc35c07740 20 client.25747 populate_metadata read hostname 'smithi045' 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.835+0000 7fcc35c07740 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5621ead23900 0x5621ead25cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.835+0000 7fcc35c07740 4 client.25747 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.835+0000 7fcc35c07740 1 -- 172.21.15.45:0/1457670630 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621eac15320 con 0x5621ead23900 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.835+0000 7fcc2b400640 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5621ead23900 0x5621ead25cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.836+0000 7fcc16000640 10 client.25747 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.836+0000 7fcc2b400640 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5621ead23900 0x5621ead25cf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712050693c7ef748 server_cookie=8910af232873e141 in_seq=0 out_seq=0 2024-04-25T19:04:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.859+0000 7fcc16000640 1 -- 172.21.15.45:0/1457670630 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621eac15320 con 0x5621ead23900 2024-04-25T19:04:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.859+0000 7fcc16000640 10 client.25747 handle_command_reply: tid=0 2024-04-25T19:04:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.859+0000 7fcc35c07740 1 client.25747 shutdown 2024-04-25T19:04:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 1 -- 172.21.15.45:0/1457670630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca1820 msgr2=0x5621eacb23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca1820 0x5621eacb23a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009900 tx=0x7fcc1c004950 comp rx=0 tx=0).stop 2024-04-25T19:04:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 1 -- 172.21.15.45:0/1457670630 shutdown_connections 2024-04-25T19:04:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5621ead23900 0x5621ead25cf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5621eacaca00 0x5621eacb2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5621eaca2140 0x5621eacb28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 1 --2- 172.21.15.45:0/1457670630 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5621eaca1820 0x5621eacb23a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc16000640 0 client.25747 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 1 -- 172.21.15.45:0/1457670630 >> 172.21.15.45:0/1457670630 conn(0x5621eac9dd30 msgr2=0x5621eac9ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 1 -- 172.21.15.45:0/1457670630 shutdown_connections 2024-04-25T19:04:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 1 -- 172.21.15.45:0/1457670630 wait complete. 2024-04-25T19:04:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.860+0000 7fcc35c07740 20 client.25747 trim_cache size 0 max 16384 2024-04-25T19:04:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:04:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:04:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:04:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:04:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.861+0000 7fcc32e00640 1 -- 172.21.15.45:0/1595664150 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fcc100a3d90 msgr2=0x7fcc100a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.861+0000 7fcc32e00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fcc100a3d90 0x7fcc100a6250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcc2c06a3a0 tx=0x7fcc24009500 comp rx=0 tx=0).stop 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.861+0000 7fcc32e00640 1 -- 172.21.15.45:0/1595664150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2c101680 msgr2=0x7fcc2c1a0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.861+0000 7fcc32e00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2c101680 0x7fcc2c1a0b70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800e9f0 tx=0x7fcc1800edf0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.861+0000 7fcc32e00640 1 -- 172.21.15.45:0/1595664150 shutdown_connections 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.861+0000 7fcc32e00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fcc100a3d90 0x7fcc100a6250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.861+0000 7fcc32e00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fcc2c10e910 0x7fcc2c19a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.861+0000 7fcc32e00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcc2c101680 0x7fcc2c1a0b70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.861+0000 7fcc32e00640 1 --2- 172.21.15.45:0/1595664150 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc2c100d60 0x7fcc2c1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.861+0000 7fcc32e00640 1 -- 172.21.15.45:0/1595664150 >> 172.21.15.45:0/1595664150 conn(0x7fcc2c077fd0 msgr2=0x7fcc2c10fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.862+0000 7fcc32e00640 1 -- 172.21.15.45:0/1595664150 shutdown_connections 2024-04-25T19:04:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:29.862+0000 7fcc32e00640 1 -- 172.21.15.45:0/1595664150 wait complete. 2024-04-25T19:04:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:30 smithi079 ceph-mon[33002]: pgmap v158: 97 pgs: 97 active+clean; 41 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 318 op/s 2024-04-25T19:04:30.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:30 smithi045 ceph-mon[27776]: pgmap v158: 97 pgs: 97 active+clean; 41 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 318 op/s 2024-04-25T19:04:30.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:30 smithi062 ceph-mon[32844]: pgmap v158: 97 pgs: 97 active+clean; 41 MiB data, 841 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.8 MiB/s wr, 318 op/s 2024-04-25T19:04:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:32 smithi079 ceph-mon[33002]: pgmap v159: 97 pgs: 97 active+clean; 46 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 461 op/s 2024-04-25T19:04:32.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:32 smithi045 ceph-mon[27776]: pgmap v159: 97 pgs: 97 active+clean; 46 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 461 op/s 2024-04-25T19:04:32.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:32 smithi062 ceph-mon[32844]: pgmap v159: 97 pgs: 97 active+clean; 46 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 461 op/s 2024-04-25T19:04:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:34 smithi079 ceph-mon[33002]: pgmap v160: 97 pgs: 97 active+clean; 46 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 287 op/s 2024-04-25T19:04:34.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:34 smithi045 ceph-mon[27776]: pgmap v160: 97 pgs: 97 active+clean; 46 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 287 op/s 2024-04-25T19:04:34.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:34 smithi062 ceph-mon[32844]: pgmap v160: 97 pgs: 97 active+clean; 46 MiB data, 899 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 287 op/s 2024-04-25T19:04:35.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:35 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:04:35.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:35 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:04:35.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:35 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:04:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:36 smithi079 ceph-mon[33002]: pgmap v161: 97 pgs: 97 active+clean; 46 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 426 op/s 2024-04-25T19:04:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:36 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:04:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:36 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:04:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:36 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:04:36.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:36 smithi045 ceph-mon[27776]: pgmap v161: 97 pgs: 97 active+clean; 46 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 426 op/s 2024-04-25T19:04:36.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:36 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:04:36.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:36 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:04:36.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:36 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:04:36.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:36 smithi062 ceph-mon[32844]: pgmap v161: 97 pgs: 97 active+clean; 46 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 426 op/s 2024-04-25T19:04:36.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:36 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:04:36.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:36 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:04:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:36 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:04:37.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:37 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:04:37.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:37 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:04:37.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:37 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:04:38.040 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0x9ff773 2024-04-25T19:04:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:38 smithi079 ceph-mon[33002]: pgmap v162: 97 pgs: 97 active+clean; 46 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-04-25T19:04:38.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:38 smithi045 ceph-mon[27776]: pgmap v162: 97 pgs: 97 active+clean; 46 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-04-25T19:04:38.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:38 smithi062 ceph-mon[32844]: pgmap v162: 97 pgs: 97 active+clean; 46 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-04-25T19:04:38.830 INFO:tasks.workunit.client.0.smithi045.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-04-25T19:04:40.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:40 smithi045 ceph-mon[27776]: pgmap v163: 97 pgs: 97 active+clean; 46 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-04-25T19:04:40.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:40 smithi062 ceph-mon[32844]: pgmap v163: 97 pgs: 97 active+clean; 46 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-04-25T19:04:40.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:40 smithi079 ceph-mon[33002]: pgmap v163: 97 pgs: 97 active+clean; 46 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.6 MiB/s wr, 282 op/s 2024-04-25T19:04:42.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:42 smithi045 ceph-mon[27776]: pgmap v164: 97 pgs: 97 active+clean; 42 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.6 MiB/s wr, 445 op/s 2024-04-25T19:04:42.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:42 smithi062 ceph-mon[32844]: pgmap v164: 97 pgs: 97 active+clean; 42 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.6 MiB/s wr, 445 op/s 2024-04-25T19:04:42.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:42 smithi079 ceph-mon[33002]: pgmap v164: 97 pgs: 97 active+clean; 42 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.6 MiB/s wr, 445 op/s 2024-04-25T19:04:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.018+0000 7f4972e00640 1 -- 172.21.15.45:0/3863405135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c100650 msgr2=0x7f496c100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.018+0000 7f4972e00640 1 --2- 172.21.15.45:0/3863405135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c100650 0x7f496c100a30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f495c009900 tx=0x7f495c02a410 comp rx=0 tx=0).stop 2024-04-25T19:04:43.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.018+0000 7f4972e00640 1 -- 172.21.15.45:0/3863405135 shutdown_connections 2024-04-25T19:04:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.018+0000 7f4972e00640 1 --2- 172.21.15.45:0/3863405135 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c10c5c0 0x7f496c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.018+0000 7f4972e00640 1 --2- 172.21.15.45:0/3863405135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c100f70 0x7f496c10bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.018+0000 7f4972e00640 1 --2- 172.21.15.45:0/3863405135 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c100650 0x7f496c100a30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.018+0000 7f4972e00640 1 -- 172.21.15.45:0/3863405135 >> 172.21.15.45:0/3863405135 conn(0x7f496c075d60 msgr2=0x7f496c076170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.018+0000 7f4972e00640 1 -- 172.21.15.45:0/3863405135 shutdown_connections 2024-04-25T19:04:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.018+0000 7f4972e00640 1 -- 172.21.15.45:0/3863405135 wait complete. 2024-04-25T19:04:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.019+0000 7f4972e00640 1 Processor -- start 2024-04-25T19:04:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.019+0000 7f4972e00640 1 -- start start 2024-04-25T19:04:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f4972e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c100650 0x7f496c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f4972e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c100f70 0x7f496c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f4972e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c10c5c0 0x7f496c19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f4972e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1121e0 con 0x7f496c10c5c0 2024-04-25T19:04:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f4972e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f496c112060 con 0x7f496c100f70 2024-04-25T19:04:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f4972e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f496c112360 con 0x7f496c100650 2024-04-25T19:04:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f496be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c100650 0x7f496c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f496be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c100650 0x7f496c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:47840/0 (socket says 172.21.15.45:47840) 2024-04-25T19:04:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f496be00640 1 -- 172.21.15.45:0/1052186032 learned_addr learned my addr 172.21.15.45:0/1052186032 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f496b400640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c100f70 0x7f496c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.020+0000 7f4970e00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c10c5c0 0x7f496c19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f496be00640 1 -- 172.21.15.45:0/1052186032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c100f70 msgr2=0x7f496c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f496be00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c100f70 0x7f496c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f496be00640 1 -- 172.21.15.45:0/1052186032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c10c5c0 msgr2=0x7f496c19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f496be00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c10c5c0 0x7f496c19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f496be00640 1 -- 172.21.15.45:0/1052186032 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f496c19d6f0 con 0x7f496c100650 2024-04-25T19:04:43.023 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f496be00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c100650 0x7f496c198700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f495c005ab0 tx=0x7f495c003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f4968c00640 1 -- 172.21.15.45:0/1052186032 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c03f070 con 0x7f496c100650 2024-04-25T19:04:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f4968c00640 1 -- 172.21.15.45:0/1052186032 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f495c031b50 con 0x7f496c100650 2024-04-25T19:04:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f4972e00640 1 -- 172.21.15.45:0/1052186032 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f496c19d980 con 0x7f496c100650 2024-04-25T19:04:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f4968c00640 1 -- 172.21.15.45:0/1052186032 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c035030 con 0x7f496c100650 2024-04-25T19:04:43.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.021+0000 7f4972e00640 1 -- 172.21.15.45:0/1052186032 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f496c1a5270 con 0x7f496c100650 2024-04-25T19:04:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.025+0000 7f4968c00640 1 -- 172.21.15.45:0/1052186032 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f495c042070 con 0x7f496c100650 2024-04-25T19:04:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.025+0000 7f4975bba740 1 Processor -- start 2024-04-25T19:04:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.025+0000 7f4975bba740 1 -- start start 2024-04-25T19:04:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.025+0000 7f4975bba740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d4ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4975bba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4975bba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d59e10 0x55d923d5c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4975bba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d923cb0900 con 0x55d923d4b060 2024-04-25T19:04:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4975bba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d923c56ef0 con 0x55d923d59e10 2024-04-25T19:04:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4975bba740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d923cac8c0 con 0x55d923d4a740 2024-04-25T19:04:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4970e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d4ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4970e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d4ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:47848/0 (socket says 172.21.15.45:47848) 2024-04-25T19:04:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f496b400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d59e10 0x55d923d5c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4970e00640 1 -- 172.21.15.45:0/236334517 learned_addr learned my addr 172.21.15.45:0/236334517 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f496be00640 1 --2- 172.21.15.45:0/236334517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4970e00640 1 -- 172.21.15.45:0/236334517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d59e10 msgr2=0x55d923d5c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4970e00640 1 --2- 172.21.15.45:0/236334517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d59e10 0x55d923d5c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4970e00640 1 -- 172.21.15.45:0/236334517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 msgr2=0x55d923d598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4970e00640 1 --2- 172.21.15.45:0/236334517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4970e00640 1 -- 172.21.15.45:0/236334517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9239c3210 con 0x55d923d4a740 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.026+0000 7f4968c00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49500a3d90 0x7f49500a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.027+0000 7f4968c00640 1 -- 172.21.15.45:0/1052186032 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f495c0f0aa0 con 0x7f496c100650 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.027+0000 7f4970e00640 1 --2- 172.21.15.45:0/236334517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d4ab20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4964009f80 tx=0x7f4964010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4264e6dc534e9a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.027+0000 7f496be00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49500a3d90 0x7f49500a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.027+0000 7f4961600640 1 -- 172.21.15.45:0/236334517 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964003b70 con 0x55d923d4a740 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.027+0000 7f4961600640 1 -- 172.21.15.45:0/236334517 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4964003d10 con 0x55d923d4a740 2024-04-25T19:04:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.027+0000 7f4961600640 1 -- 172.21.15.45:0/236334517 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964011ae0 con 0x55d923d4a740 2024-04-25T19:04:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f4975bba740 1 -- 172.21.15.45:0/236334517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 msgr2=0x55d923d4ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:43.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f496be00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49500a3d90 0x7f49500a6250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55d923d4d340 tx=0x7f495c003f90 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f4975bba740 1 --2- 172.21.15.45:0/236334517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d4ab20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4964009f80 tx=0x7f4964010860 comp rx=0 tx=0).stop 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f4975bba740 1 -- 172.21.15.45:0/236334517 shutdown_connections 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f4975bba740 1 --2- 172.21.15.45:0/236334517 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d59e10 0x55d923d5c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f4975bba740 1 --2- 172.21.15.45:0/236334517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f4975bba740 1 --2- 172.21.15.45:0/236334517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d4ab20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f4975bba740 1 -- 172.21.15.45:0/236334517 >> 172.21.15.45:0/236334517 conn(0x55d923d46d30 msgr2=0x55d923d47110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f4975bba740 1 -- 172.21.15.45:0/236334517 shutdown_connections 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f4975bba740 1 -- 172.21.15.45:0/236334517 wait complete. 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.028+0000 7f4975bba740 1 -- start start 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4975bba740 1 Processor -- start 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4975bba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d56e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4975bba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d57340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.034 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4975bba740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d59e10 0x55d923d57880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4975bba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55d923d5ca40 con 0x55d923d4b060 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4975bba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55d923c6a310 con 0x55d923d4a740 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4975bba740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55d923d5cbc0 con 0x55d923d59e10 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4975bba740 10 client.0 _ll_register_callbacks cb 0x55d923c38cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4970e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d56e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f496b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d57340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4970e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d56e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:47304/0 (socket says 172.21.15.45:47304) 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.029+0000 7f4970e00640 1 -- 172.21.15.45:0/756734402 learned_addr learned my addr 172.21.15.45:0/756734402 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.030+0000 7f4970e00640 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d59e10 0x55d923d57880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.030+0000 7f496b400640 1 -- 172.21.15.45:0/756734402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d59e10 msgr2=0x55d923d57880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.030+0000 7f496b400640 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d59e10 0x55d923d57880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.030+0000 7f496b400640 1 -- 172.21.15.45:0/756734402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 msgr2=0x55d923d56e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.030+0000 7f496b400640 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d56e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.030+0000 7f496b400640 1 -- 172.21.15.45:0/756734402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d923a8f620 con 0x55d923d4b060 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.030+0000 7f496b400640 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d57340 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f496c0fff30 tx=0x7f4958012b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9178abc74ccb1eee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:43.035 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.030+0000 7f4956000640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.030+0000 7f4956000640 1 -- 172.21.15.45:0/756734402 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495800f920 con 0x55d923d4b060 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.031+0000 7f4956000640 1 -- 172.21.15.45:0/756734402 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f495800fac0 con 0x55d923d4b060 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.031+0000 7f4975bba740 1 -- 172.21.15.45:0/756734402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4964003d10 con 0x55d923d4b060 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.031+0000 7f4956000640 1 -- 172.21.15.45:0/756734402 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f495800fcc0 con 0x55d923d4b060 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.031+0000 7f4975bba740 10 client.15606 fetch_fsmap learned FSMap version 30 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.031+0000 7f4975bba740 1 -- 172.21.15.45:0/756734402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d923d68d60 con 0x55d923d4b060 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.031+0000 7f4956000640 1 -- 172.21.15.45:0/756734402 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4964003d10 con 0x55d923d4b060 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.031+0000 7f4956000640 1 -- 172.21.15.45:0/756734402 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f495801c420 con 0x55d923d4b060 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.031+0000 7f4975bba740 10 client.15606 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.031+0000 7f4975bba740 10 client.15606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.031+0000 7f4975bba740 20 client.15606 populate_metadata read hostname 'smithi045' 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.032+0000 7f4975bba740 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55d923d6a270 0x55d923d6c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.032+0000 7f4975bba740 4 client.15606 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.032+0000 7f4975bba740 1 -- 172.21.15.45:0/756734402 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d923cc48b0 con 0x55d923d6a270 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.032+0000 7f4970e00640 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55d923d6a270 0x55d923d6c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.036 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.032+0000 7f4970e00640 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55d923d6a270 0x55d923d6c6c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc9f1d4311fb20bf server_cookie=511210fbadeaf4f4 in_seq=0 out_seq=0 2024-04-25T19:04:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.032+0000 7f4956000640 10 client.15606 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:43.037 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4956000640 1 -- 172.21.15.45:0/756734402 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d923cc48b0 con 0x55d923d6a270 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4956000640 10 client.15606 handle_command_reply: tid=0 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4975bba740 1 client.15606 shutdown 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4975bba740 1 -- 172.21.15.45:0/756734402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 msgr2=0x55d923d57340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4975bba740 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d57340 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f496c0fff30 tx=0x7f4958012b60 comp rx=0 tx=0).stop 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4975bba740 1 -- 172.21.15.45:0/756734402 shutdown_connections 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4975bba740 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55d923d6a270 0x55d923d6c6c0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4975bba740 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d59e10 0x55d923d57880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4975bba740 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d57340 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4975bba740 1 --2- 172.21.15.45:0/756734402 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d56e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4956000640 0 client.15606 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.033+0000 7f4975bba740 1 -- 172.21.15.45:0/756734402 >> 172.21.15.45:0/756734402 conn(0x55d923d46d30 msgr2=0x55d923d4cba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.034+0000 7f4975bba740 1 -- 172.21.15.45:0/756734402 shutdown_connections 2024-04-25T19:04:43.038 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.034+0000 7f4975bba740 1 -- 172.21.15.45:0/756734402 wait complete. 2024-04-25T19:04:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.034+0000 7f4975bba740 20 client.15606 trim_cache size 0 max 16384 2024-04-25T19:04:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.055+0000 7f4975bba740 1 Processor -- start 2024-04-25T19:04:43.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.055+0000 7f4975bba740 1 -- start start 2024-04-25T19:04:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f4975bba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d65900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f4975bba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d71f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f4975bba740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d59e10 0x55d923d724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.057 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f4975bba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1ae200 con 0x55d923d4b060 2024-04-25T19:04:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f4975bba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1abad0 con 0x55d923d4a740 2024-04-25T19:04:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f4975bba740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1a9920 con 0x55d923d59e10 2024-04-25T19:04:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f4970e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d65900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f4970e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d65900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:47312/0 (socket says 172.21.15.45:47312) 2024-04-25T19:04:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f4970e00640 1 -- 172.21.15.45:0/2374352127 learned_addr learned my addr 172.21.15.45:0/2374352127 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f496b400640 1 --2- 172.21.15.45:0/2374352127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d71f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.058 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.056+0000 7f4970e00640 1 --2- 172.21.15.45:0/2374352127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d59e10 0x55d923d724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.057+0000 7f4970e00640 1 -- 172.21.15.45:0/2374352127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d59e10 msgr2=0x55d923d724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.057+0000 7f4970e00640 1 --2- 172.21.15.45:0/2374352127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d59e10 0x55d923d724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.057+0000 7f4970e00640 1 -- 172.21.15.45:0/2374352127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 msgr2=0x55d923d71f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.057+0000 7f4970e00640 1 --2- 172.21.15.45:0/2374352127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d71f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.057+0000 7f4970e00640 1 -- 172.21.15.45:0/2374352127 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d923d55dd0 con 0x55d923d4a740 2024-04-25T19:04:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.057+0000 7f496b400640 1 --2- 172.21.15.45:0/2374352127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d71f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:04:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.057+0000 7f4970e00640 1 --2- 172.21.15.45:0/2374352127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d65900 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4964000f80 tx=0x7f4964011500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5c8b23d9f3f0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:43.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.058+0000 7f4960c00640 1 -- 172.21.15.45:0/2374352127 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964026070 con 0x55d923d4a740 2024-04-25T19:04:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.058+0000 7f4960c00640 1 -- 172.21.15.45:0/2374352127 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4964027040 con 0x55d923d4a740 2024-04-25T19:04:43.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.058+0000 7f4960c00640 1 -- 172.21.15.45:0/2374352127 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964011770 con 0x55d923d4a740 2024-04-25T19:04:43.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 -- 172.21.15.45:0/2374352127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 msgr2=0x55d923d65900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 --2- 172.21.15.45:0/2374352127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d65900 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4964000f80 tx=0x7f4964011500 comp rx=0 tx=0).stop 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 -- 172.21.15.45:0/2374352127 shutdown_connections 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 --2- 172.21.15.45:0/2374352127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d59e10 0x55d923d724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 --2- 172.21.15.45:0/2374352127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d71f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 --2- 172.21.15.45:0/2374352127 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d4a740 0x55d923d65900 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 -- 172.21.15.45:0/2374352127 >> 172.21.15.45:0/2374352127 conn(0x55d923d46d30 msgr2=0x55d923d45e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 -- 172.21.15.45:0/2374352127 shutdown_connections 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 -- 172.21.15.45:0/2374352127 wait complete. 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 -- start start 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.059+0000 7f4975bba740 1 Processor -- start 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4975bba740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d61ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4975bba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d61ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4975bba740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d59e10 0x55d923d5ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4975bba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1a9f90 con 0x55d923d4b060 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4975bba740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1ab400 con 0x55d923d59e10 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4975bba740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f496c1aa660 con 0x55d923d4a740 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4975bba740 10 client.0 _ll_register_callbacks cb 0x7f4958009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4970e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d61ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4970e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d61ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:47876/0 (socket says 172.21.15.45:47876) 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4970e00640 1 -- 172.21.15.45:0/1319927006 learned_addr learned my addr 172.21.15.45:0/1319927006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f4970e00640 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d59e10 0x55d923d5ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.060+0000 7f496b400640 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d61ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.061+0000 7f4970e00640 1 -- 172.21.15.45:0/1319927006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d59e10 msgr2=0x55d923d5ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.061+0000 7f4970e00640 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d59e10 0x55d923d5ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.061+0000 7f4970e00640 1 -- 172.21.15.45:0/1319927006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 msgr2=0x55d923d61ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.061+0000 7f4970e00640 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d61ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.063 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.061+0000 7f4970e00640 1 -- 172.21.15.45:0/1319927006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d923d606b0 con 0x55d923d4a740 2024-04-25T19:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.061+0000 7f4970e00640 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d61ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4964009560 tx=0x7f496402f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11737b0d23598c4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.061+0000 7f4956000640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.061+0000 7f4956000640 1 -- 172.21.15.45:0/1319927006 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4964026070 con 0x55d923d4a740 2024-04-25T19:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.061+0000 7f4956000640 1 -- 172.21.15.45:0/1319927006 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f4964004ca0 con 0x55d923d4a740 2024-04-25T19:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.061+0000 7f4975bba740 1 -- 172.21.15.45:0/1319927006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4964027040 con 0x55d923d4a740 2024-04-25T19:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4956000640 1 -- 172.21.15.45:0/1319927006 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4964011770 con 0x55d923d4a740 2024-04-25T19:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4975bba740 10 client.25765 fetch_fsmap learned FSMap version 30 2024-04-25T19:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4975bba740 1 -- 172.21.15.45:0/1319927006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d923dd1210 con 0x55d923d4a740 2024-04-25T19:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4956000640 1 -- 172.21.15.45:0/1319927006 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4964027040 con 0x55d923d4a740 2024-04-25T19:04:43.065 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4956000640 1 -- 172.21.15.45:0/1319927006 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f49640052d0 con 0x55d923d4a740 2024-04-25T19:04:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4975bba740 10 client.25765 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T19:04:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4975bba740 10 client.25765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4975bba740 20 client.25765 populate_metadata read hostname 'smithi045' 2024-04-25T19:04:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4975bba740 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55d923dd1640 0x55d923dd3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4975bba740 4 client.25765 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:04:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.062+0000 7f4975bba740 1 -- 172.21.15.45:0/1319927006 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d923cbe320 con 0x55d923dd1640 2024-04-25T19:04:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.063+0000 7f496b400640 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55d923dd1640 0x55d923dd3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.063+0000 7f496b400640 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55d923dd1640 0x55d923dd3a60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551cf712eccf8265 server_cookie=e331aedc18da15e in_seq=0 out_seq=0 2024-04-25T19:04:43.066 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.063+0000 7f4956000640 10 client.25765 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4956000640 1 -- 172.21.15.45:0/1319927006 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d923cbe320 con 0x55d923dd1640 2024-04-25T19:04:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4956000640 10 client.25765 handle_command_reply: tid=0 2024-04-25T19:04:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 client.25765 shutdown 2024-04-25T19:04:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 -- 172.21.15.45:0/1319927006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 msgr2=0x55d923d61ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d61ab0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4964009560 tx=0x7f496402f6c0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 -- 172.21.15.45:0/1319927006 shutdown_connections 2024-04-25T19:04:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55d923dd1640 0x55d923dd3a60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.081 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55d923d59e10 0x55d923d5ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55d923d4b060 0x55d923d61ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 --2- 172.21.15.45:0/1319927006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55d923d4a740 0x55d923d61ab0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 -- 172.21.15.45:0/1319927006 >> 172.21.15.45:0/1319927006 conn(0x55d923d46d30 msgr2=0x55d923d56da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4956000640 0 client.25765 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 -- 172.21.15.45:0/1319927006 shutdown_connections 2024-04-25T19:04:43.082 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.080+0000 7f4975bba740 1 -- 172.21.15.45:0/1319927006 wait complete. 2024-04-25T19:04:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.081+0000 7f4975bba740 20 client.25765 trim_cache size 0 max 16384 2024-04-25T19:04:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:04:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:04:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:04:43.083 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.081+0000 7f4972e00640 1 -- 172.21.15.45:0/1052186032 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49500a3d90 msgr2=0x7f49500a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.081+0000 7f4972e00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49500a3d90 0x7f49500a6250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55d923d4d340 tx=0x7f495c003f90 comp rx=0 tx=0).stop 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.081+0000 7f4972e00640 1 -- 172.21.15.45:0/1052186032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c100650 msgr2=0x7f496c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.081+0000 7f4972e00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c100650 0x7f496c198700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f495c005ab0 tx=0x7f495c003b90 comp rx=0 tx=0).stop 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.082+0000 7f4972e00640 1 -- 172.21.15.45:0/1052186032 shutdown_connections 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.082+0000 7f4972e00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f49500a3d90 0x7f49500a6250 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.082+0000 7f4972e00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f496c10c5c0 0x7f496c19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.082+0000 7f4972e00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f496c100f70 0x7f496c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.082+0000 7f4972e00640 1 --2- 172.21.15.45:0/1052186032 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f496c100650 0x7f496c198700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.082+0000 7f4972e00640 1 -- 172.21.15.45:0/1052186032 >> 172.21.15.45:0/1052186032 conn(0x7f496c075d60 msgr2=0x7f496c0fd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.082+0000 7f4972e00640 1 -- 172.21.15.45:0/1052186032 shutdown_connections 2024-04-25T19:04:43.084 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:43.082+0000 7f4972e00640 1 -- 172.21.15.45:0/1052186032 wait complete. 2024-04-25T19:04:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:44 smithi045 ceph-mon[27776]: pgmap v165: 97 pgs: 97 active+clean; 42 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-04-25T19:04:44.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:44 smithi062 ceph-mon[32844]: pgmap v165: 97 pgs: 97 active+clean; 42 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-04-25T19:04:44.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:44 smithi079 ceph-mon[33002]: pgmap v165: 97 pgs: 97 active+clean; 42 MiB data, 884 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.7 MiB/s wr, 302 op/s 2024-04-25T19:04:46.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:46 smithi045 ceph-mon[27776]: pgmap v166: 97 pgs: 97 active+clean; 44 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 460 op/s 2024-04-25T19:04:46.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:46 smithi062 ceph-mon[32844]: pgmap v166: 97 pgs: 97 active+clean; 44 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 460 op/s 2024-04-25T19:04:46.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:46 smithi079 ceph-mon[33002]: pgmap v166: 97 pgs: 97 active+clean; 44 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.6 MiB/s wr, 460 op/s 2024-04-25T19:04:48.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:48 smithi045 ceph-mon[27776]: pgmap v167: 97 pgs: 97 active+clean; 44 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-04-25T19:04:48.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:48 smithi062 ceph-mon[32844]: pgmap v167: 97 pgs: 97 active+clean; 44 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-04-25T19:04:48.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:48 smithi079 ceph-mon[33002]: pgmap v167: 97 pgs: 97 active+clean; 44 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-04-25T19:04:50.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:50 smithi062 ceph-mon[32844]: pgmap v168: 97 pgs: 97 active+clean; 44 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-04-25T19:04:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:50 smithi079 ceph-mon[33002]: pgmap v168: 97 pgs: 97 active+clean; 44 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-04-25T19:04:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:50 smithi045 ceph-mon[27776]: pgmap v168: 97 pgs: 97 active+clean; 44 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-04-25T19:04:52.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:52 smithi062 ceph-mon[32844]: pgmap v169: 97 pgs: 97 active+clean; 42 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.6 MiB/s wr, 444 op/s 2024-04-25T19:04:52.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:52 smithi079 ceph-mon[33002]: pgmap v169: 97 pgs: 97 active+clean; 42 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.6 MiB/s wr, 444 op/s 2024-04-25T19:04:52.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:52 smithi045 ceph-mon[27776]: pgmap v169: 97 pgs: 97 active+clean; 42 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.6 MiB/s wr, 444 op/s 2024-04-25T19:04:54.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:54 smithi062 ceph-mon[32844]: pgmap v170: 97 pgs: 97 active+clean; 42 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-04-25T19:04:54.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:54 smithi079 ceph-mon[33002]: pgmap v170: 97 pgs: 97 active+clean; 42 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-04-25T19:04:54.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:54 smithi045 ceph-mon[27776]: pgmap v170: 97 pgs: 97 active+clean; 42 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.6 MiB/s wr, 281 op/s 2024-04-25T19:04:56.002 INFO:tasks.workunit.client.0.smithi045.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-04-25T19:04:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.248+0000 7f4ab3c00640 1 -- 172.21.15.45:0/2945715973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aac100d40 msgr2=0x7f4aac101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.248+0000 7f4ab3c00640 1 --2- 172.21.15.45:0/2945715973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aac100d40 0x7f4aac101120 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a98009900 tx=0x7f4a9802a400 comp rx=0 tx=0).stop 2024-04-25T19:04:56.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.248+0000 7f4ab3c00640 1 -- 172.21.15.45:0/2945715973 shutdown_connections 2024-04-25T19:04:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.248+0000 7f4ab3c00640 1 --2- 172.21.15.45:0/2945715973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac10e8f0 0x7f4aac110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.248+0000 7f4ab3c00640 1 --2- 172.21.15.45:0/2945715973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac101660 0x7f4aac10e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.248+0000 7f4ab3c00640 1 --2- 172.21.15.45:0/2945715973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aac100d40 0x7f4aac101120 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.248+0000 7f4ab3c00640 1 -- 172.21.15.45:0/2945715973 >> 172.21.15.45:0/2945715973 conn(0x7f4aac077f70 msgr2=0x7f4aac078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.248+0000 7f4ab3c00640 1 -- 172.21.15.45:0/2945715973 shutdown_connections 2024-04-25T19:04:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.248+0000 7f4ab3c00640 1 -- 172.21.15.45:0/2945715973 wait complete. 2024-04-25T19:04:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.249+0000 7f4ab3c00640 1 Processor -- start 2024-04-25T19:04:56.250 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab3c00640 1 -- start start 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab3c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac100d40 0x7f4aac1a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab3c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac101660 0x7f4aac1a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab3c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aac10e8f0 0x7f4aac19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab3c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac114440 con 0x7f4aac100d40 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab3c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1142c0 con 0x7f4aac101660 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab3c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1145c0 con 0x7f4aac10e8f0 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab1c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aac10e8f0 0x7f4aac19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac100d40 0x7f4aac1a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab1c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aac10e8f0 0x7f4aac19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:58958/0 (socket says 172.21.15.45:58958) 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4aabe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac101660 0x7f4aac1a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.250+0000 7f4ab1c00640 1 -- 172.21.15.45:0/1731853081 learned_addr learned my addr 172.21.15.45:0/1731853081 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.251+0000 7f4ab1c00640 1 -- 172.21.15.45:0/1731853081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac101660 msgr2=0x7f4aac1a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:56.252 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.251+0000 7f4ab1c00640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac101660 0x7f4aac1a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.251+0000 7f4ab1c00640 1 -- 172.21.15.45:0/1731853081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac100d40 msgr2=0x7f4aac1a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.251+0000 7f4ab1c00640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac100d40 0x7f4aac1a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.251+0000 7f4ab1c00640 1 -- 172.21.15.45:0/1731853081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aac19af70 con 0x7f4aac10e8f0 2024-04-25T19:04:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.251+0000 7f4ab1c00640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aac10e8f0 0x7f4aac19a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400ec50 tx=0x7f4aa400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:56.253 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.251+0000 7f4aa9600640 1 -- 172.21.15.45:0/1731853081 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4003b70 con 0x7f4aac10e8f0 2024-04-25T19:04:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.251+0000 7f4aa9600640 1 -- 172.21.15.45:0/1731853081 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa4003d10 con 0x7f4aac10e8f0 2024-04-25T19:04:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.251+0000 7f4aa9600640 1 -- 172.21.15.45:0/1731853081 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40108d0 con 0x7f4aac10e8f0 2024-04-25T19:04:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.252+0000 7f4ab3c00640 1 -- 172.21.15.45:0/1731853081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aac19b260 con 0x7f4aac10e8f0 2024-04-25T19:04:56.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.252+0000 7f4ab3c00640 1 -- 172.21.15.45:0/1731853081 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aac1a7480 con 0x7f4aac10e8f0 2024-04-25T19:04:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.255+0000 7f4aa9600640 1 -- 172.21.15.45:0/1731853081 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4aa4003ec0 con 0x7f4aac10e8f0 2024-04-25T19:04:56.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.255+0000 7f4ab69d2740 1 Processor -- start 2024-04-25T19:04:56.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4ab69d2740 1 -- start start 2024-04-25T19:04:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a09c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a0a140 0x557dc0a144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a14a00 0x557dc0a16df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557dc096f900 con 0x557dc0a0a140 2024-04-25T19:04:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557dc0915ef0 con 0x557dc0a14a00 2024-04-25T19:04:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557dc096b8c0 con 0x557dc0a09820 2024-04-25T19:04:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4aabe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a09c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4aabe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a09c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:58962/0 (socket says 172.21.15.45:58962) 2024-04-25T19:04:56.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4aabe00640 1 -- 172.21.15.45:0/2917641536 learned_addr learned my addr 172.21.15.45:0/2917641536 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4aabe00640 1 --2- 172.21.15.45:0/2917641536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a14a00 0x557dc0a16df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.256+0000 7f4ab1200640 1 --2- 172.21.15.45:0/2917641536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a0a140 0x557dc0a144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4aabe00640 1 -- 172.21.15.45:0/2917641536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a14a00 msgr2=0x557dc0a16df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4aabe00640 1 --2- 172.21.15.45:0/2917641536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a14a00 0x557dc0a16df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4aabe00640 1 -- 172.21.15.45:0/2917641536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a0a140 msgr2=0x557dc0a144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4aabe00640 1 --2- 172.21.15.45:0/2917641536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a0a140 0x557dc0a144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4aabe00640 1 -- 172.21.15.45:0/2917641536 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dc0682210 con 0x557dc0a09820 2024-04-25T19:04:56.259 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4aabe00640 1 --2- 172.21.15.45:0/2917641536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a09c00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00137c0 tx=0x7f4aa0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d87455c0199e999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4aa9600640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4a940a3d90 0x7f4a940a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4a9e000640 1 -- 172.21.15.45:0/2917641536 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0013d40 con 0x557dc0a09820 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4a9e000640 1 -- 172.21.15.45:0/2917641536 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa0013130 con 0x557dc0a09820 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4a9e000640 1 -- 172.21.15.45:0/2917641536 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0005620 con 0x557dc0a09820 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.257+0000 7f4aa9600640 1 -- 172.21.15.45:0/1731853081 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f4aa40d0dc0 con 0x7f4aac10e8f0 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.258+0000 7f4ab1200640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4a940a3d90 0x7f4a940a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.258+0000 7f4ab69d2740 1 -- 172.21.15.45:0/2917641536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 msgr2=0x557dc0a09c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.258+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/2917641536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a09c00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00137c0 tx=0x7f4aa0005f90 comp rx=0 tx=0).stop 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.258+0000 7f4ab69d2740 1 -- 172.21.15.45:0/2917641536 shutdown_connections 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.258+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/2917641536 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a14a00 0x557dc0a16df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.258+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/2917641536 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a0a140 0x557dc0a144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.258+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/2917641536 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a09c00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.258+0000 7f4ab69d2740 1 -- 172.21.15.45:0/2917641536 >> 172.21.15.45:0/2917641536 conn(0x557dc0a05d30 msgr2=0x557dc0a06110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.258+0000 7f4ab1200640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4a940a3d90 0x7f4a940a6250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a98005ab0 tx=0x7f4a98005a20 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.258+0000 7f4ab69d2740 1 -- 172.21.15.45:0/2917641536 shutdown_connections 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.259+0000 7f4ab69d2740 1 -- 172.21.15.45:0/2917641536 wait complete. 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.259+0000 7f4ab69d2740 1 -- start start 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.259+0000 7f4ab69d2740 1 Processor -- start 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a119c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 0x557dc0a11f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 0x557dc0a12440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557dc0a175d0 con 0x557dc0a14a00 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557dc0929310 con 0x557dc0a0a140 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557dc0a17750 con 0x557dc0a09820 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab69d2740 10 client.0 _ll_register_callbacks cb 0x557dc08f7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:56.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab1200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a119c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab1200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a119c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:58970/0 (socket says 172.21.15.45:58970) 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab1200640 1 -- 172.21.15.45:0/3418956470 learned_addr learned my addr 172.21.15.45:0/3418956470 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4aabe00640 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 0x557dc0a11f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab1c00640 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 0x557dc0a12440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.260+0000 7f4ab1200640 1 -- 172.21.15.45:0/3418956470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 msgr2=0x557dc0a11f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.261+0000 7f4ab1200640 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 0x557dc0a11f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.261+0000 7f4ab1200640 1 -- 172.21.15.45:0/3418956470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 msgr2=0x557dc0a12440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.261+0000 7f4ab1200640 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 0x557dc0a12440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.261+0000 7f4ab1200640 1 -- 172.21.15.45:0/3418956470 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dc074e620 con 0x557dc0a09820 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.261+0000 7f4ab1c00640 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 0x557dc0a12440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.261+0000 7f4ab1200640 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a119c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a98033000 tx=0x7f4a98009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36662829001551f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:56.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.261+0000 7f4a92a00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.261+0000 7f4a92a00640 1 -- 172.21.15.45:0/3418956470 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a9804d070 con 0x557dc0a09820 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.261+0000 7f4ab69d2740 1 -- 172.21.15.45:0/3418956470 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa0013130 con 0x557dc0a09820 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.261+0000 7f4a92a00640 1 -- 172.21.15.45:0/3418956470 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f4a98004d80 con 0x557dc0a09820 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4a92a00640 1 -- 172.21.15.45:0/3418956470 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4a98059050 con 0x557dc0a09820 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4ab69d2740 10 client.25777 fetch_fsmap learned FSMap version 30 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4ab69d2740 1 -- 172.21.15.45:0/3418956470 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557dc0a238b0 con 0x557dc0a09820 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4a92a00640 1 -- 172.21.15.45:0/3418956470 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa0013130 con 0x557dc0a09820 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4a92a00640 1 -- 172.21.15.45:0/3418956470 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4a98032670 con 0x557dc0a09820 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4ab69d2740 10 client.25777 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4ab69d2740 10 client.25777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4ab69d2740 20 client.25777 populate_metadata read hostname 'smithi045' 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557dc0a24d20 0x557dc0a27190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4ab69d2740 4 client.25777 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:04:56.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.262+0000 7f4ab69d2740 1 -- 172.21.15.45:0/3418956470 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dc09838b0 con 0x557dc0a24d20 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.263+0000 7f4aabe00640 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557dc0a24d20 0x557dc0a27190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.264+0000 7f4aabe00640 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557dc0a24d20 0x557dc0a27190 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5473fae3c2ea8f22 server_cookie=6a46702ac3c3e10a in_seq=0 out_seq=0 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.264+0000 7f4a92a00640 10 client.25777 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4a92a00640 1 -- 172.21.15.45:0/3418956470 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557dc09838b0 con 0x557dc0a24d20 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4a92a00640 10 client.25777 handle_command_reply: tid=0 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4ab69d2740 1 client.25777 shutdown 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4ab69d2740 1 -- 172.21.15.45:0/3418956470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 msgr2=0x557dc0a119c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a119c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a98033000 tx=0x7f4a98009260 comp rx=0 tx=0).stop 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4ab69d2740 1 -- 172.21.15.45:0/3418956470 shutdown_connections 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557dc0a24d20 0x557dc0a27190 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 0x557dc0a12440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 0x557dc0a11f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4a92a00640 0 client.25777 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:56.268 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/3418956470 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a119c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.265+0000 7f4ab69d2740 1 -- 172.21.15.45:0/3418956470 >> 172.21.15.45:0/3418956470 conn(0x557dc0a05d30 msgr2=0x557dc0a0bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.266+0000 7f4ab69d2740 1 -- 172.21.15.45:0/3418956470 shutdown_connections 2024-04-25T19:04:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.266+0000 7f4ab69d2740 1 -- 172.21.15.45:0/3418956470 wait complete. 2024-04-25T19:04:56.269 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.266+0000 7f4ab69d2740 20 client.25777 trim_cache size 0 max 16384 2024-04-25T19:04:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.284+0000 7f4ab69d2740 1 Processor -- start 2024-04-25T19:04:56.285 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.284+0000 7f4ab69d2740 1 -- start start 2024-04-25T19:04:56.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a20490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 0x557dc0a0c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 0x557dc0a2caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1b2760 con 0x557dc0a14a00 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1b0030 con 0x557dc0a0a140 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1ade80 con 0x557dc0a09820 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4ab1200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a20490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4aabe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 0x557dc0a0c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4ab1200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a20490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:58980/0 (socket says 172.21.15.45:58980) 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4aabe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 0x557dc0a0c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:33382/0 (socket says 172.21.15.45:33382) 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4ab1200640 1 -- 172.21.15.45:0/479518209 learned_addr learned my addr 172.21.15.45:0/479518209 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.285+0000 7f4ab1c00640 1 --2- 172.21.15.45:0/479518209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 0x557dc0a2caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.286+0000 7f4aabe00640 1 -- 172.21.15.45:0/479518209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 msgr2=0x557dc0a20490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.286+0000 7f4aabe00640 1 --2- 172.21.15.45:0/479518209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a20490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.286+0000 7f4aabe00640 1 -- 172.21.15.45:0/479518209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 msgr2=0x557dc0a2caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.286+0000 7f4aabe00640 1 --2- 172.21.15.45:0/479518209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 0x557dc0a2caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.286+0000 7f4aabe00640 1 -- 172.21.15.45:0/479518209 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dc0a17160 con 0x557dc0a0a140 2024-04-25T19:04:56.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.286+0000 7f4ab1200640 1 --2- 172.21.15.45:0/479518209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a20490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.286+0000 7f4aabe00640 1 --2- 172.21.15.45:0/479518209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 0x557dc0a0c230 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00097e0 tx=0x7f4aa001c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a25c458e265afcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.286+0000 7f4a9d600640 1 -- 172.21.15.45:0/479518209 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0028bc0 con 0x557dc0a0a140 2024-04-25T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.286+0000 7f4a9d600640 1 -- 172.21.15.45:0/479518209 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa0017040 con 0x557dc0a0a140 2024-04-25T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.286+0000 7f4a9d600640 1 -- 172.21.15.45:0/479518209 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000fa80 con 0x557dc0a0a140 2024-04-25T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.287+0000 7f4ab69d2740 1 -- 172.21.15.45:0/479518209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 msgr2=0x557dc0a0c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.287+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/479518209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 0x557dc0a0c230 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4aa00097e0 tx=0x7f4aa001c040 comp rx=0 tx=0).stop 2024-04-25T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.287+0000 7f4ab69d2740 1 -- 172.21.15.45:0/479518209 shutdown_connections 2024-04-25T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.287+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/479518209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a14a00 0x557dc0a2caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.287+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/479518209 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a0a140 0x557dc0a0c230 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.287+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/479518209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a09820 0x557dc0a20490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.287+0000 7f4ab69d2740 1 -- 172.21.15.45:0/479518209 >> 172.21.15.45:0/479518209 conn(0x557dc0a05d30 msgr2=0x557dc0a04e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.287+0000 7f4ab69d2740 1 -- 172.21.15.45:0/479518209 shutdown_connections 2024-04-25T19:04:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.288+0000 7f4ab69d2740 1 -- 172.21.15.45:0/479518209 wait complete. 2024-04-25T19:04:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.288+0000 7f4ab69d2740 1 -- start start 2024-04-25T19:04:56.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.288+0000 7f4ab69d2740 1 Processor -- start 2024-04-25T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a09820 0x557dc0a1a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a0a140 0x557dc0a1aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab69d2740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a14a00 0x557dc0a1afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1ae4f0 con 0x557dc0a0a140 2024-04-25T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1af960 con 0x557dc0a09820 2024-04-25T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab69d2740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4aac1aebc0 con 0x557dc0a14a00 2024-04-25T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab69d2740 10 client.0 _ll_register_callbacks cb 0x7f4a98004480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a14a00 0x557dc0a1afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a09820 0x557dc0a1a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a14a00 0x557dc0a1afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:58988/0 (socket says 172.21.15.45:58988) 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a09820 0x557dc0a1a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:33384/0 (socket says 172.21.15.45:33384) 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1c00640 1 -- 172.21.15.45:0/646731368 learned_addr learned my addr 172.21.15.45:0/646731368 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4aabe00640 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a0a140 0x557dc0a1aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1c00640 1 -- 172.21.15.45:0/646731368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a09820 msgr2=0x557dc0a1a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1c00640 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a09820 0x557dc0a1a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1c00640 1 -- 172.21.15.45:0/646731368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a0a140 msgr2=0x557dc0a1aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1c00640 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a0a140 0x557dc0a1aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1c00640 1 -- 172.21.15.45:0/646731368 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dc0a8a420 con 0x557dc0a14a00 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.289+0000 7f4ab1200640 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a09820 0x557dc0a1a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.290+0000 7f4ab1c00640 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a14a00 0x557dc0a1afc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4099160 tx=0x7f4aa409d4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de4818732563865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.290+0000 7f4a92a00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.290+0000 7f4a92a00640 1 -- 172.21.15.45:0/646731368 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40bd2d0 con 0x557dc0a14a00 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.290+0000 7f4ab69d2740 1 -- 172.21.15.45:0/646731368 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa0017040 con 0x557dc0a14a00 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.290+0000 7f4a92a00640 1 -- 172.21.15.45:0/646731368 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8527+0+0 (secure 0 0 0) 0x7f4aa4004b40 con 0x557dc0a14a00 2024-04-25T19:04:56.293 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.290+0000 7f4a92a00640 1 -- 172.21.15.45:0/646731368 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4aa40a5970 con 0x557dc0a14a00 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.290+0000 7f4ab69d2740 10 client.25783 fetch_fsmap learned FSMap version 30 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.290+0000 7f4ab69d2740 1 -- 172.21.15.45:0/646731368 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557dc0a8b780 con 0x557dc0a14a00 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4a92a00640 1 -- 172.21.15.45:0/646731368 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4aa0017040 con 0x557dc0a14a00 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4a92a00640 1 -- 172.21.15.45:0/646731368 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 30) ==== 8338+0+0 (secure 0 0 0) 0x7f4aa4098b70 con 0x557dc0a14a00 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4ab69d2740 10 client.25783 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4ab69d2740 10 client.25783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4ab69d2740 20 client.25783 populate_metadata read hostname 'smithi045' 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557dc0a8be80 0x557dc0a8e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4ab69d2740 4 client.25783 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4ab69d2740 1 -- 172.21.15.45:0/646731368 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557dc097d320 con 0x557dc0a8be80 2024-04-25T19:04:56.294 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4ab1200640 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557dc0a8be80 0x557dc0a8e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4ab1200640 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557dc0a8be80 0x557dc0a8e2a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f97eb2aff753af server_cookie=898ba67a3890e5e5 in_seq=0 out_seq=0 2024-04-25T19:04:56.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.291+0000 7f4a92a00640 10 client.25783 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4a92a00640 1 -- 172.21.15.45:0/646731368 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557dc097d320 con 0x557dc0a8be80 2024-04-25T19:04:56.306 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4a92a00640 10 client.25783 handle_command_reply: tid=0 2024-04-25T19:04:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 client.25783 shutdown 2024-04-25T19:04:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 -- 172.21.15.45:0/646731368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a14a00 msgr2=0x557dc0a1afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:56.307 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a14a00 0x557dc0a1afc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4099160 tx=0x7f4aa409d4c0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 -- 172.21.15.45:0/646731368 shutdown_connections 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557dc0a8be80 0x557dc0a8e2a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557dc0a14a00 0x557dc0a1afc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557dc0a0a140 0x557dc0a1aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 --2- 172.21.15.45:0/646731368 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557dc0a09820 0x557dc0a1a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4a92a00640 0 client.25783 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 -- 172.21.15.45:0/646731368 >> 172.21.15.45:0/646731368 conn(0x557dc0a05d30 msgr2=0x557dc0a11980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 -- 172.21.15.45:0/646731368 shutdown_connections 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.306+0000 7f4ab69d2740 1 -- 172.21.15.45:0/646731368 wait complete. 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.307+0000 7f4ab69d2740 20 client.25783 trim_cache size 0 max 16384 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:04:56.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:04:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:04:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:04:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.307+0000 7f4ab3c00640 1 -- 172.21.15.45:0/1731853081 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4a940a3d90 msgr2=0x7f4a940a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.307+0000 7f4ab3c00640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4a940a3d90 0x7f4a940a6250 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a98005ab0 tx=0x7f4a98005a20 comp rx=0 tx=0).stop 2024-04-25T19:04:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.307+0000 7f4ab3c00640 1 -- 172.21.15.45:0/1731853081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aac10e8f0 msgr2=0x7f4aac19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:04:56.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.307+0000 7f4ab3c00640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aac10e8f0 0x7f4aac19a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400ec50 tx=0x7f4aa400c6a0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.308+0000 7f4ab3c00640 1 -- 172.21.15.45:0/1731853081 shutdown_connections 2024-04-25T19:04:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.308+0000 7f4ab3c00640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4a940a3d90 0x7f4a940a6250 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.308+0000 7f4ab3c00640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4aac10e8f0 0x7f4aac19a7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.308+0000 7f4ab3c00640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4aac101660 0x7f4aac1a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.308+0000 7f4ab3c00640 1 --2- 172.21.15.45:0/1731853081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4aac100d40 0x7f4aac1a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:04:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.308+0000 7f4ab3c00640 1 -- 172.21.15.45:0/1731853081 >> 172.21.15.45:0/1731853081 conn(0x7f4aac077f70 msgr2=0x7f4aac10ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:04:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.308+0000 7f4ab3c00640 1 -- 172.21.15.45:0/1731853081 shutdown_connections 2024-04-25T19:04:56.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:04:56.308+0000 7f4ab3c00640 1 -- 172.21.15.45:0/1731853081 wait complete. 2024-04-25T19:04:56.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:56 smithi062 ceph-mon[32844]: pgmap v171: 97 pgs: 97 active+clean; 48 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.4 MiB/s wr, 400 op/s 2024-04-25T19:04:56.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:56 smithi079 ceph-mon[33002]: pgmap v171: 97 pgs: 97 active+clean; 48 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.4 MiB/s wr, 400 op/s 2024-04-25T19:04:56.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:56 smithi045 ceph-mon[27776]: pgmap v171: 97 pgs: 97 active+clean; 48 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.4 MiB/s wr, 400 op/s 2024-04-25T19:04:57.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:57 smithi079 ceph-mon[33002]: daemon mds.c restarted 2024-04-25T19:04:57.434 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:04:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c[63996]: ignoring --setuser ceph since I am not root 2024-04-25T19:04:57.434 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:04:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c[63996]: ignoring --setgroup ceph since I am not root 2024-04-25T19:04:57.434 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:04:57 smithi079 ceph-mds[64000]: -- 172.21.15.79:0/1292843319 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558788295a40 con 0x558788268c00 2024-04-25T19:04:57.434 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:04:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c[63996]: starting mds.c at 2024-04-25T19:04:57.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:57 smithi062 ceph-mon[32844]: daemon mds.c restarted 2024-04-25T19:04:57.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:57 smithi045 ceph-mon[27776]: daemon mds.c restarted 2024-04-25T19:04:58.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:58 smithi062 ceph-mon[32844]: pgmap v172: 97 pgs: 97 active+clean; 48 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 242 op/s 2024-04-25T19:04:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:58 smithi062 ceph-mon[32844]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T19:04:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:58 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:04:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:58 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.79:6836/620692737,v1:172.21.15.79:6837/620692737] up:boot 2024-04-25T19:04:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:58 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T19:04:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:04:58 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:04:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:58 smithi079 ceph-mon[33002]: pgmap v172: 97 pgs: 97 active+clean; 48 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 242 op/s 2024-04-25T19:04:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:58 smithi079 ceph-mon[33002]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T19:04:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:58 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:04:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:58 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.79:6836/620692737,v1:172.21.15.79:6837/620692737] up:boot 2024-04-25T19:04:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:58 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T19:04:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:04:58 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:04:58.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:58 smithi045 ceph-mon[27776]: pgmap v172: 97 pgs: 97 active+clean; 48 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 242 op/s 2024-04-25T19:04:58.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:58 smithi045 ceph-mon[27776]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T19:04:58.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:58 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:04:58.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:58 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.79:6836/620692737,v1:172.21.15.79:6837/620692737] up:boot 2024-04-25T19:04:58.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:58 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T19:04:58.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:04:58 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:05:00.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:00 smithi062 ceph-mon[32844]: pgmap v174: 97 pgs: 97 active+clean; 48 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.7 MiB/s wr, 291 op/s 2024-04-25T19:05:00.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:00 smithi079 ceph-mon[33002]: pgmap v174: 97 pgs: 97 active+clean; 48 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.7 MiB/s wr, 291 op/s 2024-04-25T19:05:00.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:00 smithi045 ceph-mon[27776]: pgmap v174: 97 pgs: 97 active+clean; 48 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.7 MiB/s wr, 291 op/s 2024-04-25T19:05:02.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:02 smithi062 ceph-mon[32844]: pgmap v175: 97 pgs: 97 active+clean; 43 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 321 op/s 2024-04-25T19:05:02.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:02 smithi079 ceph-mon[33002]: pgmap v175: 97 pgs: 97 active+clean; 43 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 321 op/s 2024-04-25T19:05:02.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:02 smithi045 ceph-mon[27776]: pgmap v175: 97 pgs: 97 active+clean; 43 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 321 op/s 2024-04-25T19:05:03.704 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0x9ffb01 2024-04-25T19:05:04.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:04 smithi062 ceph-mon[32844]: pgmap v176: 97 pgs: 97 active+clean; 43 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 321 op/s 2024-04-25T19:05:04.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:04 smithi079 ceph-mon[33002]: pgmap v176: 97 pgs: 97 active+clean; 43 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 321 op/s 2024-04-25T19:05:04.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:04 smithi045 ceph-mon[27776]: pgmap v176: 97 pgs: 97 active+clean; 43 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 2.0 MiB/s wr, 321 op/s 2024-04-25T19:05:06.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:06 smithi062 ceph-mon[32844]: pgmap v177: 97 pgs: 97 active+clean; 45 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.3 MiB/s wr, 375 op/s 2024-04-25T19:05:06.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:06 smithi079 ceph-mon[33002]: pgmap v177: 97 pgs: 97 active+clean; 45 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.3 MiB/s wr, 375 op/s 2024-04-25T19:05:06.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:06 smithi045 ceph-mon[27776]: pgmap v177: 97 pgs: 97 active+clean; 45 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.3 MiB/s wr, 375 op/s 2024-04-25T19:05:08.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:08 smithi062 ceph-mon[32844]: pgmap v178: 97 pgs: 97 active+clean; 45 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.3 MiB/s wr, 375 op/s 2024-04-25T19:05:08.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:08 smithi079 ceph-mon[33002]: pgmap v178: 97 pgs: 97 active+clean; 45 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.3 MiB/s wr, 375 op/s 2024-04-25T19:05:08.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:08 smithi045 ceph-mon[27776]: pgmap v178: 97 pgs: 97 active+clean; 45 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 2.3 MiB/s wr, 375 op/s 2024-04-25T19:05:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.459+0000 7efcea000640 1 -- 172.21.15.45:0/2153913638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efce41031c0 msgr2=0x7efce41035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.462 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.459+0000 7efcea000640 1 --2- 172.21.15.45:0/2153913638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efce41031c0 0x7efce41035a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009860 tx=0x7efcd402a3d0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.460+0000 7efcea000640 1 -- 172.21.15.45:0/2153913638 shutdown_connections 2024-04-25T19:05:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.460+0000 7efcea000640 1 --2- 172.21.15.45:0/2153913638 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce410c590 0x7efce410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.460+0000 7efcea000640 1 --2- 172.21.15.45:0/2153913638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce4103ae0 0x7efce410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.460+0000 7efcea000640 1 --2- 172.21.15.45:0/2153913638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efce41031c0 0x7efce41035a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.460+0000 7efcea000640 1 -- 172.21.15.45:0/2153913638 >> 172.21.15.45:0/2153913638 conn(0x7efce4077f70 msgr2=0x7efce4078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.460+0000 7efcea000640 1 -- 172.21.15.45:0/2153913638 shutdown_connections 2024-04-25T19:05:09.463 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.460+0000 7efcea000640 1 -- 172.21.15.45:0/2153913638 wait complete. 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.461+0000 7efcea000640 1 Processor -- start 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.461+0000 7efcea000640 1 -- start start 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.461+0000 7efcea000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce41031c0 0x7efce41a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efcea000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efce4103ae0 0x7efce41a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efcea000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce410c590 0x7efce419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efcea000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efce4112210 con 0x7efce41031c0 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efcea000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efce4112090 con 0x7efce410c590 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efcea000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efce4112390 con 0x7efce4103ae0 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce410c590 0x7efce419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efce4103ae0 0x7efce41a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.464 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce41031c0 0x7efce41a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce410c590 0x7efce419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:59182/0 (socket says 172.21.15.45:59182) 2024-04-25T19:05:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce3e00640 1 -- 172.21.15.45:0/1906180009 learned_addr learned my addr 172.21.15.45:0/1906180009 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:09.465 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efce4103ae0 0x7efce41a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55874/0 (socket says 172.21.15.45:55874) 2024-04-25T19:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce3e00640 1 -- 172.21.15.45:0/1906180009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efce4103ae0 msgr2=0x7efce41a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce3e00640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efce4103ae0 0x7efce41a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce3e00640 1 -- 172.21.15.45:0/1906180009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce41031c0 msgr2=0x7efce41a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce3e00640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce41031c0 0x7efce41a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce3e00640 1 -- 172.21.15.45:0/1906180009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce419b020 con 0x7efce410c590 2024-04-25T19:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.462+0000 7efce2a00640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efce4103ae0 0x7efce41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.463+0000 7efce3e00640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce410c590 0x7efce419a850 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00eb40 tx=0x7efcdc00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.463+0000 7efcdbe00640 1 -- 172.21.15.45:0/1906180009 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc004b20 con 0x7efce410c590 2024-04-25T19:05:09.466 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.463+0000 7efcea000640 1 -- 172.21.15.45:0/1906180009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce419b310 con 0x7efce410c590 2024-04-25T19:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.463+0000 7efcdbe00640 1 -- 172.21.15.45:0/1906180009 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcdc004cc0 con 0x7efce410c590 2024-04-25T19:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.463+0000 7efcdbe00640 1 -- 172.21.15.45:0/1906180009 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc010720 con 0x7efce410c590 2024-04-25T19:05:09.467 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.463+0000 7efcea000640 1 -- 172.21.15.45:0/1906180009 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce41a74e0 con 0x7efce410c590 2024-04-25T19:05:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.467+0000 7efcecf7d740 1 Processor -- start 2024-04-25T19:05:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.468+0000 7efcdbe00640 1 -- 172.21.15.45:0/1906180009 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7efcdc005560 con 0x7efce410c590 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.468+0000 7efcecf7d740 1 -- start start 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.468+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c114747c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.468+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114748140 0x55c1147524c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.468+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c114754df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.468+0000 7efcecf7d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c1146ad900 con 0x55c114748140 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.468+0000 7efcecf7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c114653ef0 con 0x55c114752a00 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.468+0000 7efcecf7d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c1146a98c0 con 0x55c114747820 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c114747c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce3400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114748140 0x55c1147524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c114747c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55876/0 (socket says 172.21.15.45:55876) 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce2a00640 1 -- 172.21.15.45:0/413930905 learned_addr learned my addr 172.21.15.45:0/413930905 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce2a00640 1 --2- 172.21.15.45:0/413930905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c114754df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce3400640 1 -- 172.21.15.45:0/413930905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 msgr2=0x55c114747c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce3400640 1 --2- 172.21.15.45:0/413930905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c114747c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce3400640 1 -- 172.21.15.45:0/413930905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 msgr2=0x55c114754df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce3400640 1 --2- 172.21.15.45:0/413930905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c114754df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce3400640 1 -- 172.21.15.45:0/413930905 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1143c0210 con 0x55c114748140 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.469+0000 7efce2a00640 1 --2- 172.21.15.45:0/413930905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c114747c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.470+0000 7efcdbe00640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efcc80a3d10 0x7efcc80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.470+0000 7efce3400640 1 --2- 172.21.15.45:0/413930905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114748140 0x55c1147524c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009830 tx=0x7efcd40303a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3010393db1fbb8e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.470+0000 7efcdbe00640 1 -- 172.21.15.45:0/1906180009 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7efcdc0d0f10 con 0x7efce410c590 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.470+0000 7efce2a00640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efcc80a3d10 0x7efcc80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.470+0000 7efccfe00640 1 -- 172.21.15.45:0/413930905 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4030790 con 0x55c114748140 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.470+0000 7efccfe00640 1 -- 172.21.15.45:0/413930905 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd4041030 con 0x55c114748140 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.470+0000 7efccfe00640 1 -- 172.21.15.45:0/413930905 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4005240 con 0x55c114748140 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.471+0000 7efcecf7d740 1 -- 172.21.15.45:0/413930905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114748140 msgr2=0x55c1147524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.471+0000 7efce2a00640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efcc80a3d10 0x7efcc80a61d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55c114745150 tx=0x7efcd0009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.471+0000 7efcecf7d740 1 --2- 172.21.15.45:0/413930905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114748140 0x55c1147524c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009830 tx=0x7efcd40303a0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.471+0000 7efcecf7d740 1 -- 172.21.15.45:0/413930905 shutdown_connections 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.471+0000 7efcecf7d740 1 --2- 172.21.15.45:0/413930905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c114754df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.471+0000 7efcecf7d740 1 --2- 172.21.15.45:0/413930905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114748140 0x55c1147524c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.471+0000 7efcecf7d740 1 --2- 172.21.15.45:0/413930905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c114747c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.471+0000 7efcecf7d740 1 -- 172.21.15.45:0/413930905 >> 172.21.15.45:0/413930905 conn(0x55c114743d30 msgr2=0x55c114744110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.471+0000 7efcecf7d740 1 -- 172.21.15.45:0/413930905 shutdown_connections 2024-04-25T19:05:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.471+0000 7efcecf7d740 1 -- 172.21.15.45:0/413930905 wait complete. 2024-04-25T19:05:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.472+0000 7efcecf7d740 1 -- start start 2024-04-25T19:05:09.477 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.472+0000 7efcecf7d740 1 Processor -- start 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.472+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114747820 0x55c11474f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.472+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 0x55c11474fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.472+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c114750400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.472+0000 7efcecf7d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c1147555d0 con 0x55c114747820 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.472+0000 7efcecf7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55c114667310 con 0x55c114752a00 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.472+0000 7efcecf7d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55c114755750 con 0x55c114748140 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.472+0000 7efcecf7d740 10 client.0 _ll_register_callbacks cb 0x55c114635cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 0x55c11474fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 0x55c11474fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55892/0 (socket says 172.21.15.45:55892) 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce2a00640 1 -- 172.21.15.45:0/179103084 learned_addr learned my addr 172.21.15.45:0/179103084 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce3400640 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114747820 0x55c11474f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce3e00640 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c114750400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce3e00640 1 -- 172.21.15.45:0/179103084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 msgr2=0x55c11474fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce3e00640 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 0x55c11474fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce3e00640 1 -- 172.21.15.45:0/179103084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114747820 msgr2=0x55c11474f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce3e00640 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114747820 0x55c11474f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce3e00640 1 -- 172.21.15.45:0/179103084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c11448c620 con 0x55c114752a00 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efce3e00640 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c114750400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0ce660 tx=0x7efcdc09c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70095db7adabb8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.473+0000 7efcccc00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcccc00640 1 -- 172.21.15.45:0/179103084 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0b5020 con 0x55c114752a00 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcccc00640 1 -- 172.21.15.45:0/179103084 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7efcdc0b6220 con 0x55c114752a00 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcecf7d740 1 -- 172.21.15.45:0/179103084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcd4041030 con 0x55c114752a00 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcccc00640 1 -- 172.21.15.45:0/179103084 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efcdc0b6420 con 0x55c114752a00 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcecf7d740 10 client.25784 fetch_fsmap learned FSMap version 32 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcecf7d740 1 -- 172.21.15.45:0/179103084 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1147618b0 con 0x55c114752a00 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcccc00640 1 -- 172.21.15.45:0/179103084 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcd4041030 con 0x55c114752a00 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcccc00640 1 -- 172.21.15.45:0/179103084 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7efcdc098c00 con 0x55c114752a00 2024-04-25T19:05:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcecf7d740 10 client.25784 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:05:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcecf7d740 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.474+0000 7efcecf7d740 20 client.25784 populate_metadata read hostname 'smithi045' 2024-04-25T19:05:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.475+0000 7efcecf7d740 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55c114762d70 0x55c1147651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.475+0000 7efcecf7d740 4 client.25784 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.475+0000 7efcecf7d740 1 -- 172.21.15.45:0/179103084 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1146c18b0 con 0x55c114762d70 2024-04-25T19:05:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.475+0000 7efce3400640 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55c114762d70 0x55c1147651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.475+0000 7efcccc00640 10 client.25784 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.476+0000 7efce3400640 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55c114762d70 0x55c1147651c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a565bb2ee0c0c5 server_cookie=2a3b5f5416f238e6 in_seq=0 out_seq=0 2024-04-25T19:05:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.476+0000 7efcccc00640 1 -- 172.21.15.45:0/179103084 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1146c18b0 con 0x55c114762d70 2024-04-25T19:05:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.476+0000 7efcccc00640 10 client.25784 handle_command_reply: tid=0 2024-04-25T19:05:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 client.25784 shutdown 2024-04-25T19:05:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 -- 172.21.15.45:0/179103084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 msgr2=0x55c114750400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c114750400 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0ce660 tx=0x7efcdc09c8d0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 -- 172.21.15.45:0/179103084 shutdown_connections 2024-04-25T19:05:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55c114762d70 0x55c1147651c0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c114750400 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 0x55c11474fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcccc00640 0 client.25784 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 --2- 172.21.15.45:0/179103084 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114747820 0x55c11474f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 -- 172.21.15.45:0/179103084 >> 172.21.15.45:0/179103084 conn(0x55c114743d30 msgr2=0x55c114749a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 -- 172.21.15.45:0/179103084 shutdown_connections 2024-04-25T19:05:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 1 -- 172.21.15.45:0/179103084 wait complete. 2024-04-25T19:05:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.477+0000 7efcecf7d740 20 client.25784 trim_cache size 0 max 16384 2024-04-25T19:05:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.495+0000 7efcecf7d740 1 Processor -- start 2024-04-25T19:05:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efcecf7d740 1 -- start start 2024-04-25T19:05:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114747820 0x55c11475e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 0x55c11474a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c11476aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efcecf7d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efce41b26a0 con 0x55c114747820 2024-04-25T19:05:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efcecf7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efce41aff70 con 0x55c114752a00 2024-04-25T19:05:09.498 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efcecf7d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efce41addc0 con 0x55c114748140 2024-04-25T19:05:09.499 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efce3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c11476aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efce2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 0x55c11474a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efce3e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c11476aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:59216/0 (socket says 172.21.15.45:59216) 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efce2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 0x55c11474a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55904/0 (socket says 172.21.15.45:55904) 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efce3e00640 1 -- 172.21.15.45:0/4157294012 learned_addr learned my addr 172.21.15.45:0/4157294012 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.496+0000 7efce3400640 1 --2- 172.21.15.45:0/4157294012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114747820 0x55c11475e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.497+0000 7efce3e00640 1 -- 172.21.15.45:0/4157294012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 msgr2=0x55c11474a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.497+0000 7efce3e00640 1 --2- 172.21.15.45:0/4157294012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 0x55c11474a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.497+0000 7efce3e00640 1 -- 172.21.15.45:0/4157294012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114747820 msgr2=0x55c11475e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.497+0000 7efce3e00640 1 --2- 172.21.15.45:0/4157294012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114747820 0x55c11475e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.497+0000 7efce3e00640 1 -- 172.21.15.45:0/4157294012 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c11473e2a0 con 0x55c114752a00 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.497+0000 7efce3e00640 1 --2- 172.21.15.45:0/4157294012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c11476aad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0ce710 tx=0x7efcdc09d030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e405a9b94f807681 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.497+0000 7efccf400640 1 -- 172.21.15.45:0/4157294012 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc09c9f0 con 0x55c114752a00 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.498+0000 7efccf400640 1 -- 172.21.15.45:0/4157294012 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcdc0a1260 con 0x55c114752a00 2024-04-25T19:05:09.500 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.498+0000 7efccf400640 1 -- 172.21.15.45:0/4157294012 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0b6220 con 0x55c114752a00 2024-04-25T19:05:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.498+0000 7efcecf7d740 1 -- 172.21.15.45:0/4157294012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 msgr2=0x55c11476aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.498+0000 7efcecf7d740 1 --2- 172.21.15.45:0/4157294012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c11476aad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efcdc0ce710 tx=0x7efcdc09d030 comp rx=0 tx=0).stop 2024-04-25T19:05:09.501 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.499+0000 7efcecf7d740 1 -- 172.21.15.45:0/4157294012 shutdown_connections 2024-04-25T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.499+0000 7efcecf7d740 1 --2- 172.21.15.45:0/4157294012 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114752a00 0x55c11476aad0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.499+0000 7efcecf7d740 1 --2- 172.21.15.45:0/4157294012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114748140 0x55c11474a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.499+0000 7efcecf7d740 1 --2- 172.21.15.45:0/4157294012 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114747820 0x55c11475e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.499+0000 7efcecf7d740 1 -- 172.21.15.45:0/4157294012 >> 172.21.15.45:0/4157294012 conn(0x55c114743d30 msgr2=0x55c114742e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.499+0000 7efcecf7d740 1 -- 172.21.15.45:0/4157294012 shutdown_connections 2024-04-25T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.499+0000 7efcecf7d740 1 -- 172.21.15.45:0/4157294012 wait complete. 2024-04-25T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.499+0000 7efcecf7d740 1 -- start start 2024-04-25T19:05:09.502 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.499+0000 7efcecf7d740 1 Processor -- start 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c11475ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114748140 0x55c114758010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efcecf7d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114752a00 0x55c114758550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efcecf7d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efce41ae430 con 0x55c114752a00 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efcecf7d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7efce41af8a0 con 0x55c114748140 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efcecf7d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efce41aeb00 con 0x55c114747820 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efcecf7d740 10 client.0 _ll_register_callbacks cb 0x7efcdc09ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efce3400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c11475ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efce3400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c11475ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55920/0 (socket says 172.21.15.45:55920) 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efce3400640 1 -- 172.21.15.45:0/755698973 learned_addr learned my addr 172.21.15.45:0/755698973 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efce2a00640 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114748140 0x55c114758010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.500+0000 7efce3e00640 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114752a00 0x55c114758550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efce3400640 1 -- 172.21.15.45:0/755698973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114748140 msgr2=0x55c114758010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.505 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efce3400640 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114748140 0x55c114758010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efce3400640 1 -- 172.21.15.45:0/755698973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114752a00 msgr2=0x55c114758550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efce3400640 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114752a00 0x55c114758550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efce3400640 1 -- 172.21.15.45:0/755698973 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c114758c90 con 0x55c114747820 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efce3e00640 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114752a00 0x55c114758550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efce3400640 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c11475ac80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009860 tx=0x7efcd40305a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af9a07f3949aa833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efcccc00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efcccc00640 1 -- 172.21.15.45:0/755698973 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4030b80 con 0x55c114747820 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efcecf7d740 1 -- 172.21.15.45:0/755698973 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcdc0a1260 con 0x55c114747820 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efcccc00640 1 -- 172.21.15.45:0/755698973 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7efcd4030d20 con 0x55c114747820 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efcccc00640 1 -- 172.21.15.45:0/755698973 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efcd4061050 con 0x55c114747820 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.501+0000 7efcecf7d740 10 client.25807 fetch_fsmap learned FSMap version 32 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.502+0000 7efcecf7d740 1 -- 172.21.15.45:0/755698973 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1147c92f0 con 0x55c114747820 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.502+0000 7efcccc00640 1 -- 172.21.15.45:0/755698973 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcdc0a1260 con 0x55c114747820 2024-04-25T19:05:09.506 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.502+0000 7efcccc00640 1 -- 172.21.15.45:0/755698973 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7efcd405c030 con 0x55c114747820 2024-04-25T19:05:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.502+0000 7efcecf7d740 10 client.25807 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:05:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.502+0000 7efcecf7d740 10 client.25807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.502+0000 7efcecf7d740 20 client.25807 populate_metadata read hostname 'smithi045' 2024-04-25T19:05:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.502+0000 7efcecf7d740 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55c1147c96f0 0x55c1147cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.502+0000 7efcecf7d740 4 client.25807 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:05:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.502+0000 7efcecf7d740 1 -- 172.21.15.45:0/755698973 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1146bb320 con 0x55c1147c96f0 2024-04-25T19:05:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.502+0000 7efce2a00640 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55c1147c96f0 0x55c1147cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.503+0000 7efce2a00640 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55c1147c96f0 0x55c1147cbae0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26259d8d467f5c2f server_cookie=cd111e0f0957c02c in_seq=0 out_seq=0 2024-04-25T19:05:09.507 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.503+0000 7efcccc00640 10 client.25807 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.516+0000 7efcccc00640 1 -- 172.21.15.45:0/755698973 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1146bb320 con 0x55c1147c96f0 2024-04-25T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.516+0000 7efcccc00640 10 client.25807 handle_command_reply: tid=0 2024-04-25T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.516+0000 7efcecf7d740 1 client.25807 shutdown 2024-04-25T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.516+0000 7efcecf7d740 1 -- 172.21.15.45:0/755698973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 msgr2=0x55c11475ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcecf7d740 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c11475ac80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009860 tx=0x7efcd40305a0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcecf7d740 1 -- 172.21.15.45:0/755698973 shutdown_connections 2024-04-25T19:05:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcecf7d740 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55c1147c96f0 0x55c1147cbae0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcecf7d740 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c114752a00 0x55c114758550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcecf7d740 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55c114748140 0x55c114758010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcecf7d740 1 --2- 172.21.15.45:0/755698973 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55c114747820 0x55c11475ac80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcecf7d740 1 -- 172.21.15.45:0/755698973 >> 172.21.15.45:0/755698973 conn(0x55c114743d30 msgr2=0x55c114742600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcccc00640 0 client.25807 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcecf7d740 1 -- 172.21.15.45:0/755698973 shutdown_connections 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcecf7d740 1 -- 172.21.15.45:0/755698973 wait complete. 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.517+0000 7efcecf7d740 20 client.25807 trim_cache size 0 max 16384 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:05:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 -- 172.21.15.45:0/1906180009 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efcc80a3d10 msgr2=0x7efcc80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efcc80a3d10 0x7efcc80a61d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55c114745150 tx=0x7efcd0009260 comp rx=0 tx=0).stop 2024-04-25T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 -- 172.21.15.45:0/1906180009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce410c590 msgr2=0x7efce419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce410c590 0x7efce419a850 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00eb40 tx=0x7efcdc00c2d0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 -- 172.21.15.45:0/1906180009 shutdown_connections 2024-04-25T19:05:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efcc80a3d10 0x7efcc80a61d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7efce410c590 0x7efce419a850 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efce4103ae0 0x7efce41a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 --2- 172.21.15.45:0/1906180009 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efce41031c0 0x7efce41a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 -- 172.21.15.45:0/1906180009 >> 172.21.15.45:0/1906180009 conn(0x7efce4077f70 msgr2=0x7efce41084f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 -- 172.21.15.45:0/1906180009 shutdown_connections 2024-04-25T19:05:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:09.518+0000 7efcea000640 1 -- 172.21.15.45:0/1906180009 wait complete. 2024-04-25T19:05:10.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:10 smithi062 ceph-mon[32844]: pgmap v179: 97 pgs: 97 active+clean; 45 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-04-25T19:05:10.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:10 smithi079 ceph-mon[33002]: pgmap v179: 97 pgs: 97 active+clean; 45 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-04-25T19:05:10.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:10 smithi045 ceph-mon[27776]: pgmap v179: 97 pgs: 97 active+clean; 45 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.9 MiB/s wr, 321 op/s 2024-04-25T19:05:10.948 INFO:tasks.workunit.client.0.smithi045.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-04-25T19:05:12.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:12 smithi062 ceph-mon[32844]: pgmap v180: 97 pgs: 97 active+clean; 44 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 2.7 MiB/s wr, 467 op/s 2024-04-25T19:05:12.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:12 smithi079 ceph-mon[33002]: pgmap v180: 97 pgs: 97 active+clean; 44 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 2.7 MiB/s wr, 467 op/s 2024-04-25T19:05:12.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:12 smithi045 ceph-mon[27776]: pgmap v180: 97 pgs: 97 active+clean; 44 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 2.7 MiB/s wr, 467 op/s 2024-04-25T19:05:14.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:14 smithi062 ceph-mon[32844]: pgmap v181: 97 pgs: 97 active+clean; 44 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-25T19:05:14.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:14 smithi079 ceph-mon[33002]: pgmap v181: 97 pgs: 97 active+clean; 44 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-25T19:05:14.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:14 smithi045 ceph-mon[27776]: pgmap v181: 97 pgs: 97 active+clean; 44 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 1.9 MiB/s wr, 318 op/s 2024-04-25T19:05:16.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:16 smithi062 ceph-mon[32844]: pgmap v182: 97 pgs: 97 active+clean; 49 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 462 op/s 2024-04-25T19:05:16.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:16 smithi079 ceph-mon[33002]: pgmap v182: 97 pgs: 97 active+clean; 49 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 462 op/s 2024-04-25T19:05:16.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:16 smithi045 ceph-mon[27776]: pgmap v182: 97 pgs: 97 active+clean; 49 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.8 MiB/s wr, 462 op/s 2024-04-25T19:05:18.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:18 smithi062 ceph-mon[32844]: pgmap v183: 97 pgs: 97 active+clean; 49 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-04-25T19:05:18.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:18 smithi079 ceph-mon[33002]: pgmap v183: 97 pgs: 97 active+clean; 49 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-04-25T19:05:18.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:18 smithi045 ceph-mon[27776]: pgmap v183: 97 pgs: 97 active+clean; 49 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-04-25T19:05:20.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:20 smithi062 ceph-mon[32844]: pgmap v184: 97 pgs: 97 active+clean; 49 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-04-25T19:05:20.683 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:20 smithi079 ceph-mon[33002]: pgmap v184: 97 pgs: 97 active+clean; 49 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-04-25T19:05:20.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:20 smithi045 ceph-mon[27776]: pgmap v184: 97 pgs: 97 active+clean; 49 MiB data, 996 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 298 op/s 2024-04-25T19:05:22.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:22 smithi062 ceph-mon[32844]: pgmap v185: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.7 MiB/s wr, 436 op/s 2024-04-25T19:05:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.670+0000 7f3cb2800640 1 -- 172.21.15.45:0/4057529627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cac10c370 msgr2=0x7f3cac10e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.670+0000 7f3cb2800640 1 --2- 172.21.15.45:0/4057529627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cac10c370 0x7f3cac10e810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009ca0 tx=0x7f3ca402a940 comp rx=0 tx=0).stop 2024-04-25T19:05:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.670+0000 7f3cb2800640 1 -- 172.21.15.45:0/4057529627 shutdown_connections 2024-04-25T19:05:22.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.670+0000 7f3cb2800640 1 --2- 172.21.15.45:0/4057529627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cac10c370 0x7f3cac10e810 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.670+0000 7f3cb2800640 1 --2- 172.21.15.45:0/4057529627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cac103870 0x7f3cac10bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.670+0000 7f3cb2800640 1 --2- 172.21.15.45:0/4057529627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac102f50 0x7f3cac103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.670+0000 7f3cb2800640 1 -- 172.21.15.45:0/4057529627 >> 172.21.15.45:0/4057529627 conn(0x7f3cac0fc770 msgr2=0x7f3cac0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.670+0000 7f3cb2800640 1 -- 172.21.15.45:0/4057529627 shutdown_connections 2024-04-25T19:05:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.670+0000 7f3cb2800640 1 -- 172.21.15.45:0/4057529627 wait complete. 2024-04-25T19:05:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.671+0000 7f3cb2800640 1 Processor -- start 2024-04-25T19:05:22.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.671+0000 7f3cb2800640 1 -- start start 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb2800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cac102f50 0x7f3cac198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb2800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac103870 0x7f3cac198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb2800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cac10c370 0x7f3cac19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb2800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac112030 con 0x7f3cac10c370 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb2800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac111eb0 con 0x7f3cac103870 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb2800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1121b0 con 0x7f3cac102f50 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cac102f50 0x7f3cac198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac103870 0x7f3cac198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cac102f50 0x7f3cac198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:36090/0 (socket says 172.21.15.45:36090) 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac103870 0x7f3cac198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:56398/0 (socket says 172.21.15.45:56398) 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb1400640 1 -- 172.21.15.45:0/1843425788 learned_addr learned my addr 172.21.15.45:0/1843425788 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.672+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cac10c370 0x7f3cac19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3cb1400640 1 -- 172.21.15.45:0/1843425788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac103870 msgr2=0x7f3cac198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:22.676 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3cb1400640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac103870 0x7f3cac198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3cb1400640 1 -- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cac10c370 msgr2=0x7f3cac19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3cb1400640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cac10c370 0x7f3cac19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3cb1400640 1 -- 172.21.15.45:0/1843425788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cac19d530 con 0x7f3cac102f50 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3cb0a00640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac103870 0x7f3cac198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cac10c370 0x7f3cac19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3cb1400640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cac102f50 0x7f3cac198590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000e9f0 tx=0x7f3ca000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3caa000640 1 -- 172.21.15.45:0/1843425788 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000cd80 con 0x7f3cac102f50 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3caa000640 1 -- 172.21.15.45:0/1843425788 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca00108e0 con 0x7f3cac102f50 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3cb2800640 1 -- 172.21.15.45:0/1843425788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cac19d820 con 0x7f3cac102f50 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.673+0000 7f3caa000640 1 -- 172.21.15.45:0/1843425788 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0018980 con 0x7f3cac102f50 2024-04-25T19:05:22.677 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.674+0000 7f3cb2800640 1 -- 172.21.15.45:0/1843425788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cac1a50b0 con 0x7f3cac102f50 2024-04-25T19:05:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.678+0000 7f3caa000640 1 -- 172.21.15.45:0/1843425788 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f3ca0010430 con 0x7f3cac102f50 2024-04-25T19:05:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.678+0000 7f3cb6d72740 1 Processor -- start 2024-04-25T19:05:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.678+0000 7f3cb6d72740 1 -- start start 2024-04-25T19:05:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 0x561e805add90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805b8b90 0x561e805baf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561e80513900 con 0x561e805b8b90 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561e804b9ef0 con 0x561e805ae2d0 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561e8050f8c0 con 0x561e805ad9b0 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 0x561e805add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 0x561e805add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:36092/0 (socket says 172.21.15.45:36092) 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb1e00640 1 -- 172.21.15.45:0/2766488553 learned_addr learned my addr 172.21.15.45:0/2766488553 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:56412/0 (socket says 172.21.15.45:56412) 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.679+0000 7f3cb0a00640 1 --2- 172.21.15.45:0/2766488553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805b8b90 0x561e805baf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.680+0000 7f3caa000640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f3c940a3d80 0x7f3c940a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.680+0000 7f3caa000640 1 -- 172.21.15.45:0/1843425788 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f3ca00d0b80 con 0x7f3cac102f50 2024-04-25T19:05:22.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.680+0000 7f3cb1400640 1 -- 172.21.15.45:0/2766488553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 msgr2=0x561e805add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.680+0000 7f3cb1400640 1 --2- 172.21.15.45:0/2766488553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 0x561e805add90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.680+0000 7f3cb1400640 1 -- 172.21.15.45:0/2766488553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805b8b90 msgr2=0x561e805baf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.680+0000 7f3cb1400640 1 --2- 172.21.15.45:0/2766488553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805b8b90 0x561e805baf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.680+0000 7f3cb1400640 1 -- 172.21.15.45:0/2766488553 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e80226210 con 0x561e805ae2d0 2024-04-25T19:05:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.680+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f3c940a3d80 0x7f3c940a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.680+0000 7f3cb1400640 1 --2- 172.21.15.45:0/2766488553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b8650 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00ce270 tx=0x7f3ca000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec432d08ab7a2270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.680+0000 7f3c9e600640 1 -- 172.21.15.45:0/2766488553 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00d4240 con 0x561e805ae2d0 2024-04-25T19:05:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.681+0000 7f3c9e600640 1 -- 172.21.15.45:0/2766488553 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca009d1c0 con 0x561e805ae2d0 2024-04-25T19:05:22.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.681+0000 7f3c9e600640 1 -- 172.21.15.45:0/2766488553 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0099130 con 0x561e805ae2d0 2024-04-25T19:05:22.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:22 smithi079 ceph-mon[33002]: pgmap v185: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.7 MiB/s wr, 436 op/s 2024-04-25T19:05:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.681+0000 7f3cb6d72740 1 -- 172.21.15.45:0/2766488553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 msgr2=0x561e805b8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.681+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f3c940a3d80 0x7f3c940a6240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009a60 tx=0x7f3ca40095c0 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.681+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/2766488553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b8650 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3ca00ce270 tx=0x7f3ca000cf20 comp rx=0 tx=0).stop 2024-04-25T19:05:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.682+0000 7f3cb6d72740 1 -- 172.21.15.45:0/2766488553 shutdown_connections 2024-04-25T19:05:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.682+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/2766488553 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805b8b90 0x561e805baf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.682+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/2766488553 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b8650 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.682+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/2766488553 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 0x561e805add90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.682+0000 7f3cb6d72740 1 -- 172.21.15.45:0/2766488553 >> 172.21.15.45:0/2766488553 conn(0x561e805a9ec0 msgr2=0x561e805aa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.682+0000 7f3cb6d72740 1 -- 172.21.15.45:0/2766488553 shutdown_connections 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.682+0000 7f3cb6d72740 1 -- 172.21.15.45:0/2766488553 wait complete. 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.682+0000 7f3cb6d72740 1 -- start start 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.682+0000 7f3cb6d72740 1 Processor -- start 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 0x561e805b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805b8b90 0x561e805b65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561e805bb760 con 0x561e805b8b90 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561e804cd310 con 0x561e805ae2d0 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561e805bb8e0 con 0x561e805ad9b0 2024-04-25T19:05:22.690 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb6d72740 10 client.0 _ll_register_callbacks cb 0x561e8049bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:56422/0 (socket says 172.21.15.45:56422) 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805b8b90 0x561e805b65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb0a00640 1 -- 172.21.15.45:0/845235999 learned_addr learned my addr 172.21.15.45:0/845235999 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.683+0000 7f3cb1400640 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 0x561e805b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3cb0a00640 1 -- 172.21.15.45:0/845235999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 msgr2=0x561e805b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3cb0a00640 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 0x561e805b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3cb0a00640 1 -- 172.21.15.45:0/845235999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805b8b90 msgr2=0x561e805b65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3cb0a00640 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805b8b90 0x561e805b65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3cb0a00640 1 -- 172.21.15.45:0/845235999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e802f2620 con 0x561e805ae2d0 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3cb0a00640 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b60b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c980129f0 tx=0x7f3c98012eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646d46e172dd90f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3c92a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3c92a00640 1 -- 172.21.15.45:0/845235999 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9800f9e0 con 0x561e805ae2d0 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3c92a00640 1 -- 172.21.15.45:0/845235999 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f3c9800fb80 con 0x561e805ae2d0 2024-04-25T19:05:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3cb6d72740 1 -- 172.21.15.45:0/845235999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ca009d1c0 con 0x561e805ae2d0 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3c92a00640 1 -- 172.21.15.45:0/845235999 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3c9800fd80 con 0x561e805ae2d0 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3cb6d72740 10 client.25802 fetch_fsmap learned FSMap version 32 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.684+0000 7f3cb6d72740 1 -- 172.21.15.45:0/845235999 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561e805c7ac0 con 0x561e805ae2d0 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.685+0000 7f3c92a00640 1 -- 172.21.15.45:0/845235999 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ca009d1c0 con 0x561e805ae2d0 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.685+0000 7f3c92a00640 1 -- 172.21.15.45:0/845235999 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f3c98003bf0 con 0x561e805ae2d0 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.685+0000 7f3cb6d72740 10 client.25802 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.685+0000 7f3cb6d72740 10 client.25802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.685+0000 7f3cb6d72740 20 client.25802 populate_metadata read hostname 'smithi045' 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.685+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561e805c8f80 0x561e805cb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.685+0000 7f3cb6d72740 4 client.25802 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.685+0000 7f3cb6d72740 1 -- 172.21.15.45:0/845235999 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e805278b0 con 0x561e805c8f80 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.685+0000 7f3cb1400640 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561e805c8f80 0x561e805cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.686+0000 7f3cb1400640 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561e805c8f80 0x561e805cb390 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e877c26c7c815d server_cookie=eef708e07cf739a6 in_seq=0 out_seq=0 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.686+0000 7f3c92a00640 10 client.25802 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.686+0000 7f3c92a00640 1 -- 172.21.15.45:0/845235999 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561e805278b0 con 0x561e805c8f80 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.686+0000 7f3c92a00640 10 client.25802 handle_command_reply: tid=0 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.687+0000 7f3cb6d72740 1 client.25802 shutdown 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.687+0000 7f3cb6d72740 1 -- 172.21.15.45:0/845235999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 msgr2=0x561e805b60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.687+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b60b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c980129f0 tx=0x7f3c98012eb0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.687+0000 7f3cb6d72740 1 -- 172.21.15.45:0/845235999 shutdown_connections 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.687+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561e805c8f80 0x561e805cb390 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.687+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805b8b90 0x561e805b65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.687+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b60b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.687+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/845235999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ad9b0 0x561e805b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.687+0000 7f3c92a00640 0 client.25802 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.687+0000 7f3cb6d72740 1 -- 172.21.15.45:0/845235999 >> 172.21.15.45:0/845235999 conn(0x561e805a9ec0 msgr2=0x561e805afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.688+0000 7f3cb6d72740 1 -- 172.21.15.45:0/845235999 shutdown_connections 2024-04-25T19:05:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.688+0000 7f3cb6d72740 1 -- 172.21.15.45:0/845235999 wait complete. 2024-04-25T19:05:22.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.688+0000 7f3cb6d72740 20 client.25802 trim_cache size 0 max 16384 2024-04-25T19:05:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.708+0000 7f3cb6d72740 1 Processor -- start 2024-04-25T19:05:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.708+0000 7f3cb6d72740 1 -- start start 2024-04-25T19:05:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 0x561e805c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805b8b90 0x561e805a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1b0360 con 0x561e805ad9b0 2024-04-25T19:05:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1adc30 con 0x561e805ae2d0 2024-04-25T19:05:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1aba80 con 0x561e805b8b90 2024-04-25T19:05:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805b8b90 0x561e805a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805b8b90 0x561e805a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:36106/0 (socket says 172.21.15.45:36106) 2024-04-25T19:05:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb1e00640 1 -- 172.21.15.45:0/1716231487 learned_addr learned my addr 172.21.15.45:0/1716231487 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:56436/0 (socket says 172.21.15.45:56436) 2024-04-25T19:05:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.709+0000 7f3cb1400640 1 --2- 172.21.15.45:0/1716231487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 0x561e805c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.710+0000 7f3cb0a00640 1 -- 172.21.15.45:0/1716231487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805b8b90 msgr2=0x561e805a8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.710+0000 7f3cb0a00640 1 --2- 172.21.15.45:0/1716231487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805b8b90 0x561e805a8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.710+0000 7f3cb0a00640 1 -- 172.21.15.45:0/1716231487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 msgr2=0x561e805c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.710+0000 7f3cb0a00640 1 --2- 172.21.15.45:0/1716231487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 0x561e805c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.710+0000 7f3cb0a00640 1 -- 172.21.15.45:0/1716231487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e805a5820 con 0x561e805ae2d0 2024-04-25T19:05:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.710+0000 7f3cb1400640 1 --2- 172.21.15.45:0/1716231487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 0x561e805c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.710+0000 7f3cb0a00640 1 --2- 172.21.15.45:0/1716231487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b0370 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x561e805b53f0 tx=0x7f3c980059b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=420b792f76555eed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.710+0000 7f3c9dc00640 1 -- 172.21.15.45:0/1716231487 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c980050f0 con 0x561e805ae2d0 2024-04-25T19:05:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.710+0000 7f3c9dc00640 1 -- 172.21.15.45:0/1716231487 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c980256c0 con 0x561e805ae2d0 2024-04-25T19:05:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.710+0000 7f3c9dc00640 1 -- 172.21.15.45:0/1716231487 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98003850 con 0x561e805ae2d0 2024-04-25T19:05:22.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:22 smithi045 ceph-mon[27776]: pgmap v185: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 2.7 MiB/s wr, 436 op/s 2024-04-25T19:05:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.711+0000 7f3cb6d72740 1 -- 172.21.15.45:0/1716231487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 msgr2=0x561e805b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.711+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/1716231487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b0370 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x561e805b53f0 tx=0x7f3c980059b0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.711+0000 7f3cb6d72740 1 -- 172.21.15.45:0/1716231487 shutdown_connections 2024-04-25T19:05:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.711+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/1716231487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805b8b90 0x561e805a8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.711+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/1716231487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805ae2d0 0x561e805b0370 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.711+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/1716231487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 0x561e805c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.711+0000 7f3cb6d72740 1 -- 172.21.15.45:0/1716231487 >> 172.21.15.45:0/1716231487 conn(0x561e805a9ec0 msgr2=0x561e805a8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.712+0000 7f3cb6d72740 1 -- 172.21.15.45:0/1716231487 shutdown_connections 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.712+0000 7f3cb6d72740 1 -- 172.21.15.45:0/1716231487 wait complete. 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.712+0000 7f3cb6d72740 1 -- start start 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.712+0000 7f3cb6d72740 1 Processor -- start 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 0x561e805c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ae2d0 0x561e805be140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb6d72740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805b8b90 0x561e805be680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1ac0f0 con 0x561e805ad9b0 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1ad560 con 0x561e805b8b90 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb6d72740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3cac1ac7c0 con 0x561e805ae2d0 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb6d72740 10 client.0 _ll_register_callbacks cb 0x7f3c98012620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb1400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 0x561e805c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.717 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805b8b90 0x561e805be680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb0a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ae2d0 0x561e805be140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb1e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805b8b90 0x561e805be680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:56442/0 (socket says 172.21.15.45:56442) 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.713+0000 7f3cb1e00640 1 -- 172.21.15.45:0/3020776611 learned_addr learned my addr 172.21.15.45:0/3020776611 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3cb1e00640 1 -- 172.21.15.45:0/3020776611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ae2d0 msgr2=0x561e805be140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ae2d0 0x561e805be140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3cb1e00640 1 -- 172.21.15.45:0/3020776611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 msgr2=0x561e805c0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 0x561e805c0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3cb1e00640 1 -- 172.21.15.45:0/3020776611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e805bed20 con 0x561e805b8b90 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3cb0a00640 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ae2d0 0x561e805be140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3cb1e00640 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805b8b90 0x561e805be680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4006010 tx=0x7f3ca402a940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=648747e348e59ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3c92a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3c92a00640 1 -- 172.21.15.45:0/3020776611 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4034040 con 0x561e805b8b90 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3c92a00640 1 -- 172.21.15.45:0/3020776611 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f3ca4003970 con 0x561e805b8b90 2024-04-25T19:05:22.718 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.714+0000 7f3cb6d72740 1 -- 172.21.15.45:0/3020776611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c980256c0 con 0x561e805b8b90 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3c92a00640 1 -- 172.21.15.45:0/3020776611 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ca4003b70 con 0x561e805b8b90 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3cb6d72740 10 client.25808 fetch_fsmap learned FSMap version 32 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3cb6d72740 1 -- 172.21.15.45:0/3020776611 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561e806337f0 con 0x561e805b8b90 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3c92a00640 1 -- 172.21.15.45:0/3020776611 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c980256c0 con 0x561e805b8b90 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3c92a00640 1 -- 172.21.15.45:0/3020776611 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f3ca4030ce0 con 0x561e805b8b90 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3cb6d72740 10 client.25808 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3cb6d72740 10 client.25808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3cb6d72740 20 client.25808 populate_metadata read hostname 'smithi045' 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561e80633f70 0x561e80636390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3cb6d72740 4 client.25808 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3cb6d72740 1 -- 172.21.15.45:0/3020776611 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e80521320 con 0x561e80633f70 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.715+0000 7f3cb1400640 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561e80633f70 0x561e80636390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:22.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.716+0000 7f3cb1400640 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561e80633f70 0x561e80636390 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a5d314f4f2d52c5 server_cookie=9a5b417ed0e75596 in_seq=0 out_seq=0 2024-04-25T19:05:22.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.716+0000 7f3c92a00640 10 client.25808 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3c92a00640 1 -- 172.21.15.45:0/3020776611 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e80521320 con 0x561e80633f70 2024-04-25T19:05:22.731 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3c92a00640 10 client.25808 handle_command_reply: tid=0 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 client.25808 shutdown 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 -- 172.21.15.45:0/3020776611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805b8b90 msgr2=0x561e805be680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805b8b90 0x561e805be680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4006010 tx=0x7f3ca402a940 comp rx=0 tx=0).stop 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 -- 172.21.15.45:0/3020776611 shutdown_connections 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561e80633f70 0x561e80636390 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561e805b8b90 0x561e805be680 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561e805ae2d0 0x561e805be140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3c92a00640 0 client.25808 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 --2- 172.21.15.45:0/3020776611 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561e805ad9b0 0x561e805c0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 -- 172.21.15.45:0/3020776611 >> 172.21.15.45:0/3020776611 conn(0x561e805a9ec0 msgr2=0x561e805b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 -- 172.21.15.45:0/3020776611 shutdown_connections 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.729+0000 7f3cb6d72740 1 -- 172.21.15.45:0/3020776611 wait complete. 2024-04-25T19:05:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb6d72740 20 client.25808 trim_cache size 0 max 16384 2024-04-25T19:05:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:05:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:05:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:05:22.733 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:05:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb2800640 1 -- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f3c940a3d80 msgr2=0x7f3c940a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb2800640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f3c940a3d80 0x7f3c940a6240 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009a60 tx=0x7f3ca40095c0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb2800640 1 -- 172.21.15.45:0/1843425788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cac102f50 msgr2=0x7f3cac198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb2800640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cac102f50 0x7f3cac198590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000e9f0 tx=0x7f3ca000eeb0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb2800640 1 -- 172.21.15.45:0/1843425788 shutdown_connections 2024-04-25T19:05:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb2800640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f3c940a3d80 0x7f3c940a6240 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb2800640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3cac10c370 0x7f3cac19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb2800640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f3cac103870 0x7f3cac198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb2800640 1 --2- 172.21.15.45:0/1843425788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cac102f50 0x7f3cac198590 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:22.734 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.730+0000 7f3cb2800640 1 -- 172.21.15.45:0/1843425788 >> 172.21.15.45:0/1843425788 conn(0x7f3cac0fc770 msgr2=0x7f3cac0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.731+0000 7f3cb2800640 1 -- 172.21.15.45:0/1843425788 shutdown_connections 2024-04-25T19:05:22.735 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:22.731+0000 7f3cb2800640 1 -- 172.21.15.45:0/1843425788 wait complete. 2024-04-25T19:05:24.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:24 smithi062 ceph-mon[32844]: pgmap v186: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 282 op/s 2024-04-25T19:05:24.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:24 smithi079 ceph-mon[33002]: pgmap v186: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 282 op/s 2024-04-25T19:05:24.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:24 smithi045 ceph-mon[27776]: pgmap v186: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.8 MiB/s wr, 282 op/s 2024-04-25T19:05:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:26 smithi062 ceph-mon[32844]: pgmap v187: 97 pgs: 97 active+clean; 49 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 416 op/s 2024-04-25T19:05:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:26 smithi079 ceph-mon[33002]: pgmap v187: 97 pgs: 97 active+clean; 49 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 416 op/s 2024-04-25T19:05:26.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:26 smithi045 ceph-mon[27776]: pgmap v187: 97 pgs: 97 active+clean; 49 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 2.7 MiB/s wr, 416 op/s 2024-04-25T19:05:27.926 INFO:tasks.workunit.client.0.smithi045.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-04-25T19:05:27.927 INFO:tasks.workunit.client.0.smithi045.stdout:All 50000 operations completed A-OK! 2024-04-25T19:05:27.939 INFO:tasks.workunit.client.0.smithi045.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-04-25T19:05:27.942 INFO:tasks.workunit.client.0.smithi045.stdout:Seed set to 1 2024-04-25T19:05:28.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:28 smithi062 ceph-mon[32844]: pgmap v188: 97 pgs: 97 active+clean; 49 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-04-25T19:05:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:28 smithi079 ceph-mon[33002]: pgmap v188: 97 pgs: 97 active+clean; 49 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-04-25T19:05:28.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:28 smithi045 ceph-mon[27776]: pgmap v188: 97 pgs: 97 active+clean; 49 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-04-25T19:05:29.372 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T19:05:29.372 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T19:05:29.376 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T19:05:29.376 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T19:05:29.376 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T19:05:29.380 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T19:05:29.385 INFO:tasks.workunit.client.0.smithi045.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T19:05:29.385 INFO:tasks.workunit.client.0.smithi045.stdout:skipping zero size read 2024-04-25T19:05:29.405 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0x1c37b90 2024-04-25T19:05:29.420 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0x5bd330d 2024-04-25T19:05:29.453 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0x637bf99 2024-04-25T19:05:29.960 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0x63d9b9c 2024-04-25T19:05:30.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:30 smithi062 ceph-mon[32844]: pgmap v189: 97 pgs: 97 active+clean; 49 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-04-25T19:05:30.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:30 smithi079 ceph-mon[33002]: pgmap v189: 97 pgs: 97 active+clean; 49 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-04-25T19:05:30.716 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:30 smithi045 ceph-mon[27776]: pgmap v189: 97 pgs: 97 active+clean; 49 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.8 MiB/s wr, 272 op/s 2024-04-25T19:05:32.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:32 smithi062 ceph-mon[32844]: pgmap v190: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 358 op/s 2024-04-25T19:05:32.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:32 smithi079 ceph-mon[33002]: pgmap v190: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 358 op/s 2024-04-25T19:05:32.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:32 smithi045 ceph-mon[27776]: pgmap v190: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.3 MiB/s wr, 358 op/s 2024-04-25T19:05:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:34 smithi062 ceph-mon[32844]: pgmap v191: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-04-25T19:05:34.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:34 smithi079 ceph-mon[33002]: pgmap v191: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-04-25T19:05:34.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:34 smithi045 ceph-mon[27776]: pgmap v191: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-04-25T19:05:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.880+0000 7fe570a00640 1 -- 172.21.15.45:0/2723658236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c104ba0 msgr2=0x7fe56c108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.880+0000 7fe570a00640 1 --2- 172.21.15.45:0/2723658236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c104ba0 0x7fe56c108be0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe55c01ab20 tx=0x7fe55c03b650 comp rx=0 tx=0).stop 2024-04-25T19:05:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.880+0000 7fe570a00640 1 -- 172.21.15.45:0/2723658236 shutdown_connections 2024-04-25T19:05:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.880+0000 7fe570a00640 1 --2- 172.21.15.45:0/2723658236 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56c1092c0 0x7fe56c110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.880+0000 7fe570a00640 1 --2- 172.21.15.45:0/2723658236 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c104ba0 0x7fe56c108be0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.880+0000 7fe570a00640 1 --2- 172.21.15.45:0/2723658236 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c1041f0 0x7fe56c1045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.880+0000 7fe570a00640 1 -- 172.21.15.45:0/2723658236 >> 172.21.15.45:0/2723658236 conn(0x7fe56c0ffa50 msgr2=0x7fe56c101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.880+0000 7fe570a00640 1 -- 172.21.15.45:0/2723658236 shutdown_connections 2024-04-25T19:05:35.883 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.880+0000 7fe570a00640 1 -- 172.21.15.45:0/2723658236 wait complete. 2024-04-25T19:05:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.881+0000 7fe570a00640 1 Processor -- start 2024-04-25T19:05:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe570a00640 1 -- start start 2024-04-25T19:05:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe570a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1041f0 0x7fe56c1109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe570a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56c1092c0 0x7fe56c10cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe570a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c10d010 0x7fe56c10d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe570a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c06e200 con 0x7fe56c10d010 2024-04-25T19:05:35.885 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe570a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c06e080 con 0x7fe56c1092c0 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe570a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c06e380 con 0x7fe56c1041f0 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe56b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1041f0 0x7fe56c1109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe56b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1041f0 0x7fe56c1109f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55234/0 (socket says 172.21.15.45:55234) 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe56b400640 1 -- 172.21.15.45:0/3592092875 learned_addr learned my addr 172.21.15.45:0/3592092875 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.882+0000 7fe56aa00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56c1092c0 0x7fe56c10cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.883+0000 7fe56be00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c10d010 0x7fe56c10d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.883+0000 7fe56b400640 1 -- 172.21.15.45:0/3592092875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56c1092c0 msgr2=0x7fe56c10cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.883+0000 7fe56b400640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56c1092c0 0x7fe56c10cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.883+0000 7fe56b400640 1 -- 172.21.15.45:0/3592092875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c10d010 msgr2=0x7fe56c10d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.883+0000 7fe56b400640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c10d010 0x7fe56c10d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.883+0000 7fe56b400640 1 -- 172.21.15.45:0/3592092875 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c19e740 con 0x7fe56c1041f0 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.883+0000 7fe56aa00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56c1092c0 0x7fe56c10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:05:35.886 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.883+0000 7fe56b400640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1041f0 0x7fe56c1109f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe55400e9f0 tx=0x7fe55400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.884+0000 7fe563e00640 1 -- 172.21.15.45:0/3592092875 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55400cd80 con 0x7fe56c1041f0 2024-04-25T19:05:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.884+0000 7fe563e00640 1 -- 172.21.15.45:0/3592092875 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5540108e0 con 0x7fe56c1041f0 2024-04-25T19:05:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.884+0000 7fe570a00640 1 -- 172.21.15.45:0/3592092875 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c19e970 con 0x7fe56c1041f0 2024-04-25T19:05:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.884+0000 7fe563e00640 1 -- 172.21.15.45:0/3592092875 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554018980 con 0x7fe56c1041f0 2024-04-25T19:05:35.887 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.884+0000 7fe570a00640 1 -- 172.21.15.45:0/3592092875 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c19eee0 con 0x7fe56c1041f0 2024-04-25T19:05:35.889 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.887+0000 7fe563e00640 1 -- 172.21.15.45:0/3592092875 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe554010430 con 0x7fe56c1041f0 2024-04-25T19:05:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.889+0000 7fe563e00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe54c0a3d80 0x7fe54c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.889+0000 7fe563e00640 1 -- 172.21.15.45:0/3592092875 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fe5540d0ba0 con 0x7fe56c1041f0 2024-04-25T19:05:35.891 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.889+0000 7fe574f69740 1 Processor -- start 2024-04-25T19:05:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe574f69740 1 -- start start 2024-04-25T19:05:35.892 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe56aa00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe54c0a3d80 0x7fe54c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe574f69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60c1c0 0x55ff0e60c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe574f69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60cae0 0x55ff0e616e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe574f69740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e619790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe574f69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e569900 con 0x55ff0e60cae0 2024-04-25T19:05:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe574f69740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e50fef0 con 0x55ff0e6173a0 2024-04-25T19:05:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe574f69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e5658c0 con 0x55ff0e60c1c0 2024-04-25T19:05:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe56b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60c1c0 0x55ff0e60c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe56b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60c1c0 0x55ff0e60c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55250/0 (socket says 172.21.15.45:55250) 2024-04-25T19:05:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.890+0000 7fe56b400640 1 -- 172.21.15.45:0/3939065947 learned_addr learned my addr 172.21.15.45:0/3939065947 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:35.894 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe56be00640 1 --2- 172.21.15.45:0/3939065947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e619790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe56aa00640 1 --2- 172.21.15.45:0/3939065947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60cae0 0x55ff0e616e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe56b400640 1 -- 172.21.15.45:0/3939065947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 msgr2=0x55ff0e619790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe56b400640 1 --2- 172.21.15.45:0/3939065947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e619790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe56b400640 1 -- 172.21.15.45:0/3939065947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60cae0 msgr2=0x55ff0e616e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe56b400640 1 --2- 172.21.15.45:0/3939065947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60cae0 0x55ff0e616e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe56b400640 1 -- 172.21.15.45:0/3939065947 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff0e27c210 con 0x55ff0e60c1c0 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe56aa00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe54c0a3d80 0x7fe54c0a6240 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe55c006010 tx=0x7fe55c0051d0 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe56b400640 1 --2- 172.21.15.45:0/3939065947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60c1c0 0x55ff0e60c5a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe554098f40 tx=0x7fe5540107e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13cb89eee06bc3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe55be00640 1 -- 172.21.15.45:0/3939065947 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5540a0e90 con 0x55ff0e60c1c0 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe55be00640 1 -- 172.21.15.45:0/3939065947 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5540a5750 con 0x55ff0e60c1c0 2024-04-25T19:05:35.895 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.891+0000 7fe55be00640 1 -- 172.21.15.45:0/3939065947 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554018b20 con 0x55ff0e60c1c0 2024-04-25T19:05:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.893+0000 7fe574f69740 1 -- 172.21.15.45:0/3939065947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60c1c0 msgr2=0x55ff0e60c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:35.897 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.893+0000 7fe574f69740 1 --2- 172.21.15.45:0/3939065947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60c1c0 0x55ff0e60c5a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe554098f40 tx=0x7fe5540107e0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.893+0000 7fe574f69740 1 -- 172.21.15.45:0/3939065947 shutdown_connections 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.893+0000 7fe574f69740 1 --2- 172.21.15.45:0/3939065947 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e619790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.893+0000 7fe574f69740 1 --2- 172.21.15.45:0/3939065947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60cae0 0x55ff0e616e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.893+0000 7fe574f69740 1 --2- 172.21.15.45:0/3939065947 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60c1c0 0x55ff0e60c5a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.893+0000 7fe574f69740 1 -- 172.21.15.45:0/3939065947 >> 172.21.15.45:0/3939065947 conn(0x55ff0e6086d0 msgr2=0x55ff0e608ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.893+0000 7fe574f69740 1 -- 172.21.15.45:0/3939065947 shutdown_connections 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.893+0000 7fe574f69740 1 -- 172.21.15.45:0/3939065947 wait complete. 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.893+0000 7fe574f69740 1 -- start start 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe574f69740 1 Processor -- start 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe574f69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e6143a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe574f69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e6148e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe574f69740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e614e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.898 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe574f69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e619fd0 con 0x55ff0e60c1c0 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe574f69740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e523310 con 0x55ff0e6173a0 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe574f69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55ff0e61a150 con 0x55ff0e60cae0 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe574f69740 10 client.0 _ll_register_callbacks cb 0x55ff0e4f1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe56aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e6148e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe56be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e614e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe56aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e6148e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55258/0 (socket says 172.21.15.45:55258) 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.894+0000 7fe56aa00640 1 -- 172.21.15.45:0/17174264 learned_addr learned my addr 172.21.15.45:0/17174264 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.895+0000 7fe56b400640 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e6143a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.895+0000 7fe56b400640 1 -- 172.21.15.45:0/17174264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 msgr2=0x55ff0e6148e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.895+0000 7fe56b400640 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e6148e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.895+0000 7fe56b400640 1 -- 172.21.15.45:0/17174264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 msgr2=0x55ff0e614e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:35.899 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.895+0000 7fe56b400640 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e614e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.895+0000 7fe56b400640 1 -- 172.21.15.45:0/17174264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff0e348620 con 0x55ff0e60c1c0 2024-04-25T19:05:35.900 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.895+0000 7fe56be00640 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e614e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.896+0000 7fe56b400640 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e6143a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe554023000 tx=0x7fe55409cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a6fb8b8affcee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.896+0000 7fe558c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.896+0000 7fe558c00640 1 -- 172.21.15.45:0/17174264 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5540b5070 con 0x55ff0e60c1c0 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.896+0000 7fe558c00640 1 -- 172.21.15.45:0/17174264 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fe55409bf30 con 0x55ff0e60c1c0 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.896+0000 7fe574f69740 1 -- 172.21.15.45:0/17174264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5540a5750 con 0x55ff0e60c1c0 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.897+0000 7fe558c00640 1 -- 172.21.15.45:0/17174264 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe55409c130 con 0x55ff0e60c1c0 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.897+0000 7fe574f69740 10 client.15675 fetch_fsmap learned FSMap version 32 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.897+0000 7fe574f69740 1 -- 172.21.15.45:0/17174264 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ff0e626290 con 0x55ff0e60c1c0 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.897+0000 7fe558c00640 1 -- 172.21.15.45:0/17174264 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5540a5750 con 0x55ff0e60c1c0 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.897+0000 7fe558c00640 1 -- 172.21.15.45:0/17174264 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fe55409b9a0 con 0x55ff0e60c1c0 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.897+0000 7fe574f69740 10 client.15675 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:05:35.903 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.897+0000 7fe574f69740 10 client.15675 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.898+0000 7fe574f69740 20 client.15675 populate_metadata read hostname 'smithi045' 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.898+0000 7fe574f69740 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ff0e6275e0 0x55ff0e6299d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.898+0000 7fe574f69740 4 client.15675 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.898+0000 7fe574f69740 1 -- 172.21.15.45:0/17174264 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff0e57d8b0 con 0x55ff0e6275e0 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.898+0000 7fe56aa00640 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ff0e6275e0 0x55ff0e6299d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.899+0000 7fe56aa00640 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ff0e6275e0 0x55ff0e6299d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a963caa907bb64 server_cookie=24627ccc6a551f96 in_seq=0 out_seq=0 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.899+0000 7fe558c00640 10 client.15675 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.899+0000 7fe558c00640 1 -- 172.21.15.45:0/17174264 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff0e57d8b0 con 0x55ff0e6275e0 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.899+0000 7fe558c00640 10 client.15675 handle_command_reply: tid=0 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.899+0000 7fe574f69740 1 client.15675 shutdown 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 1 -- 172.21.15.45:0/17174264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 msgr2=0x55ff0e6143a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e6143a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe554023000 tx=0x7fe55409cc60 comp rx=0 tx=0).stop 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 1 -- 172.21.15.45:0/17174264 shutdown_connections 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ff0e6275e0 0x55ff0e6299d0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e614e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e6148e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 1 --2- 172.21.15.45:0/17174264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e6143a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 1 -- 172.21.15.45:0/17174264 >> 172.21.15.45:0/17174264 conn(0x55ff0e6086d0 msgr2=0x55ff0e5ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe558c00640 0 client.15675 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 1 -- 172.21.15.45:0/17174264 shutdown_connections 2024-04-25T19:05:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 1 -- 172.21.15.45:0/17174264 wait complete. 2024-04-25T19:05:35.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.900+0000 7fe574f69740 20 client.15675 trim_cache size 0 max 16384 2024-04-25T19:05:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.921+0000 7fe574f69740 1 Processor -- start 2024-04-25T19:05:35.924 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.922+0000 7fe574f69740 1 -- start start 2024-04-25T19:05:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe574f69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e622e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe574f69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e60e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe574f69740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e62f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe574f69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1a59f0 con 0x55ff0e60c1c0 2024-04-25T19:05:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe574f69740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1a32c0 con 0x55ff0e6173a0 2024-04-25T19:05:35.925 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe574f69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1a1110 con 0x55ff0e60cae0 2024-04-25T19:05:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe56b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e622e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe56aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e60e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe56aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e60e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55272/0 (socket says 172.21.15.45:55272) 2024-04-25T19:05:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe56aa00640 1 -- 172.21.15.45:0/3805280716 learned_addr learned my addr 172.21.15.45:0/3805280716 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.923+0000 7fe56be00640 1 --2- 172.21.15.45:0/3805280716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e62f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.924+0000 7fe56aa00640 1 -- 172.21.15.45:0/3805280716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 msgr2=0x55ff0e62f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:35.926 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.924+0000 7fe56aa00640 1 --2- 172.21.15.45:0/3805280716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e62f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.924+0000 7fe56aa00640 1 -- 172.21.15.45:0/3805280716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 msgr2=0x55ff0e622e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.924+0000 7fe56aa00640 1 --2- 172.21.15.45:0/3805280716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e622e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.924+0000 7fe56aa00640 1 -- 172.21.15.45:0/3805280716 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff0e5fb820 con 0x55ff0e60cae0 2024-04-25T19:05:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.924+0000 7fe56aa00640 1 --2- 172.21.15.45:0/3805280716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e60e280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe55c03bc40 tx=0x7fe55c050000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=524aeea1a1c6fa31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.925+0000 7fe55b400640 1 -- 172.21.15.45:0/3805280716 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c068070 con 0x55ff0e60cae0 2024-04-25T19:05:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.925+0000 7fe55b400640 1 -- 172.21.15.45:0/3805280716 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe55c004b30 con 0x55ff0e60cae0 2024-04-25T19:05:35.927 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.925+0000 7fe55b400640 1 -- 172.21.15.45:0/3805280716 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c046050 con 0x55ff0e60cae0 2024-04-25T19:05:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.926+0000 7fe574f69740 1 -- 172.21.15.45:0/3805280716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 msgr2=0x55ff0e60e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.926+0000 7fe574f69740 1 --2- 172.21.15.45:0/3805280716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e60e280 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe55c03bc40 tx=0x7fe55c050000 comp rx=0 tx=0).stop 2024-04-25T19:05:35.928 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.926+0000 7fe574f69740 1 -- 172.21.15.45:0/3805280716 shutdown_connections 2024-04-25T19:05:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.926+0000 7fe574f69740 1 --2- 172.21.15.45:0/3805280716 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e62f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.926+0000 7fe574f69740 1 --2- 172.21.15.45:0/3805280716 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e60e280 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.926+0000 7fe574f69740 1 --2- 172.21.15.45:0/3805280716 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e622e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.926+0000 7fe574f69740 1 -- 172.21.15.45:0/3805280716 >> 172.21.15.45:0/3805280716 conn(0x55ff0e6086d0 msgr2=0x55ff0e60db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.926+0000 7fe574f69740 1 -- 172.21.15.45:0/3805280716 shutdown_connections 2024-04-25T19:05:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.926+0000 7fe574f69740 1 -- 172.21.15.45:0/3805280716 wait complete. 2024-04-25T19:05:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.927+0000 7fe574f69740 1 -- start start 2024-04-25T19:05:35.929 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.927+0000 7fe574f69740 1 Processor -- start 2024-04-25T19:05:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe574f69740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e61cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe574f69740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e61d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe574f69740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e61d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe574f69740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1a1780 con 0x55ff0e60c1c0 2024-04-25T19:05:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe574f69740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1a2bf0 con 0x55ff0e6173a0 2024-04-25T19:05:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe574f69740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c1a1e50 con 0x55ff0e60cae0 2024-04-25T19:05:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe574f69740 10 client.0 _ll_register_callbacks cb 0x7fe55409c440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe56aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e61d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe56b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e61cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.931 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe56aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e61d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:55288/0 (socket says 172.21.15.45:55288) 2024-04-25T19:05:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe56aa00640 1 -- 172.21.15.45:0/930516356 learned_addr learned my addr 172.21.15.45:0/930516356 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.928+0000 7fe56be00640 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e61d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.929+0000 7fe56aa00640 1 -- 172.21.15.45:0/930516356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 msgr2=0x55ff0e61d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.929+0000 7fe56aa00640 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e61d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.929+0000 7fe56aa00640 1 -- 172.21.15.45:0/930516356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 msgr2=0x55ff0e61cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.929+0000 7fe56aa00640 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e61cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.929+0000 7fe56aa00640 1 -- 172.21.15.45:0/930516356 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff0e68c9e0 con 0x55ff0e60cae0 2024-04-25T19:05:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.929+0000 7fe56aa00640 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e61d4b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe55c01aa70 tx=0x7fe55c03b650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a59b7715ece3656c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:35.932 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.929+0000 7fe558c00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:05:35.933 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.929+0000 7fe558c00640 1 -- 172.21.15.45:0/930516356 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c068070 con 0x55ff0e60cae0 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.929+0000 7fe558c00640 1 -- 172.21.15.45:0/930516356 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7fe55c0622f0 con 0x55ff0e60cae0 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.929+0000 7fe574f69740 1 -- 172.21.15.45:0/930516356 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe55c004b30 con 0x55ff0e60cae0 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe558c00640 1 -- 172.21.15.45:0/930516356 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe55c0624f0 con 0x55ff0e60cae0 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe574f69740 10 client.25843 fetch_fsmap learned FSMap version 32 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe574f69740 1 -- 172.21.15.45:0/930516356 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ff0e68dcd0 con 0x55ff0e60cae0 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe558c00640 1 -- 172.21.15.45:0/930516356 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe55c004b30 con 0x55ff0e60cae0 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe558c00640 1 -- 172.21.15.45:0/930516356 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fe55c06f470 con 0x55ff0e60cae0 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe574f69740 10 client.25843 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe574f69740 10 client.25843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe574f69740 20 client.25843 populate_metadata read hostname 'smithi045' 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe574f69740 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ff0e68e420 0x55ff0e690840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe574f69740 4 client.25843 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.930+0000 7fe574f69740 1 -- 172.21.15.45:0/930516356 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff0e577320 con 0x55ff0e68e420 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.931+0000 7fe56b400640 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ff0e68e420 0x55ff0e690840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:35.934 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.931+0000 7fe56b400640 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ff0e68e420 0x55ff0e690840 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5017136275985ab9 server_cookie=4bd95f422895a37a in_seq=0 out_seq=0 2024-04-25T19:05:35.935 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.931+0000 7fe558c00640 10 client.25843 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe558c00640 1 -- 172.21.15.45:0/930516356 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ff0e577320 con 0x55ff0e68e420 2024-04-25T19:05:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe558c00640 10 client.25843 handle_command_reply: tid=0 2024-04-25T19:05:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe574f69740 1 client.25843 shutdown 2024-04-25T19:05:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe574f69740 1 -- 172.21.15.45:0/930516356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 msgr2=0x55ff0e61d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe574f69740 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e61d4b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe55c01aa70 tx=0x7fe55c03b650 comp rx=0 tx=0).stop 2024-04-25T19:05:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe574f69740 1 -- 172.21.15.45:0/930516356 shutdown_connections 2024-04-25T19:05:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe574f69740 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55ff0e68e420 0x55ff0e690840 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe574f69740 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55ff0e6173a0 0x55ff0e61d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe574f69740 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55ff0e60cae0 0x55ff0e61d4b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe574f69740 1 --2- 172.21.15.45:0/930516356 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ff0e60c1c0 0x55ff0e61cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe558c00640 0 client.25843 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe574f69740 1 -- 172.21.15.45:0/930516356 >> 172.21.15.45:0/930516356 conn(0x55ff0e6086d0 msgr2=0x55ff0e614bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.946+0000 7fe574f69740 1 -- 172.21.15.45:0/930516356 shutdown_connections 2024-04-25T19:05:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.947+0000 7fe574f69740 1 -- 172.21.15.45:0/930516356 wait complete. 2024-04-25T19:05:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.947+0000 7fe574f69740 20 client.25843 trim_cache size 0 max 16384 2024-04-25T19:05:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:05:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:05:35.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:05:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:05:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.947+0000 7fe570a00640 1 -- 172.21.15.45:0/3592092875 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe54c0a3d80 msgr2=0x7fe54c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.947+0000 7fe570a00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe54c0a3d80 0x7fe54c0a6240 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe55c006010 tx=0x7fe55c0051d0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.947+0000 7fe570a00640 1 -- 172.21.15.45:0/3592092875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1041f0 msgr2=0x7fe56c1109f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.947+0000 7fe570a00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1041f0 0x7fe56c1109f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe55400e9f0 tx=0x7fe55400eeb0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.948+0000 7fe570a00640 1 -- 172.21.15.45:0/3592092875 shutdown_connections 2024-04-25T19:05:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.948+0000 7fe570a00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe54c0a3d80 0x7fe54c0a6240 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.951 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.948+0000 7fe570a00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe56c10d010 0x7fe56c10d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.948+0000 7fe570a00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe56c1092c0 0x7fe56c10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.948+0000 7fe570a00640 1 --2- 172.21.15.45:0/3592092875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe56c1041f0 0x7fe56c1109f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.948+0000 7fe570a00640 1 -- 172.21.15.45:0/3592092875 >> 172.21.15.45:0/3592092875 conn(0x7fe56c0ffa50 msgr2=0x7fe56c100dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.948+0000 7fe570a00640 1 -- 172.21.15.45:0/3592092875 shutdown_connections 2024-04-25T19:05:35.952 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:35.948+0000 7fe570a00640 1 -- 172.21.15.45:0/3592092875 wait complete. 2024-04-25T19:05:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:36 smithi062 ceph-mon[32844]: pgmap v192: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 354 op/s 2024-04-25T19:05:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:36 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:05:36.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:36 smithi079 ceph-mon[33002]: pgmap v192: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 354 op/s 2024-04-25T19:05:36.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:36 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:05:36.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:36 smithi045 ceph-mon[27776]: pgmap v192: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 1.8 MiB/s wr, 354 op/s 2024-04-25T19:05:36.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:36 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:05:37.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:37 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:05:37.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:37 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:05:37.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:37 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:05:37.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:37 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:05:37.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:37 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:05:37.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:37 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:05:37.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:37 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:05:37.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:37 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:05:37.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:37 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:05:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:38 smithi079 ceph-mon[33002]: pgmap v193: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 905 KiB/s wr, 219 op/s 2024-04-25T19:05:38.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:38 smithi045 ceph-mon[27776]: pgmap v193: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 905 KiB/s wr, 219 op/s 2024-04-25T19:05:38.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:38 smithi062 ceph-mon[32844]: pgmap v193: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 905 KiB/s wr, 219 op/s 2024-04-25T19:05:39.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:39 smithi079 ceph-mon[33002]: pgmap v194: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 905 KiB/s wr, 219 op/s 2024-04-25T19:05:39.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:39 smithi045 ceph-mon[27776]: pgmap v194: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 905 KiB/s wr, 219 op/s 2024-04-25T19:05:39.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:39 smithi062 ceph-mon[32844]: pgmap v194: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 905 KiB/s wr, 219 op/s 2024-04-25T19:05:42.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:41 smithi062 ceph-mon[32844]: pgmap v195: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.4 MiB/s wr, 388 op/s 2024-04-25T19:05:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:41 smithi079 ceph-mon[33002]: pgmap v195: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.4 MiB/s wr, 388 op/s 2024-04-25T19:05:42.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:41 smithi045 ceph-mon[27776]: pgmap v195: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.4 MiB/s wr, 388 op/s 2024-04-25T19:05:44.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:44 smithi062 ceph-mon[32844]: pgmap v196: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 878 KiB/s wr, 303 op/s 2024-04-25T19:05:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:44 smithi079 ceph-mon[33002]: pgmap v196: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 878 KiB/s wr, 303 op/s 2024-04-25T19:05:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:44 smithi045 ceph-mon[27776]: pgmap v196: 97 pgs: 97 active+clean; 57 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 878 KiB/s wr, 303 op/s 2024-04-25T19:05:46.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:46 smithi062 ceph-mon[32844]: pgmap v197: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.2 MiB/s wr, 449 op/s 2024-04-25T19:05:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:46 smithi079 ceph-mon[33002]: pgmap v197: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.2 MiB/s wr, 449 op/s 2024-04-25T19:05:46.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:46 smithi045 ceph-mon[27776]: pgmap v197: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.2 MiB/s wr, 449 op/s 2024-04-25T19:05:48.266 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:48 smithi045 ceph-mon[27776]: pgmap v198: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 877 KiB/s wr, 316 op/s 2024-04-25T19:05:48.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:48 smithi062 ceph-mon[32844]: pgmap v198: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 877 KiB/s wr, 316 op/s 2024-04-25T19:05:48.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:48 smithi079 ceph-mon[33002]: pgmap v198: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 877 KiB/s wr, 316 op/s 2024-04-25T19:05:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.109+0000 7f699b200640 1 -- 172.21.15.45:0/2415490207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c101fd0 msgr2=0x7f699c1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.109+0000 7f699b200640 1 --2- 172.21.15.45:0/2415490207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c101fd0 0x7f699c1023b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f698c009ca0 tx=0x7f698c02a8c0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.109+0000 7f699b200640 1 -- 172.21.15.45:0/2415490207 shutdown_connections 2024-04-25T19:05:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.110+0000 7f699b200640 1 --2- 172.21.15.45:0/2415490207 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c1070a0 0x7f699c10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.110+0000 7f699b200640 1 --2- 172.21.15.45:0/2415490207 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c102980 0x7f699c1069c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.110+0000 7f699b200640 1 --2- 172.21.15.45:0/2415490207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c101fd0 0x7f699c1023b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.111 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.110+0000 7f699b200640 1 -- 172.21.15.45:0/2415490207 >> 172.21.15.45:0/2415490207 conn(0x7f699c0fd890 msgr2=0x7f699c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.110+0000 7f699b200640 1 -- 172.21.15.45:0/2415490207 shutdown_connections 2024-04-25T19:05:49.112 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.110+0000 7f699b200640 1 -- 172.21.15.45:0/2415490207 wait complete. 2024-04-25T19:05:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.111+0000 7f699b200640 1 Processor -- start 2024-04-25T19:05:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.111+0000 7f699b200640 1 -- start start 2024-04-25T19:05:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.111+0000 7f699b200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c101fd0 0x7f699c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.111+0000 7f699b200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1070a0 0x7f699c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.111+0000 7f699b200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c074cd0 0x7f699c1a6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.111+0000 7f699b200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f699c112030 con 0x7f699c074cd0 2024-04-25T19:05:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.111+0000 7f699b200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f699c111eb0 con 0x7f699c1070a0 2024-04-25T19:05:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.111+0000 7f699b200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1121b0 con 0x7f699c101fd0 2024-04-25T19:05:49.114 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f6999e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c101fd0 0x7f699c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f6999e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c101fd0 0x7f699c074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:58050/0 (socket says 172.21.15.45:58050) 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f6999e00640 1 -- 172.21.15.45:0/3404943939 learned_addr learned my addr 172.21.15.45:0/3404943939 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f699a800640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c074cd0 0x7f699c1a6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f6991400640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1070a0 0x7f699c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f6999e00640 1 -- 172.21.15.45:0/3404943939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1070a0 msgr2=0x7f699c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f6999e00640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1070a0 0x7f699c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f6999e00640 1 -- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c074cd0 msgr2=0x7f699c1a6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f6999e00640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c074cd0 0x7f699c1a6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f6999e00640 1 -- 172.21.15.45:0/3404943939 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c1a6e20 con 0x7f699c101fd0 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f699a800640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c074cd0 0x7f699c1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.112+0000 7f6999e00640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c101fd0 0x7f699c074250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f698c005e30 tx=0x7f698c031840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:49.115 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.113+0000 7f6993400640 1 -- 172.21.15.45:0/3404943939 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c003b70 con 0x7f699c101fd0 2024-04-25T19:05:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.113+0000 7f6993400640 1 -- 172.21.15.45:0/3404943939 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c003d10 con 0x7f699c101fd0 2024-04-25T19:05:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.113+0000 7f699b200640 1 -- 172.21.15.45:0/3404943939 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c1a70b0 con 0x7f699c101fd0 2024-04-25T19:05:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.113+0000 7f6993400640 1 -- 172.21.15.45:0/3404943939 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c02fa30 con 0x7f699c101fd0 2024-04-25T19:05:49.116 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.113+0000 7f699b200640 1 -- 172.21.15.45:0/3404943939 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c1a7650 con 0x7f699c101fd0 2024-04-25T19:05:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.116+0000 7f6993400640 1 -- 172.21.15.45:0/3404943939 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f698c02fbd0 con 0x7f699c101fd0 2024-04-25T19:05:49.117 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.117+0000 7f69a3c2d740 1 Processor -- start 2024-04-25T19:05:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.117+0000 7f69a3c2d740 1 -- start start 2024-04-25T19:05:49.118 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.117+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85579b0 0x5607b8557d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.117+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 0x5607b8562650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.119 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.117+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b8564f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.117+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5607b84bd900 con 0x5607b85579b0 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.117+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5607b8463ef0 con 0x5607b8562b90 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.117+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5607b84b98c0 con 0x5607b85582d0 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f6991400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b8564f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f6991400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b8564f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36816/0 (socket says 172.21.15.45:36816) 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f6991400640 1 -- 172.21.15.45:0/250574875 learned_addr learned my addr 172.21.15.45:0/250574875 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f699a800640 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85579b0 0x5607b8557d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f6999e00640 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 0x5607b8562650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f6991400640 1 -- 172.21.15.45:0/250574875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 msgr2=0x5607b8562650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f6991400640 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 0x5607b8562650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f6991400640 1 -- 172.21.15.45:0/250574875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85579b0 msgr2=0x5607b8557d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f6991400640 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85579b0 0x5607b8557d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f6991400640 1 -- 172.21.15.45:0/250574875 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b81d0210 con 0x5607b8562b90 2024-04-25T19:05:49.120 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f699a800640 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85579b0 0x5607b8557d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.118+0000 7f6999e00640 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 0x5607b8562650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:05:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.119+0000 7f6991400640 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b8564f80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6984009c60 tx=0x7f6984011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b73b865becf8b3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:49.121 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.119+0000 7f6993400640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f69800a3db0 0x7f69800a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.119+0000 7f698b400640 1 -- 172.21.15.45:0/250574875 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984004230 con 0x5607b8562b90 2024-04-25T19:05:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.119+0000 7f698b400640 1 -- 172.21.15.45:0/250574875 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6984009e10 con 0x5607b8562b90 2024-04-25T19:05:49.124 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.119+0000 7f698b400640 1 -- 172.21.15.45:0/250574875 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984019800 con 0x5607b8562b90 2024-04-25T19:05:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.119+0000 7f6993400640 1 -- 172.21.15.45:0/3404943939 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f698c0f1a50 con 0x7f699c101fd0 2024-04-25T19:05:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.119+0000 7f699a800640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f69800a3db0 0x7f69800a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.120+0000 7f69a3c2d740 1 -- 172.21.15.45:0/250574875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 msgr2=0x5607b8564f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.120+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b8564f80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6984009c60 tx=0x7f6984011810 comp rx=0 tx=0).stop 2024-04-25T19:05:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.120+0000 7f699a800640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f69800a3db0 0x7f69800a6270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6994009cc0 tx=0x7f6994005f90 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.120+0000 7f69a3c2d740 1 -- 172.21.15.45:0/250574875 shutdown_connections 2024-04-25T19:05:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.120+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b8564f80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.120+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 0x5607b8562650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.120+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/250574875 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85579b0 0x5607b8557d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.125 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.120+0000 7f69a3c2d740 1 -- 172.21.15.45:0/250574875 >> 172.21.15.45:0/250574875 conn(0x5607b8553ec0 msgr2=0x5607b85542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.120+0000 7f69a3c2d740 1 -- 172.21.15.45:0/250574875 shutdown_connections 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.120+0000 7f69a3c2d740 1 -- 172.21.15.45:0/250574875 wait complete. 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.121+0000 7f69a3c2d740 1 -- start start 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.121+0000 7f69a3c2d740 1 Processor -- start 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.121+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b855faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.121+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 0x5607b8560030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.121+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b8562b90 0x5607b8560570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.121+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5607b85657c0 con 0x5607b8562b90 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.121+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5607b8477310 con 0x5607b85579b0 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.121+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5607b8565940 con 0x5607b85582d0 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.121+0000 7f69a3c2d740 10 client.0 _ll_register_callbacks cb 0x5607b8445cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.121+0000 7f6999e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b855faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f6991400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 0x5607b8560030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f6999e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b855faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36828/0 (socket says 172.21.15.45:36828) 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f6999e00640 1 -- 172.21.15.45:0/1341582956 learned_addr learned my addr 172.21.15.45:0/1341582956 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f699a800640 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b8562b90 0x5607b8560570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f6999e00640 1 -- 172.21.15.45:0/1341582956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 msgr2=0x5607b8560030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f6999e00640 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 0x5607b8560030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f6999e00640 1 -- 172.21.15.45:0/1341582956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b8562b90 msgr2=0x5607b8560570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f6999e00640 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b8562b90 0x5607b8560570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f6999e00640 1 -- 172.21.15.45:0/1341582956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b829c620 con 0x5607b85579b0 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f699a800640 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b8562b90 0x5607b8560570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f6999e00640 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b855faf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f698c0ef2b0 tx=0x7f698c0bd6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abaa7490d8eea231 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f697fe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f697fe00640 1 -- 172.21.15.45:0/1341582956 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0c1e50 con 0x5607b85579b0 2024-04-25T19:05:49.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f697fe00640 1 -- 172.21.15.45:0/1341582956 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f698c0ef3c0 con 0x5607b85579b0 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.122+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1341582956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6984009e10 con 0x5607b85579b0 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f697fe00640 1 -- 172.21.15.45:0/1341582956 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f698c0ef5c0 con 0x5607b85579b0 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f69a3c2d740 10 client.25838 fetch_fsmap learned FSMap version 32 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1341582956 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5607b85657c0 con 0x5607b85579b0 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f697fe00640 1 -- 172.21.15.45:0/1341582956 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6984009e10 con 0x5607b85579b0 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f697fe00640 1 -- 172.21.15.45:0/1341582956 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f698c0c6280 con 0x5607b85579b0 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f69a3c2d740 10 client.25838 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f69a3c2d740 10 client.25838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f69a3c2d740 20 client.25838 populate_metadata read hostname 'smithi045' 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607b8572e60 0x5607b85752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f69a3c2d740 4 client.25838 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.123+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1341582956 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607b84d18b0 con 0x5607b8572e60 2024-04-25T19:05:49.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.124+0000 7f6991400640 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607b8572e60 0x5607b85752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.124+0000 7f6991400640 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607b8572e60 0x5607b85752a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=150c9055037d8ed0 server_cookie=6a2f4f46e73ff4b0 in_seq=0 out_seq=0 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.124+0000 7f697fe00640 10 client.25838 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f697fe00640 1 -- 172.21.15.45:0/1341582956 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5607b84d18b0 con 0x5607b8572e60 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f697fe00640 10 client.25838 handle_command_reply: tid=0 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 client.25838 shutdown 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1341582956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 msgr2=0x5607b855faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b855faf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f698c0ef2b0 tx=0x7f698c0bd6a0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1341582956 shutdown_connections 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607b8572e60 0x5607b85752a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b8562b90 0x5607b8560570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85582d0 0x5607b8560030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1341582956 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b855faf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1341582956 >> 172.21.15.45:0/1341582956 conn(0x5607b8553ec0 msgr2=0x5607b8559c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f697fe00640 0 client.25838 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1341582956 shutdown_connections 2024-04-25T19:05:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1341582956 wait complete. 2024-04-25T19:05:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.125+0000 7f69a3c2d740 20 client.25838 trim_cache size 0 max 16384 2024-04-25T19:05:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.143+0000 7f69a3c2d740 1 Processor -- start 2024-04-25T19:05:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.143+0000 7f69a3c2d740 1 -- start start 2024-04-25T19:05:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.143+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85579b0 0x5607b856e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.143+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85582d0 0x5607b855a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.143+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b857abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.143+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b23f0 con 0x5607b85582d0 2024-04-25T19:05:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.143+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1afcc0 con 0x5607b8562b90 2024-04-25T19:05:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.143+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1adb10 con 0x5607b85579b0 2024-04-25T19:05:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.144+0000 7f699a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b857abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.144+0000 7f6999e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85579b0 0x5607b856e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.144+0000 7f699a800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b857abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36844/0 (socket says 172.21.15.45:36844) 2024-04-25T19:05:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.144+0000 7f699a800640 1 -- 172.21.15.45:0/924508526 learned_addr learned my addr 172.21.15.45:0/924508526 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.144+0000 7f6991400640 1 --2- 172.21.15.45:0/924508526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85582d0 0x5607b855a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.144+0000 7f6999e00640 1 -- 172.21.15.45:0/924508526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 msgr2=0x5607b857abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.144+0000 7f6999e00640 1 --2- 172.21.15.45:0/924508526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b857abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.144+0000 7f6999e00640 1 -- 172.21.15.45:0/924508526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85582d0 msgr2=0x5607b855a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.144+0000 7f6999e00640 1 --2- 172.21.15.45:0/924508526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85582d0 0x5607b855a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.144+0000 7f6999e00640 1 -- 172.21.15.45:0/924508526 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b85652f0 con 0x5607b85579b0 2024-04-25T19:05:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.145+0000 7f6999e00640 1 --2- 172.21.15.45:0/924508526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85579b0 0x5607b856e680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f698c006010 tx=0x7f698c0bd710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3186714f9cd92594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.145+0000 7f698aa00640 1 -- 172.21.15.45:0/924508526 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0c6710 con 0x5607b85579b0 2024-04-25T19:05:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.145+0000 7f698aa00640 1 -- 172.21.15.45:0/924508526 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c036030 con 0x5607b85579b0 2024-04-25T19:05:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.145+0000 7f698aa00640 1 -- 172.21.15.45:0/924508526 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0bcfe0 con 0x5607b85579b0 2024-04-25T19:05:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.146+0000 7f69a3c2d740 1 -- 172.21.15.45:0/924508526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85579b0 msgr2=0x5607b856e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.146+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/924508526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85579b0 0x5607b856e680 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f698c006010 tx=0x7f698c0bd710 comp rx=0 tx=0).stop 2024-04-25T19:05:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.146+0000 7f69a3c2d740 1 -- 172.21.15.45:0/924508526 shutdown_connections 2024-04-25T19:05:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.146+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/924508526 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b8562b90 0x5607b857abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.146+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/924508526 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85582d0 0x5607b855a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.146+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/924508526 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b85579b0 0x5607b856e680 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.146+0000 7f69a3c2d740 1 -- 172.21.15.45:0/924508526 >> 172.21.15.45:0/924508526 conn(0x5607b8553ec0 msgr2=0x5607b8553020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.146+0000 7f69a3c2d740 1 -- 172.21.15.45:0/924508526 shutdown_connections 2024-04-25T19:05:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.146+0000 7f69a3c2d740 1 -- 172.21.15.45:0/924508526 wait complete. 2024-04-25T19:05:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.147+0000 7f69a3c2d740 1 -- start start 2024-04-25T19:05:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.147+0000 7f69a3c2d740 1 Processor -- start 2024-04-25T19:05:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.147+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b856afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.147+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85582d0 0x5607b8568370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.147+0000 7f69a3c2d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b8562b90 0x5607b85688f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.147+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ae180 con 0x5607b85582d0 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.147+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1af5f0 con 0x5607b85579b0 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.147+0000 7f69a3c2d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ae850 con 0x5607b8562b90 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f69a3c2d740 10 client.0 _ll_register_callbacks cb 0x7f698c0b9e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f6999e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b856afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f6999e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b856afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36860/0 (socket says 172.21.15.45:36860) 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f6999e00640 1 -- 172.21.15.45:0/1778652017 learned_addr learned my addr 172.21.15.45:0/1778652017 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f6991400640 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85582d0 0x5607b8568370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f699a800640 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b8562b90 0x5607b85688f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f6999e00640 1 -- 172.21.15.45:0/1778652017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b8562b90 msgr2=0x5607b85688f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f6999e00640 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b8562b90 0x5607b85688f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f6999e00640 1 -- 172.21.15.45:0/1778652017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85582d0 msgr2=0x5607b8568370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f6999e00640 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85582d0 0x5607b8568370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.148+0000 7f6999e00640 1 -- 172.21.15.45:0/1778652017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b8569120 con 0x5607b85579b0 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.149+0000 7f6999e00640 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b856afe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f698c0c61e0 tx=0x7f698c0bd870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee42239a3759df0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.149+0000 7f697fe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.149+0000 7f697fe00640 1 -- 172.21.15.45:0/1778652017 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c0f5070 con 0x5607b85579b0 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.149+0000 7f697fe00640 1 -- 172.21.15.45:0/1778652017 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f698c0bdd00 con 0x5607b85579b0 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.149+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1778652017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f698c036030 con 0x5607b85579b0 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.149+0000 7f697fe00640 1 -- 172.21.15.45:0/1778652017 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f698c0ef5d0 con 0x5607b85579b0 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.149+0000 7f69a3c2d740 10 client.25844 fetch_fsmap learned FSMap version 32 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.149+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1778652017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5607b8569120 con 0x5607b85579b0 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.149+0000 7f697fe00640 1 -- 172.21.15.45:0/1778652017 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c036030 con 0x5607b85579b0 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.150+0000 7f697fe00640 1 -- 172.21.15.45:0/1778652017 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f698c0bc960 con 0x5607b85579b0 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.150+0000 7f69a3c2d740 10 client.25844 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.150+0000 7f69a3c2d740 10 client.25844 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.150+0000 7f69a3c2d740 20 client.25844 populate_metadata read hostname 'smithi045' 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.150+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607b85e1f90 0x5607b85e4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.150+0000 7f69a3c2d740 4 client.25844 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:05:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.150+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1778652017 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607b84cb320 con 0x5607b85e1f90 2024-04-25T19:05:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.150+0000 7f6991400640 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607b85e1f90 0x5607b85e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:05:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.151+0000 7f6991400640 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607b85e1f90 0x5607b85e4380 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=655671ec874aa40e server_cookie=9925f1c72fb362ce in_seq=0 out_seq=0 2024-04-25T19:05:49.154 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.151+0000 7f697fe00640 10 client.25844 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.168+0000 7f697fe00640 1 -- 172.21.15.45:0/1778652017 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607b84cb320 con 0x5607b85e1f90 2024-04-25T19:05:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.168+0000 7f697fe00640 10 client.25844 handle_command_reply: tid=0 2024-04-25T19:05:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.168+0000 7f69a3c2d740 1 client.25844 shutdown 2024-04-25T19:05:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.169+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1778652017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 msgr2=0x5607b856afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.169+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b856afe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f698c0c61e0 tx=0x7f698c0bd870 comp rx=0 tx=0).stop 2024-04-25T19:05:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.171+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1778652017 shutdown_connections 2024-04-25T19:05:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.171+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5607b85e1f90 0x5607b85e4380 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.171+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5607b8562b90 0x5607b85688f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.171+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5607b85582d0 0x5607b8568370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.171+0000 7f697fe00640 0 client.25844 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:05:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.171+0000 7f69a3c2d740 1 --2- 172.21.15.45:0/1778652017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5607b85579b0 0x5607b856afe0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.171+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1778652017 >> 172.21.15.45:0/1778652017 conn(0x5607b8553ec0 msgr2=0x5607b8553020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.172+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1778652017 shutdown_connections 2024-04-25T19:05:49.173 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.172+0000 7f69a3c2d740 1 -- 172.21.15.45:0/1778652017 wait complete. 2024-04-25T19:05:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.172+0000 7f69a3c2d740 20 client.25844 trim_cache size 0 max 16384 2024-04-25T19:05:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:05:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:05:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:05:49.174 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:05:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 -- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f69800a3db0 msgr2=0x7f69800a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f69800a3db0 0x7f69800a6270 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6994009cc0 tx=0x7f6994005f90 comp rx=0 tx=0).stop 2024-04-25T19:05:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 -- 172.21.15.45:0/3404943939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c101fd0 msgr2=0x7f699c074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:05:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c101fd0 0x7f699c074250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f698c005e30 tx=0x7f698c031840 comp rx=0 tx=0).stop 2024-04-25T19:05:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 -- 172.21.15.45:0/3404943939 shutdown_connections 2024-04-25T19:05:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f69800a3db0 0x7f69800a6270 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f699c074cd0 0x7f699c1a6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.175 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f699c1070a0 0x7f699c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 --2- 172.21.15.45:0/3404943939 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f699c101fd0 0x7f699c074250 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:05:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 -- 172.21.15.45:0/3404943939 >> 172.21.15.45:0/3404943939 conn(0x7f699c0fd890 msgr2=0x7f699c0fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:05:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 -- 172.21.15.45:0/3404943939 shutdown_connections 2024-04-25T19:05:49.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:05:49.173+0000 7f699b200640 1 -- 172.21.15.45:0/3404943939 wait complete. 2024-04-25T19:05:50.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:50 smithi062 ceph-mon[32844]: pgmap v199: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 877 KiB/s wr, 316 op/s 2024-04-25T19:05:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:50 smithi079 ceph-mon[33002]: pgmap v199: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 877 KiB/s wr, 316 op/s 2024-04-25T19:05:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:50 smithi045 ceph-mon[27776]: pgmap v199: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 877 KiB/s wr, 316 op/s 2024-04-25T19:05:52.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:52 smithi062 ceph-mon[32844]: pgmap v200: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.3 MiB/s wr, 475 op/s 2024-04-25T19:05:52.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:52 smithi079 ceph-mon[33002]: pgmap v200: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.3 MiB/s wr, 475 op/s 2024-04-25T19:05:52.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:52 smithi045 ceph-mon[27776]: pgmap v200: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.3 MiB/s wr, 475 op/s 2024-04-25T19:05:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:54 smithi062 ceph-mon[32844]: pgmap v201: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 822 KiB/s wr, 306 op/s 2024-04-25T19:05:54.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:54 smithi079 ceph-mon[33002]: pgmap v201: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 822 KiB/s wr, 306 op/s 2024-04-25T19:05:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:54 smithi045 ceph-mon[27776]: pgmap v201: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 822 KiB/s wr, 306 op/s 2024-04-25T19:05:56.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:56 smithi062 ceph-mon[32844]: pgmap v202: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 444 op/s 2024-04-25T19:05:56.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:56 smithi079 ceph-mon[33002]: pgmap v202: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 444 op/s 2024-04-25T19:05:56.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:56 smithi045 ceph-mon[27776]: pgmap v202: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 444 op/s 2024-04-25T19:05:58.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:05:58 smithi062 ceph-mon[32844]: pgmap v203: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 891 KiB/s wr, 297 op/s 2024-04-25T19:05:58.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:05:58 smithi079 ceph-mon[33002]: pgmap v203: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 891 KiB/s wr, 297 op/s 2024-04-25T19:05:58.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:05:58 smithi045 ceph-mon[27776]: pgmap v203: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 891 KiB/s wr, 297 op/s 2024-04-25T19:06:00.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:00 smithi062 ceph-mon[32844]: pgmap v204: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 891 KiB/s wr, 297 op/s 2024-04-25T19:06:00.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:00 smithi079 ceph-mon[33002]: pgmap v204: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 891 KiB/s wr, 297 op/s 2024-04-25T19:06:00.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:00 smithi045 ceph-mon[27776]: pgmap v204: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 891 KiB/s wr, 297 op/s 2024-04-25T19:06:01.219 INFO:tasks.workunit.client.0.smithi045.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-04-25T19:06:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.329+0000 7f1f91600640 1 -- 172.21.15.45:0/1786808194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f8c1092f0 msgr2=0x7f1f8c110ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.329+0000 7f1f91600640 1 --2- 172.21.15.45:0/1786808194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f8c1092f0 0x7f1f8c110ec0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009ca0 tx=0x7f1f8402a8b0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.330+0000 7f1f91600640 1 -- 172.21.15.45:0/1786808194 shutdown_connections 2024-04-25T19:06:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.330+0000 7f1f91600640 1 --2- 172.21.15.45:0/1786808194 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f8c1092f0 0x7f1f8c110ec0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.331 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.330+0000 7f1f91600640 1 --2- 172.21.15.45:0/1786808194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c104bd0 0x7f1f8c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.330+0000 7f1f91600640 1 --2- 172.21.15.45:0/1786808194 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f8c104220 0x7f1f8c104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.330+0000 7f1f91600640 1 -- 172.21.15.45:0/1786808194 >> 172.21.15.45:0/1786808194 conn(0x7f1f8c0ffa60 msgr2=0x7f1f8c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.330+0000 7f1f91600640 1 -- 172.21.15.45:0/1786808194 shutdown_connections 2024-04-25T19:06:02.332 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.330+0000 7f1f91600640 1 -- 172.21.15.45:0/1786808194 wait complete. 2024-04-25T19:06:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.331+0000 7f1f91600640 1 Processor -- start 2024-04-25T19:06:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.331+0000 7f1f91600640 1 -- start start 2024-04-25T19:06:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.331+0000 7f1f91600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c104220 0x7f1f8c1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.331+0000 7f1f91600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f8c1092f0 0x7f1f8c1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.331+0000 7f1f91600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f8c19a630 0x7f1f8c19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.331+0000 7f1f91600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c114290 con 0x7f1f8c1092f0 2024-04-25T19:06:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.331+0000 7f1f91600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c114110 con 0x7f1f8c19a630 2024-04-25T19:06:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.331+0000 7f1f91600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c114410 con 0x7f1f8c104220 2024-04-25T19:06:02.334 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c104220 0x7f1f8c1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c104220 0x7f1f8c1a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60932/0 (socket says 172.21.15.45:60932) 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f8be00640 1 -- 172.21.15.45:0/2207247326 learned_addr learned my addr 172.21.15.45:0/2207247326 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f90c00640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f8c19a630 0x7f1f8c19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f83e00640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f8c1092f0 0x7f1f8c1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f8be00640 1 -- 172.21.15.45:0/2207247326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f8c19a630 msgr2=0x7f1f8c19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f8be00640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f8c19a630 0x7f1f8c19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f8be00640 1 -- 172.21.15.45:0/2207247326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f8c1092f0 msgr2=0x7f1f8c1a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f8be00640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f8c1092f0 0x7f1f8c1a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f8be00640 1 -- 172.21.15.45:0/2207247326 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c19b150 con 0x7f1f8c104220 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f90c00640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f8c19a630 0x7f1f8c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.332+0000 7f1f8be00640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c104220 0x7f1f8c1a0430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00e9f0 tx=0x7f1f7c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.333+0000 7f1f89600640 1 -- 172.21.15.45:0/2207247326 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c00cd80 con 0x7f1f8c104220 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.333+0000 7f1f91600640 1 -- 172.21.15.45:0/2207247326 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c19b440 con 0x7f1f8c104220 2024-04-25T19:06:02.335 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.333+0000 7f1f89600640 1 -- 172.21.15.45:0/2207247326 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7c0108e0 con 0x7f1f8c104220 2024-04-25T19:06:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.333+0000 7f1f89600640 1 -- 172.21.15.45:0/2207247326 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c018980 con 0x7f1f8c104220 2024-04-25T19:06:02.336 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.333+0000 7f1f91600640 1 -- 172.21.15.45:0/2207247326 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f8c1a7690 con 0x7f1f8c104220 2024-04-25T19:06:02.339 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.337+0000 7f1f89600640 1 -- 172.21.15.45:0/2207247326 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f1f7c010430 con 0x7f1f8c104220 2024-04-25T19:06:02.340 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.339+0000 7f1f89600640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1f700a3d00 0x7f1f700a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.339+0000 7f1f95ba8740 1 Processor -- start 2024-04-25T19:06:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.339+0000 7f1f89600640 1 -- 172.21.15.45:0/2207247326 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f1f7c0d0ae0 con 0x7f1f8c104220 2024-04-25T19:06:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.339+0000 7f1f95ba8740 1 -- start start 2024-04-25T19:06:02.341 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f83e00640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1f700a3d00 0x7f1f700a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 0x55df5fb55d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb60650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb60b90 0x55df5fb62f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55df5fabb900 con 0x55df5fb559b0 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55df5fa61ef0 con 0x55df5fb562d0 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df5fab78c0 con 0x55df5fb60b90 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f90c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb60b90 0x55df5fb62f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 0x55df5fb55d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f90c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb60b90 0x55df5fb62f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60942/0 (socket says 172.21.15.45:60942) 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.340+0000 7f1f90c00640 1 -- 172.21.15.45:0/4005488795 learned_addr learned my addr 172.21.15.45:0/4005488795 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.341+0000 7f1f83e00640 1 --2- 172.21.15.45:0/4005488795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb60650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.341+0000 7f1f90c00640 1 -- 172.21.15.45:0/4005488795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 msgr2=0x55df5fb60650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.341+0000 7f1f90c00640 1 --2- 172.21.15.45:0/4005488795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb60650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.341+0000 7f1f90c00640 1 -- 172.21.15.45:0/4005488795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 msgr2=0x55df5fb55d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:02.343 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.341+0000 7f1f90c00640 1 --2- 172.21.15.45:0/4005488795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 0x55df5fb55d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.341+0000 7f1f90c00640 1 -- 172.21.15.45:0/4005488795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5f7ce210 con 0x55df5fb60b90 2024-04-25T19:06:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.341+0000 7f1f8be00640 1 --2- 172.21.15.45:0/4005488795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 0x55df5fb55d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.341+0000 7f1f83e00640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1f700a3d00 0x7f1f700a61c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c10a8d0 tx=0x7f1f74009f70 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.341+0000 7f1f90c00640 1 --2- 172.21.15.45:0/4005488795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb60b90 0x55df5fb62f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009ca0 tx=0x7f1f84005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b19b7875e5bbde5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.342+0000 7f1f81200640 1 -- 172.21.15.45:0/4005488795 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84030ac0 con 0x55df5fb60b90 2024-04-25T19:06:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.342+0000 7f1f81200640 1 -- 172.21.15.45:0/4005488795 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f84036030 con 0x55df5fb60b90 2024-04-25T19:06:02.344 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.342+0000 7f1f81200640 1 -- 172.21.15.45:0/4005488795 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84004670 con 0x55df5fb60b90 2024-04-25T19:06:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.343+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4005488795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb60b90 msgr2=0x55df5fb62f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.343+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/4005488795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb60b90 0x55df5fb62f80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1f84009ca0 tx=0x7f1f84005ef0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.343+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4005488795 shutdown_connections 2024-04-25T19:06:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.343+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/4005488795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb60b90 0x55df5fb62f80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.343+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/4005488795 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb60650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.343+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/4005488795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 0x55df5fb55d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.343+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4005488795 >> 172.21.15.45:0/4005488795 conn(0x55df5fb51ec0 msgr2=0x55df5fb522a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.343+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4005488795 shutdown_connections 2024-04-25T19:06:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.343+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4005488795 wait complete. 2024-04-25T19:06:02.345 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.344+0000 7f1f95ba8740 1 -- start start 2024-04-25T19:06:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.344+0000 7f1f95ba8740 1 Processor -- start 2024-04-25T19:06:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.345+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 0x55df5fb5da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.345+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb562d0 0x55df5fb5dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.345+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb60b90 0x55df5fb5e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.345+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55df5fb63760 con 0x55df5fb559b0 2024-04-25T19:06:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.345+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55df5fa75310 con 0x55df5fb60b90 2024-04-25T19:06:02.347 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.345+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55df5fb638e0 con 0x55df5fb562d0 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.345+0000 7f1f95ba8740 10 client.0 _ll_register_callbacks cb 0x55df5fa43cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f90c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb60b90 0x55df5fb5e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f90c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb60b90 0x55df5fb5e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:59276/0 (socket says 172.21.15.45:59276) 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f90c00640 1 -- 172.21.15.45:0/344650862 learned_addr learned my addr 172.21.15.45:0/344650862 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f8be00640 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 0x55df5fb5da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f83e00640 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb562d0 0x55df5fb5dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f90c00640 1 -- 172.21.15.45:0/344650862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb562d0 msgr2=0x55df5fb5dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f90c00640 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb562d0 0x55df5fb5dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f90c00640 1 -- 172.21.15.45:0/344650862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 msgr2=0x55df5fb5da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f90c00640 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 0x55df5fb5da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.348 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f90c00640 1 -- 172.21.15.45:0/344650862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df5f89a620 con 0x55df5fb60b90 2024-04-25T19:06:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f90c00640 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb60b90 0x55df5fb5e510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f84006010 tx=0x7f1f84030d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1aa870b9e69b5d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.346+0000 7f1f79600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:06:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.347+0000 7f1f79600640 1 -- 172.21.15.45:0/344650862 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f84053020 con 0x55df5fb60b90 2024-04-25T19:06:02.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.347+0000 7f1f95ba8740 1 -- 172.21.15.45:0/344650862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f84036030 con 0x55df5fb60b90 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.347+0000 7f1f79600640 1 -- 172.21.15.45:0/344650862 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f1f84004600 con 0x55df5fb60b90 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.347+0000 7f1f79600640 1 -- 172.21.15.45:0/344650862 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f8405e050 con 0x55df5fb60b90 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.347+0000 7f1f95ba8740 10 client.25856 fetch_fsmap learned FSMap version 32 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.347+0000 7f1f95ba8740 1 -- 172.21.15.45:0/344650862 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55df5fb63760 con 0x55df5fb60b90 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.347+0000 7f1f79600640 1 -- 172.21.15.45:0/344650862 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f84036030 con 0x55df5fb60b90 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.348+0000 7f1f79600640 1 -- 172.21.15.45:0/344650862 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f1f84043070 con 0x55df5fb60b90 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.348+0000 7f1f95ba8740 10 client.25856 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.348+0000 7f1f95ba8740 10 client.25856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.348+0000 7f1f95ba8740 20 client.25856 populate_metadata read hostname 'smithi045' 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.348+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55df5fb70aa0 0x55df5fb72e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.348+0000 7f1f95ba8740 4 client.25856 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.348+0000 7f1f95ba8740 1 -- 172.21.15.45:0/344650862 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df5facf8b0 con 0x55df5fb70aa0 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.348+0000 7f1f8be00640 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55df5fb70aa0 0x55df5fb72e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.349+0000 7f1f8be00640 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55df5fb70aa0 0x55df5fb72e90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d857fe9b90dd2b4 server_cookie=fb739172d2d9b84e in_seq=0 out_seq=0 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.349+0000 7f1f79600640 10 client.25856 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:02.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.349+0000 7f1f79600640 1 -- 172.21.15.45:0/344650862 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df5facf8b0 con 0x55df5fb70aa0 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.349+0000 7f1f79600640 10 client.25856 handle_command_reply: tid=0 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.349+0000 7f1f95ba8740 1 client.25856 shutdown 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 1 -- 172.21.15.45:0/344650862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb60b90 msgr2=0x55df5fb5e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb60b90 0x55df5fb5e510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1f84006010 tx=0x7f1f84030d60 comp rx=0 tx=0).stop 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 1 -- 172.21.15.45:0/344650862 shutdown_connections 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55df5fb70aa0 0x55df5fb72e90 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb60b90 0x55df5fb5e510 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb562d0 0x55df5fb5dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/344650862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb559b0 0x55df5fb5da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f79600640 0 client.25856 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 1 -- 172.21.15.45:0/344650862 >> 172.21.15.45:0/344650862 conn(0x55df5fb51ec0 msgr2=0x55df5fb57c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 1 -- 172.21.15.45:0/344650862 shutdown_connections 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 1 -- 172.21.15.45:0/344650862 wait complete. 2024-04-25T19:06:02.353 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.350+0000 7f1f95ba8740 20 client.25856 trim_cache size 0 max 16384 2024-04-25T19:06:02.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:02 smithi062 ceph-mon[32844]: pgmap v205: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-04-25T19:06:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.370+0000 7f1f95ba8740 1 Processor -- start 2024-04-25T19:06:02.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.370+0000 7f1f95ba8740 1 -- start start 2024-04-25T19:06:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb6c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb58370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 0x55df5fb78670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1b24d0 con 0x55df5fb60b90 2024-04-25T19:06:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1afda0 con 0x55df5fb562d0 2024-04-25T19:06:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1adbf0 con 0x55df5fb559b0 2024-04-25T19:06:02.373 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb58370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb6c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f83e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb58370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:59280/0 (socket says 172.21.15.45:59280) 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb6c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60962/0 (socket says 172.21.15.45:60962) 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f83e00640 1 -- 172.21.15.45:0/2394648901 learned_addr learned my addr 172.21.15.45:0/2394648901 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.371+0000 7f1f90c00640 1 --2- 172.21.15.45:0/2394648901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 0x55df5fb78670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.372+0000 7f1f8be00640 1 -- 172.21.15.45:0/2394648901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 msgr2=0x55df5fb58370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.372+0000 7f1f8be00640 1 --2- 172.21.15.45:0/2394648901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb58370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.372+0000 7f1f8be00640 1 -- 172.21.15.45:0/2394648901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 msgr2=0x55df5fb78670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.372+0000 7f1f8be00640 1 --2- 172.21.15.45:0/2394648901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 0x55df5fb78670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.372+0000 7f1f8be00640 1 -- 172.21.15.45:0/2394648901 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df5fb5bfe0 con 0x55df5fb559b0 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.372+0000 7f1f90c00640 1 --2- 172.21.15.45:0/2394648901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 0x55df5fb78670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.372+0000 7f1f8be00640 1 --2- 172.21.15.45:0/2394648901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb6c620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0ce230 tx=0x7f1f7c01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c68b49ecab6d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:02.374 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.372+0000 7f1f7be00640 1 -- 172.21.15.45:0/2394648901 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c09ae30 con 0x55df5fb559b0 2024-04-25T19:06:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.372+0000 7f1f7be00640 1 -- 172.21.15.45:0/2394648901 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7c010a90 con 0x55df5fb559b0 2024-04-25T19:06:02.375 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.372+0000 7f1f7be00640 1 -- 172.21.15.45:0/2394648901 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c018d80 con 0x55df5fb559b0 2024-04-25T19:06:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.373+0000 7f1f95ba8740 1 -- 172.21.15.45:0/2394648901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 msgr2=0x55df5fb6c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.373+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/2394648901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb6c620 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0ce230 tx=0x7f1f7c01a040 comp rx=0 tx=0).stop 2024-04-25T19:06:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.373+0000 7f1f95ba8740 1 -- 172.21.15.45:0/2394648901 shutdown_connections 2024-04-25T19:06:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.373+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/2394648901 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 0x55df5fb78670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.373+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/2394648901 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb58370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.373+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/2394648901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb6c620 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.378 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.373+0000 7f1f95ba8740 1 -- 172.21.15.45:0/2394648901 >> 172.21.15.45:0/2394648901 conn(0x55df5fb51ec0 msgr2=0x55df5fb5e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.373+0000 7f1f95ba8740 1 -- 172.21.15.45:0/2394648901 shutdown_connections 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.373+0000 7f1f95ba8740 1 -- 172.21.15.45:0/2394648901 wait complete. 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.374+0000 7f1f95ba8740 1 -- start start 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.374+0000 7f1f95ba8740 1 Processor -- start 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.374+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb68b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb69070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f95ba8740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 0x55df5fb662b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1ae260 con 0x55df5fb60b90 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1af6d0 con 0x55df5fb562d0 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f95ba8740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1f8c1ae930 con 0x55df5fb559b0 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f95ba8740 10 client.0 _ll_register_callbacks cb 0x7f1f84009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb68b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f8be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb68b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60974/0 (socket says 172.21.15.45:60974) 2024-04-25T19:06:02.379 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f8be00640 1 -- 172.21.15.45:0/4021971834 learned_addr learned my addr 172.21.15.45:0/4021971834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f90c00640 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 0x55df5fb662b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f83e00640 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb69070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f8be00640 1 -- 172.21.15.45:0/4021971834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 msgr2=0x55df5fb69070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f8be00640 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb69070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f8be00640 1 -- 172.21.15.45:0/4021971834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 msgr2=0x55df5fb662b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f8be00640 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 0x55df5fb662b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f8be00640 1 -- 172.21.15.45:0/4021971834 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df5fb66ab0 con 0x55df5fb559b0 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f83e00640 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb69070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.375+0000 7f1f90c00640 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 0x55df5fb662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.376+0000 7f1f8be00640 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb68b30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c098f90 tx=0x7f1f7c0bd170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a19d37ab2af14b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.376+0000 7f1f79600640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.376+0000 7f1f79600640 1 -- 172.21.15.45:0/4021971834 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0a0e30 con 0x55df5fb559b0 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.376+0000 7f1f79600640 1 -- 172.21.15.45:0/4021971834 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8527+0+0 (secure 0 0 0) 0x7f1f7c09bf80 con 0x55df5fb559b0 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.376+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4021971834 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f7c010a90 con 0x55df5fb559b0 2024-04-25T19:06:02.380 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.376+0000 7f1f79600640 1 -- 172.21.15.45:0/4021971834 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f7c09c180 con 0x55df5fb559b0 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.376+0000 7f1f95ba8740 10 client.25876 fetch_fsmap learned FSMap version 32 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.376+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4021971834 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55df5fbd6ba0 con 0x55df5fb559b0 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.376+0000 7f1f79600640 1 -- 172.21.15.45:0/4021971834 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f7c010a90 con 0x55df5fb559b0 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.377+0000 7f1f79600640 1 -- 172.21.15.45:0/4021971834 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f1f7c09ba80 con 0x55df5fb559b0 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.377+0000 7f1f95ba8740 10 client.25876 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.377+0000 7f1f95ba8740 10 client.25876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.377+0000 7f1f95ba8740 20 client.25876 populate_metadata read hostname 'smithi045' 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.377+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55df5fbd7230 0x55df5fbd9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.377+0000 7f1f95ba8740 4 client.25876 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.377+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4021971834 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df5fac9320 con 0x55df5fbd7230 2024-04-25T19:06:02.381 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.377+0000 7f1f83e00640 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55df5fbd7230 0x55df5fbd9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.378+0000 7f1f79600640 10 client.25876 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:02.382 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.377+0000 7f1f83e00640 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55df5fbd7230 0x55df5fbd9650 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b5f37b6432a42c server_cookie=f5d64500354301f3 in_seq=0 out_seq=0 2024-04-25T19:06:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.390+0000 7f1f79600640 1 -- 172.21.15.45:0/4021971834 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df5fac9320 con 0x55df5fbd7230 2024-04-25T19:06:02.391 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.390+0000 7f1f79600640 10 client.25876 handle_command_reply: tid=0 2024-04-25T19:06:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.390+0000 7f1f95ba8740 1 client.25876 shutdown 2024-04-25T19:06:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.391+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4021971834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 msgr2=0x55df5fb68b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.391+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb68b30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c098f90 tx=0x7f1f7c0bd170 comp rx=0 tx=0).stop 2024-04-25T19:06:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.391+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4021971834 shutdown_connections 2024-04-25T19:06:02.392 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.391+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55df5fbd7230 0x55df5fbd9650 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.391+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55df5fb60b90 0x55df5fb662b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.391+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55df5fb562d0 0x55df5fb69070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.391+0000 7f1f95ba8740 1 --2- 172.21.15.45:0/4021971834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55df5fb559b0 0x55df5fb68b30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.391+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4021971834 >> 172.21.15.45:0/4021971834 conn(0x55df5fb51ec0 msgr2=0x55df5fb50790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.391+0000 7f1f79600640 0 client.25876 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:02.393 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.392+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4021971834 shutdown_connections 2024-04-25T19:06:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.392+0000 7f1f95ba8740 1 -- 172.21.15.45:0/4021971834 wait complete. 2024-04-25T19:06:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.392+0000 7f1f95ba8740 20 client.25876 trim_cache size 0 max 16384 2024-04-25T19:06:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:06:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:06:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:06:02.394 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:06:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 -- 172.21.15.45:0/2207247326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1f700a3d00 msgr2=0x7f1f700a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1f700a3d00 0x7f1f700a61c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c10a8d0 tx=0x7f1f74009f70 comp rx=0 tx=0).stop 2024-04-25T19:06:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 -- 172.21.15.45:0/2207247326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c104220 msgr2=0x7f1f8c1a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c104220 0x7f1f8c1a0430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00e9f0 tx=0x7f1f7c00eeb0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 -- 172.21.15.45:0/2207247326 shutdown_connections 2024-04-25T19:06:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f1f700a3d00 0x7f1f700a61c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.395 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f1f8c19a630 0x7f1f8c19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1f8c1092f0 0x7f1f8c1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 --2- 172.21.15.45:0/2207247326 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f8c104220 0x7f1f8c1a0430 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 -- 172.21.15.45:0/2207247326 >> 172.21.15.45:0/2207247326 conn(0x7f1f8c0ffa60 msgr2=0x7f1f8c101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 -- 172.21.15.45:0/2207247326 shutdown_connections 2024-04-25T19:06:02.396 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:02.393+0000 7f1f91600640 1 -- 172.21.15.45:0/2207247326 wait complete. 2024-04-25T19:06:02.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:02 smithi079 ceph-mon[33002]: pgmap v205: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-04-25T19:06:02.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:02 smithi045 ceph-mon[27776]: pgmap v205: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.3 MiB/s wr, 445 op/s 2024-04-25T19:06:04.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:04 smithi062 ceph-mon[32844]: pgmap v206: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 936 KiB/s wr, 285 op/s 2024-04-25T19:06:04.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:04 smithi079 ceph-mon[33002]: pgmap v206: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 936 KiB/s wr, 285 op/s 2024-04-25T19:06:04.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:04 smithi045 ceph-mon[27776]: pgmap v206: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 936 KiB/s wr, 285 op/s 2024-04-25T19:06:06.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:06 smithi062 ceph-mon[32844]: pgmap v207: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.5 MiB/s wr, 477 op/s 2024-04-25T19:06:06.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:06 smithi079 ceph-mon[33002]: pgmap v207: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.5 MiB/s wr, 477 op/s 2024-04-25T19:06:06.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:06 smithi045 ceph-mon[27776]: pgmap v207: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.5 MiB/s wr, 477 op/s 2024-04-25T19:06:07.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:07 smithi062 ceph-mon[32844]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T19:06:07.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:07 smithi079 ceph-mon[33002]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T19:06:07.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:07 smithi045 ceph-mon[27776]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T19:06:08.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:08 smithi062 ceph-mon[32844]: pgmap v209: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 406 op/s 2024-04-25T19:06:08.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:08 smithi079 ceph-mon[33002]: pgmap v209: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 406 op/s 2024-04-25T19:06:08.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:08 smithi045 ceph-mon[27776]: pgmap v209: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 406 op/s 2024-04-25T19:06:10.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:10 smithi062 ceph-mon[32844]: pgmap v210: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 406 op/s 2024-04-25T19:06:10.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:10 smithi079 ceph-mon[33002]: pgmap v210: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 406 op/s 2024-04-25T19:06:10.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:10 smithi045 ceph-mon[27776]: pgmap v210: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 406 op/s 2024-04-25T19:06:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:12 smithi062 ceph-mon[32844]: pgmap v211: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 434 op/s 2024-04-25T19:06:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:12 smithi062 ceph-mon[32844]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T19:06:12.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:12 smithi079 ceph-mon[33002]: pgmap v211: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 434 op/s 2024-04-25T19:06:12.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:12 smithi079 ceph-mon[33002]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T19:06:12.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:12 smithi045 ceph-mon[27776]: pgmap v211: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 434 op/s 2024-04-25T19:06:12.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:12 smithi045 ceph-mon[27776]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T19:06:14.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:14 smithi062 ceph-mon[32844]: pgmap v213: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 658 KiB/s wr, 255 op/s 2024-04-25T19:06:14.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:14 smithi079 ceph-mon[33002]: pgmap v213: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 658 KiB/s wr, 255 op/s 2024-04-25T19:06:14.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:14 smithi045 ceph-mon[27776]: pgmap v213: 97 pgs: 97 active+clean; 85 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 658 KiB/s wr, 255 op/s 2024-04-25T19:06:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.560+0000 7f0014c00640 1 -- 172.21.15.45:0/1471994170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0010117080 msgr2=0x7f0010119470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:15.562 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.560+0000 7f0014c00640 1 --2- 172.21.15.45:0/1471994170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0010117080 0x7f0010119470 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0008009ca0 tx=0x7f000802a8c0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.560+0000 7f0014c00640 1 -- 172.21.15.45:0/1471994170 shutdown_connections 2024-04-25T19:06:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.560+0000 7f0014c00640 1 --2- 172.21.15.45:0/1471994170 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f0010117080 0x7f0010119470 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.560+0000 7f0014c00640 1 --2- 172.21.15.45:0/1471994170 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0010109730 0x7f0010109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.560+0000 7f0014c00640 1 --2- 172.21.15.45:0/1471994170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f001010c6b0 0x7f0010109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.560+0000 7f0014c00640 1 -- 172.21.15.45:0/1471994170 >> 172.21.15.45:0/1471994170 conn(0x7f001006d030 msgr2=0x7f001006d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.560+0000 7f0014c00640 1 -- 172.21.15.45:0/1471994170 shutdown_connections 2024-04-25T19:06:15.563 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.560+0000 7f0014c00640 1 -- 172.21.15.45:0/1471994170 wait complete. 2024-04-25T19:06:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.561+0000 7f0014c00640 1 Processor -- start 2024-04-25T19:06:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.561+0000 7f0014c00640 1 -- start start 2024-04-25T19:06:15.565 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f0014c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0010109730 0x7f00101a3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f0014c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001010c6b0 0x7f00101a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f0014c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0010117080 0x7f00101a7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f0014c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f001011cc00 con 0x7f0010117080 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f0014c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f001011ca80 con 0x7f001010c6b0 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f0014c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f001011cd80 con 0x7f0010109730 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f000d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001010c6b0 0x7f00101a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f000de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0010109730 0x7f00101a3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f000de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0010109730 0x7f00101a3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33798/0 (socket says 172.21.15.45:33798) 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f000d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001010c6b0 0x7f00101a3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:39264/0 (socket says 172.21.15.45:39264) 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f000de00640 1 -- 172.21.15.45:0/3346699902 learned_addr learned my addr 172.21.15.45:0/3346699902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.562+0000 7f000e800640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0010117080 0x7f00101a7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.563+0000 7f000d400640 1 -- 172.21.15.45:0/3346699902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0010109730 msgr2=0x7f00101a3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:15.566 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.563+0000 7f000d400640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0010109730 0x7f00101a3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.563+0000 7f000d400640 1 -- 172.21.15.45:0/3346699902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0010117080 msgr2=0x7f00101a7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.563+0000 7f000d400640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0010117080 0x7f00101a7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.563+0000 7f000d400640 1 -- 172.21.15.45:0/3346699902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00101a8130 con 0x7f001010c6b0 2024-04-25T19:06:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.563+0000 7f000de00640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0010109730 0x7f00101a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:15.567 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.563+0000 7f000d400640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001010c6b0 0x7f00101a3680 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f000000e9c0 tx=0x7f000000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.564+0000 7f0006a00640 1 -- 172.21.15.45:0/3346699902 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000000ccc0 con 0x7f001010c6b0 2024-04-25T19:06:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.564+0000 7f0006a00640 1 -- 172.21.15.45:0/3346699902 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00000035f0 con 0x7f001010c6b0 2024-04-25T19:06:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.564+0000 7f0006a00640 1 -- 172.21.15.45:0/3346699902 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000000ce60 con 0x7f001010c6b0 2024-04-25T19:06:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.564+0000 7f0014c00640 1 -- 172.21.15.45:0/3346699902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00101a8420 con 0x7f001010c6b0 2024-04-25T19:06:15.568 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.564+0000 7f0014c00640 1 -- 172.21.15.45:0/3346699902 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00101afcb0 con 0x7f001010c6b0 2024-04-25T19:06:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.569+0000 7f0017a6b740 1 Processor -- start 2024-04-25T19:06:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.569+0000 7f0017a6b740 1 -- start start 2024-04-25T19:06:15.571 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.569+0000 7f0006a00640 1 -- 172.21.15.45:0/3346699902 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f0000004000 con 0x7f001010c6b0 2024-04-25T19:06:15.572 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.569+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.569+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.569+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113b0a00 0x5631113b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.569+0000 7f0017a6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x56311130b900 con 0x5631113a5820 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.569+0000 7f0017a6b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631112b1ef0 con 0x5631113a6140 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.569+0000 7f0017a6b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5631113078c0 con 0x5631113b0a00 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.569+0000 7f000e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.570+0000 7f000de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.570+0000 7f000e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:41182/0 (socket says 172.21.15.45:41182) 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.570+0000 7f000d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113b0a00 0x5631113b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.570+0000 7f000e800640 1 -- 172.21.15.45:0/3937314199 learned_addr learned my addr 172.21.15.45:0/3937314199 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.570+0000 7f000de00640 1 -- 172.21.15.45:0/3937314199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113b0a00 msgr2=0x5631113b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.570+0000 7f000de00640 1 --2- 172.21.15.45:0/3937314199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113b0a00 0x5631113b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.573 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.570+0000 7f000de00640 1 -- 172.21.15.45:0/3937314199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 msgr2=0x5631113a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.570+0000 7f000de00640 1 --2- 172.21.15.45:0/3937314199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113a5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.570+0000 7f000de00640 1 -- 172.21.15.45:0/3937314199 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311101e210 con 0x5631113a6140 2024-04-25T19:06:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.570+0000 7f000de00640 1 --2- 172.21.15.45:0/3937314199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113b04c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efff8005d00 tx=0x7efff8011950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6808ddb91c07756f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.571+0000 7effff000640 1 -- 172.21.15.45:0/3937314199 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8005020 con 0x5631113a6140 2024-04-25T19:06:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.571+0000 7effff000640 1 -- 172.21.15.45:0/3937314199 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff80047a0 con 0x5631113a6140 2024-04-25T19:06:15.574 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.571+0000 7effff000640 1 -- 172.21.15.45:0/3937314199 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff800f860 con 0x5631113a6140 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.571+0000 7f0006a00640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efff40a3d90 0x7efff40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.571+0000 7f0006a00640 1 -- 172.21.15.45:0/3346699902 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f00000d0be0 con 0x7f001010c6b0 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.571+0000 7f0017a6b740 1 -- 172.21.15.45:0/3937314199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 msgr2=0x5631113b04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.571+0000 7f0017a6b740 1 --2- 172.21.15.45:0/3937314199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113b04c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efff8005d00 tx=0x7efff8011950 comp rx=0 tx=0).stop 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.571+0000 7f000e800640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efff40a3d90 0x7efff40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.572+0000 7f0017a6b740 1 -- 172.21.15.45:0/3937314199 shutdown_connections 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.572+0000 7f0017a6b740 1 --2- 172.21.15.45:0/3937314199 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113b0a00 0x5631113b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.572+0000 7f0017a6b740 1 --2- 172.21.15.45:0/3937314199 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113b04c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.572+0000 7f0017a6b740 1 --2- 172.21.15.45:0/3937314199 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.572+0000 7f0017a6b740 1 -- 172.21.15.45:0/3937314199 >> 172.21.15.45:0/3937314199 conn(0x5631113a1d30 msgr2=0x5631113a2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.572+0000 7f0017a6b740 1 -- 172.21.15.45:0/3937314199 shutdown_connections 2024-04-25T19:06:15.577 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.572+0000 7f0017a6b740 1 -- 172.21.15.45:0/3937314199 wait complete. 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.572+0000 7f000e800640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efff40a3d90 0x7efff40a6250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0008009ca0 tx=0x7f0008005d80 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.572+0000 7f0017a6b740 1 -- start start 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.573+0000 7f0017a6b740 1 Processor -- start 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.573+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113ad960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.573+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.573+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113b0a00 0x5631113ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.573+0000 7f0017a6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5631113b3630 con 0x5631113a5820 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.573+0000 7f0017a6b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5631112c5310 con 0x5631113a6140 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.573+0000 7f0017a6b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5631113b37b0 con 0x5631113b0a00 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.573+0000 7f0017a6b740 10 client.0 _ll_register_callbacks cb 0x563111293cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.573+0000 7f000d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.573+0000 7f000de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7f000d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:39288/0 (socket says 172.21.15.45:39288) 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7f000e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113b0a00 0x5631113ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7f000d400640 1 -- 172.21.15.45:0/1303126649 learned_addr learned my addr 172.21.15.45:0/1303126649 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7f000d400640 1 -- 172.21.15.45:0/1303126649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113b0a00 msgr2=0x5631113ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7f000d400640 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113b0a00 0x5631113ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7f000d400640 1 -- 172.21.15.45:0/1303126649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 msgr2=0x5631113ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7f000d400640 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7f000d400640 1 -- 172.21.15.45:0/1303126649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631110ea620 con 0x5631113a6140 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7f000d400640 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113adea0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f000009ca80 tx=0x7f0000098e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ae1a5551d023bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7efff3400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.574+0000 7efff3400640 1 -- 172.21.15.45:0/1303126649 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000009d1f0 con 0x5631113a6140 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7efff3400640 1 -- 172.21.15.45:0/1303126649 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f00000037a0 con 0x5631113a6140 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7f0017a6b740 1 -- 172.21.15.45:0/1303126649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff80047a0 con 0x5631113a6140 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7efff3400640 1 -- 172.21.15.45:0/1303126649 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f000009caf0 con 0x5631113a6140 2024-04-25T19:06:15.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7f0017a6b740 10 client.25874 fetch_fsmap learned FSMap version 32 2024-04-25T19:06:15.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7f0017a6b740 1 -- 172.21.15.45:0/1303126649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631113bf850 con 0x5631113a6140 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7efff3400640 1 -- 172.21.15.45:0/1303126649 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff80047a0 con 0x5631113a6140 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7efff3400640 1 -- 172.21.15.45:0/1303126649 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f00000b8070 con 0x5631113a6140 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7f0017a6b740 10 client.25874 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7f0017a6b740 10 client.25874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7f0017a6b740 20 client.25874 populate_metadata read hostname 'smithi045' 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7f0017a6b740 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5631113c0d60 0x5631113c3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7f0017a6b740 4 client.25874 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.575+0000 7f0017a6b740 1 -- 172.21.15.45:0/1303126649 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56311131f8b0 con 0x5631113c0d60 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.576+0000 7f000de00640 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5631113c0d60 0x5631113c3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.576+0000 7f000de00640 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5631113c0d60 0x5631113c3170 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eeb683466616b3b server_cookie=ea0c02244c94cc9d in_seq=0 out_seq=0 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.576+0000 7efff3400640 10 client.25874 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7efff3400640 1 -- 172.21.15.45:0/1303126649 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56311131f8b0 con 0x5631113c0d60 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7efff3400640 10 client.25874 handle_command_reply: tid=0 2024-04-25T19:06:15.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 client.25874 shutdown 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 -- 172.21.15.45:0/1303126649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 msgr2=0x5631113adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113adea0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f000009ca80 tx=0x7f0000098e80 comp rx=0 tx=0).stop 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 -- 172.21.15.45:0/1303126649 shutdown_connections 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5631113c0d60 0x5631113c3170 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113b0a00 0x5631113ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113a6140 0x5631113adea0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 --2- 172.21.15.45:0/1303126649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 -- 172.21.15.45:0/1303126649 >> 172.21.15.45:0/1303126649 conn(0x5631113a1d30 msgr2=0x5631113a7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7efff3400640 0 client.25874 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 -- 172.21.15.45:0/1303126649 shutdown_connections 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 1 -- 172.21.15.45:0/1303126649 wait complete. 2024-04-25T19:06:15.582 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.578+0000 7f0017a6b740 20 client.25874 trim_cache size 0 max 16384 2024-04-25T19:06:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.598+0000 7f0017a6b740 1 Processor -- start 2024-04-25T19:06:15.601 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.598+0000 7f0017a6b740 1 -- start start 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113bc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a6140 0x5631113c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113b0a00 0x5631113c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f0017a6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00101bb060 con 0x5631113a5820 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f0017a6b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b8930 con 0x5631113b0a00 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f0017a6b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b67a0 con 0x5631113a6140 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a6140 0x5631113c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a6140 0x5631113c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33838/0 (socket says 172.21.15.45:33838) 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000d400640 1 -- 172.21.15.45:0/1981108977 learned_addr learned my addr 172.21.15.45:0/1981108977 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000e800640 1 --2- 172.21.15.45:0/1981108977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113b0a00 0x5631113c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000de00640 1 --2- 172.21.15.45:0/1981108977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113bc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000d400640 1 -- 172.21.15.45:0/1981108977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113b0a00 msgr2=0x5631113c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:15.602 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000d400640 1 --2- 172.21.15.45:0/1981108977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113b0a00 0x5631113c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000d400640 1 -- 172.21.15.45:0/1981108977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 msgr2=0x5631113bc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000d400640 1 --2- 172.21.15.45:0/1981108977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113bc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000d400640 1 -- 172.21.15.45:0/1981108977 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631113b3120 con 0x5631113a6140 2024-04-25T19:06:15.603 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.599+0000 7f000de00640 1 --2- 172.21.15.45:0/1981108977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113bc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.600+0000 7f000d400640 1 --2- 172.21.15.45:0/1981108977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a6140 0x5631113c8a80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f000000c6a0 tx=0x7f00000d0e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa1d8fa46d1085be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.600+0000 7efffe600640 1 -- 172.21.15.45:0/1981108977 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000009c630 con 0x5631113a6140 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.600+0000 7efffe600640 1 -- 172.21.15.45:0/1981108977 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000009d1f0 con 0x5631113a6140 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.600+0000 7efffe600640 1 -- 172.21.15.45:0/1981108977 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00000042f0 con 0x5631113a6140 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.601+0000 7f0017a6b740 1 -- 172.21.15.45:0/1981108977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a6140 msgr2=0x5631113c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.601+0000 7f0017a6b740 1 --2- 172.21.15.45:0/1981108977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a6140 0x5631113c8a80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f000000c6a0 tx=0x7f00000d0e60 comp rx=0 tx=0).stop 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.601+0000 7f0017a6b740 1 -- 172.21.15.45:0/1981108977 shutdown_connections 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.601+0000 7f0017a6b740 1 --2- 172.21.15.45:0/1981108977 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113b0a00 0x5631113c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.601+0000 7f0017a6b740 1 --2- 172.21.15.45:0/1981108977 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a6140 0x5631113c8a80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.601+0000 7f0017a6b740 1 --2- 172.21.15.45:0/1981108977 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a5820 0x5631113bc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.601+0000 7f0017a6b740 1 -- 172.21.15.45:0/1981108977 >> 172.21.15.45:0/1981108977 conn(0x5631113a1d30 msgr2=0x5631113a2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.601+0000 7f0017a6b740 1 -- 172.21.15.45:0/1981108977 shutdown_connections 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.602+0000 7f0017a6b740 1 -- 172.21.15.45:0/1981108977 wait complete. 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.602+0000 7f0017a6b740 1 -- start start 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.602+0000 7f0017a6b740 1 Processor -- start 2024-04-25T19:06:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a5820 0x5631113b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a6140 0x5631113b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f0017a6b740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113b0a00 0x56311142eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f0017a6b740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b6df0 con 0x5631113a6140 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f0017a6b740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b8260 con 0x5631113b0a00 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f0017a6b740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f00101b74c0 con 0x5631113a5820 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f0017a6b740 10 client.0 _ll_register_callbacks cb 0x7f0000098670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a5820 0x5631113b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a5820 0x5631113b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:33848/0 (socket says 172.21.15.45:33848) 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000de00640 1 -- 172.21.15.45:0/3345562905 learned_addr learned my addr 172.21.15.45:0/3345562905 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000d400640 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a6140 0x5631113b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000e800640 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113b0a00 0x56311142eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000de00640 1 -- 172.21.15.45:0/3345562905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113b0a00 msgr2=0x56311142eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000de00640 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113b0a00 0x56311142eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000de00640 1 -- 172.21.15.45:0/3345562905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a6140 msgr2=0x5631113b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000de00640 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a6140 0x5631113b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000de00640 1 -- 172.21.15.45:0/3345562905 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311142f050 con 0x5631113a5820 2024-04-25T19:06:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.603+0000 7f000d400640 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a6140 0x5631113b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.604+0000 7f000de00640 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a5820 0x5631113b6ac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efff8005d00 tx=0x7efff8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92c51d31caed688c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.604+0000 7efff3400640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:06:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.604+0000 7efff3400640 1 -- 172.21.15.45:0/3345562905 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff802bb50 con 0x5631113a5820 2024-04-25T19:06:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.604+0000 7efff3400640 1 -- 172.21.15.45:0/3345562905 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7efff8016040 con 0x5631113a5820 2024-04-25T19:06:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.604+0000 7f0017a6b740 1 -- 172.21.15.45:0/3345562905 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f000009d1f0 con 0x5631113a5820 2024-04-25T19:06:15.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.604+0000 7efff3400640 1 -- 172.21.15.45:0/3345562905 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efff803b050 con 0x5631113a5820 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.604+0000 7f0017a6b740 10 client.25894 fetch_fsmap learned FSMap version 32 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.604+0000 7f0017a6b740 1 -- 172.21.15.45:0/3345562905 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5631114302c0 con 0x5631113a5820 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.604+0000 7efff3400640 1 -- 172.21.15.45:0/3345562905 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000009d1f0 con 0x5631113a5820 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.605+0000 7efff3400640 1 -- 172.21.15.45:0/3345562905 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7efff802b470 con 0x5631113a5820 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.605+0000 7f0017a6b740 10 client.25894 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.605+0000 7f0017a6b740 10 client.25894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.605+0000 7f0017a6b740 20 client.25894 populate_metadata read hostname 'smithi045' 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.605+0000 7f0017a6b740 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563111430850 0x563111432c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.605+0000 7f0017a6b740 4 client.25894 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.605+0000 7f0017a6b740 1 -- 172.21.15.45:0/3345562905 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563111319320 con 0x563111430850 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.605+0000 7f000d400640 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563111430850 0x563111432c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.605+0000 7f000d400640 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563111430850 0x563111432c40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1812ddae9d1da9 server_cookie=4225120396f049f5 in_seq=0 out_seq=0 2024-04-25T19:06:15.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.605+0000 7efff3400640 10 client.25894 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7efff3400640 1 -- 172.21.15.45:0/3345562905 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563111319320 con 0x563111430850 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7efff3400640 10 client.25894 handle_command_reply: tid=0 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 client.25894 shutdown 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 -- 172.21.15.45:0/3345562905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a5820 msgr2=0x5631113b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a5820 0x5631113b6ac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efff8005d00 tx=0x7efff8009310 comp rx=0 tx=0).stop 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 -- 172.21.15.45:0/3345562905 shutdown_connections 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x563111430850 0x563111432c40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5631113b0a00 0x56311142eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631113a6140 0x5631113b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 --2- 172.21.15.45:0/3345562905 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5631113a5820 0x5631113b6ac0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7efff3400640 0 client.25894 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 -- 172.21.15.45:0/3345562905 >> 172.21.15.45:0/3345562905 conn(0x5631113a1d30 msgr2=0x5631113ad920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 -- 172.21.15.45:0/3345562905 shutdown_connections 2024-04-25T19:06:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.619+0000 7f0017a6b740 1 -- 172.21.15.45:0/3345562905 wait complete. 2024-04-25T19:06:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0017a6b740 20 client.25894 trim_cache size 0 max 16384 2024-04-25T19:06:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:06:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:06:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:06:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:06:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0014c00640 1 -- 172.21.15.45:0/3346699902 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efff40a3d90 msgr2=0x7efff40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0014c00640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efff40a3d90 0x7efff40a6250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0008009ca0 tx=0x7f0008005d80 comp rx=0 tx=0).stop 2024-04-25T19:06:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0014c00640 1 -- 172.21.15.45:0/3346699902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001010c6b0 msgr2=0x7f00101a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0014c00640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001010c6b0 0x7f00101a3680 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f000000e9c0 tx=0x7f000000ee80 comp rx=0 tx=0).stop 2024-04-25T19:06:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0014c00640 1 -- 172.21.15.45:0/3346699902 shutdown_connections 2024-04-25T19:06:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0014c00640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7efff40a3d90 0x7efff40a6250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0014c00640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0010117080 0x7f00101a7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0014c00640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f001010c6b0 0x7f00101a3680 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0014c00640 1 --2- 172.21.15.45:0/3346699902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0010109730 0x7f00101a3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.620+0000 7f0014c00640 1 -- 172.21.15.45:0/3346699902 >> 172.21.15.45:0/3346699902 conn(0x7f001006d030 msgr2=0x7f0010117460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.621+0000 7f0014c00640 1 -- 172.21.15.45:0/3346699902 shutdown_connections 2024-04-25T19:06:15.625 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:15.621+0000 7f0014c00640 1 -- 172.21.15.45:0/3346699902 wait complete. 2024-04-25T19:06:16.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:16 smithi062 ceph-mon[32844]: pgmap v214: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 428 op/s 2024-04-25T19:06:16.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:16 smithi079 ceph-mon[33002]: pgmap v214: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 428 op/s 2024-04-25T19:06:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:16 smithi045 ceph-mon[27776]: pgmap v214: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.2 MiB/s wr, 428 op/s 2024-04-25T19:06:18.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:18 smithi079 ceph-mon[33002]: pgmap v215: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.0 MiB/s wr, 380 op/s 2024-04-25T19:06:18.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:18 smithi045 ceph-mon[27776]: pgmap v215: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.0 MiB/s wr, 380 op/s 2024-04-25T19:06:18.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:18 smithi062 ceph-mon[32844]: pgmap v215: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.0 MiB/s wr, 380 op/s 2024-04-25T19:06:19.197 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0x63e5d2a 2024-04-25T19:06:20.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:20 smithi079 ceph-mon[33002]: pgmap v216: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.0 MiB/s wr, 380 op/s 2024-04-25T19:06:20.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:20 smithi045 ceph-mon[27776]: pgmap v216: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.0 MiB/s wr, 380 op/s 2024-04-25T19:06:20.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:20 smithi062 ceph-mon[32844]: pgmap v216: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 1.0 MiB/s wr, 380 op/s 2024-04-25T19:06:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:22 smithi079 ceph-mon[33002]: pgmap v217: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.1 MiB/s wr, 356 op/s 2024-04-25T19:06:22.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:22 smithi045 ceph-mon[27776]: pgmap v217: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.1 MiB/s wr, 356 op/s 2024-04-25T19:06:22.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:22 smithi062 ceph-mon[32844]: pgmap v217: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 1.1 MiB/s wr, 356 op/s 2024-04-25T19:06:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:24 smithi079 ceph-mon[33002]: pgmap v218: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 948 KiB/s wr, 308 op/s 2024-04-25T19:06:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:24 smithi045 ceph-mon[27776]: pgmap v218: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 948 KiB/s wr, 308 op/s 2024-04-25T19:06:24.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:24 smithi062 ceph-mon[32844]: pgmap v218: 97 pgs: 97 active+clean; 93 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 948 KiB/s wr, 308 op/s 2024-04-25T19:06:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:26 smithi079 ceph-mon[33002]: pgmap v219: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 452 op/s 2024-04-25T19:06:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:26 smithi045 ceph-mon[27776]: pgmap v219: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 452 op/s 2024-04-25T19:06:26.474 INFO:tasks.workunit.client.0.smithi045.stdout:truncating to largest ever: 0x63e9b52 2024-04-25T19:06:26.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:26 smithi062 ceph-mon[32844]: pgmap v219: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 452 op/s 2024-04-25T19:06:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:28 smithi079 ceph-mon[33002]: pgmap v220: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 910 KiB/s wr, 305 op/s 2024-04-25T19:06:28.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:28 smithi045 ceph-mon[27776]: pgmap v220: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 910 KiB/s wr, 305 op/s 2024-04-25T19:06:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:28 smithi062 ceph-mon[32844]: pgmap v220: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 910 KiB/s wr, 305 op/s 2024-04-25T19:06:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.771+0000 7fedfbe00640 1 -- 172.21.15.45:0/3030153264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedfc102980 msgr2=0x7fedfc1069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.771+0000 7fedfbe00640 1 --2- 172.21.15.45:0/3030153264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedfc102980 0x7fedfc1069c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fede8009900 tx=0x7fede802a410 comp rx=0 tx=0).stop 2024-04-25T19:06:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.772+0000 7fedfbe00640 1 -- 172.21.15.45:0/3030153264 shutdown_connections 2024-04-25T19:06:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.772+0000 7fedfbe00640 1 --2- 172.21.15.45:0/3030153264 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedfc1070a0 0x7fedfc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.774 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.772+0000 7fedfbe00640 1 --2- 172.21.15.45:0/3030153264 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedfc102980 0x7fedfc1069c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.772+0000 7fedfbe00640 1 --2- 172.21.15.45:0/3030153264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedfc101fd0 0x7fedfc1023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.772+0000 7fedfbe00640 1 -- 172.21.15.45:0/3030153264 >> 172.21.15.45:0/3030153264 conn(0x7fedfc0fd890 msgr2=0x7fedfc0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.772+0000 7fedfbe00640 1 -- 172.21.15.45:0/3030153264 shutdown_connections 2024-04-25T19:06:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.772+0000 7fedfbe00640 1 -- 172.21.15.45:0/3030153264 wait complete. 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.773+0000 7fedfbe00640 1 Processor -- start 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.773+0000 7fedfbe00640 1 -- start start 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.773+0000 7fedfbe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedfc101fd0 0x7fedfc1a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.773+0000 7fedfbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedfc1070a0 0x7fedfc1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedfc19a630 0x7fedfc19aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfbe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc112010 con 0x7fedfc19a630 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfbe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc111e90 con 0x7fedfc101fd0 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfbe00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc112190 con 0x7fedfc1070a0 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfa000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedfc1070a0 0x7fedfc1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfa000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedfc1070a0 0x7fedfc1a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43216/0 (socket says 172.21.15.45:43216) 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfa000640 1 -- 172.21.15.45:0/1646211725 learned_addr learned my addr 172.21.15.45:0/1646211725 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfaa00640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedfc101fd0 0x7fedfc1a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfb400640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedfc19a630 0x7fedfc19aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfa000640 1 -- 172.21.15.45:0/1646211725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedfc101fd0 msgr2=0x7fedfc1a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfa000640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedfc101fd0 0x7fedfc1a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfa000640 1 -- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedfc19a630 msgr2=0x7fedfc19aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfa000640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedfc19a630 0x7fedfc19aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfa000640 1 -- 172.21.15.45:0/1646211725 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedfc19b230 con 0x7fedfc1070a0 2024-04-25T19:06:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfb400640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedfc19a630 0x7fedfc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.774+0000 7fedfaa00640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedfc101fd0 0x7fedfc1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.775+0000 7fedfa000640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedfc1070a0 0x7fedfc1a0ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fede8005ab0 tx=0x7fede8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.775+0000 7fedf3400640 1 -- 172.21.15.45:0/1646211725 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede803f070 con 0x7fedfc1070a0 2024-04-25T19:06:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.775+0000 7fedf3400640 1 -- 172.21.15.45:0/1646211725 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede8031b50 con 0x7fedfc1070a0 2024-04-25T19:06:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.775+0000 7fedf3400640 1 -- 172.21.15.45:0/1646211725 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8035030 con 0x7fedfc1070a0 2024-04-25T19:06:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.775+0000 7fedfbe00640 1 -- 172.21.15.45:0/1646211725 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedfc19b4c0 con 0x7fedfc1070a0 2024-04-25T19:06:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.775+0000 7fedfbe00640 1 -- 172.21.15.45:0/1646211725 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedfc1a7650 con 0x7fedfc1070a0 2024-04-25T19:06:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.779+0000 7fee045f9740 1 Processor -- start 2024-04-25T19:06:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.779+0000 7fee045f9740 1 -- start start 2024-04-25T19:06:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.779+0000 7fedf3400640 1 -- 172.21.15.45:0/1646211725 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fede8042070 con 0x7fedfc1070a0 2024-04-25T19:06:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.779+0000 7fee045f9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27349b0 0x5636d2734d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.779+0000 7fee045f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27352d0 0x5636d273f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.779+0000 7fee045f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d273fb90 0x5636d2741f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.782 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.779+0000 7fee045f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5636d269a900 con 0x5636d273fb90 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.779+0000 7fee045f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5636d2640ef0 con 0x5636d27352d0 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.779+0000 7fee045f9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5636d26968c0 con 0x5636d27349b0 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfb400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27349b0 0x5636d2734d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfb400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27349b0 0x5636d2734d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43222/0 (socket says 172.21.15.45:43222) 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfb400640 1 -- 172.21.15.45:0/275196046 learned_addr learned my addr 172.21.15.45:0/275196046 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfaa00640 1 --2- 172.21.15.45:0/275196046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27352d0 0x5636d273f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfa000640 1 --2- 172.21.15.45:0/275196046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d273fb90 0x5636d2741f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfb400640 1 -- 172.21.15.45:0/275196046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27352d0 msgr2=0x5636d273f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfb400640 1 --2- 172.21.15.45:0/275196046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27352d0 0x5636d273f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfb400640 1 -- 172.21.15.45:0/275196046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d273fb90 msgr2=0x5636d2741f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfb400640 1 --2- 172.21.15.45:0/275196046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d273fb90 0x5636d2741f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfb400640 1 -- 172.21.15.45:0/275196046 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636d23ad210 con 0x5636d27349b0 2024-04-25T19:06:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.780+0000 7fedfaa00640 1 --2- 172.21.15.45:0/275196046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27352d0 0x5636d273f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.781+0000 7fedfb400640 1 --2- 172.21.15.45:0/275196046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27349b0 0x5636d2734d90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009fd0 tx=0x7fedf4010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3afda608d9495969 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.781+0000 7fedf3400640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fede00a3d80 0x7fede00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.781+0000 7fede7e00640 1 -- 172.21.15.45:0/275196046 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf4011700 con 0x5636d27349b0 2024-04-25T19:06:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.781+0000 7fede7e00640 1 -- 172.21.15.45:0/275196046 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4011d50 con 0x5636d27349b0 2024-04-25T19:06:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.781+0000 7fede7e00640 1 -- 172.21.15.45:0/275196046 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf400e3c0 con 0x5636d27349b0 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.781+0000 7fedf3400640 1 -- 172.21.15.45:0/1646211725 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fede80f0df0 con 0x7fedfc1070a0 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.781+0000 7fedfaa00640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fede00a3d80 0x7fede00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.782+0000 7fee045f9740 1 -- 172.21.15.45:0/275196046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27349b0 msgr2=0x5636d2734d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.782+0000 7fee045f9740 1 --2- 172.21.15.45:0/275196046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27349b0 0x5636d2734d90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009fd0 tx=0x7fedf4010860 comp rx=0 tx=0).stop 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.782+0000 7fee045f9740 1 -- 172.21.15.45:0/275196046 shutdown_connections 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.782+0000 7fee045f9740 1 --2- 172.21.15.45:0/275196046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d273fb90 0x5636d2741f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.782+0000 7fee045f9740 1 --2- 172.21.15.45:0/275196046 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27352d0 0x5636d273f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.782+0000 7fee045f9740 1 --2- 172.21.15.45:0/275196046 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27349b0 0x5636d2734d90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.782+0000 7fee045f9740 1 -- 172.21.15.45:0/275196046 >> 172.21.15.45:0/275196046 conn(0x5636d2730ec0 msgr2=0x5636d27312a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.782+0000 7fedfaa00640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fede00a3d80 0x7fede00a6240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedec009fd0 tx=0x7fedec005f90 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.782+0000 7fee045f9740 1 -- 172.21.15.45:0/275196046 shutdown_connections 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.782+0000 7fee045f9740 1 -- 172.21.15.45:0/275196046 wait complete. 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.783+0000 7fee045f9740 1 -- start start 2024-04-25T19:06:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.783+0000 7fee045f9740 1 Processor -- start 2024-04-25T19:06:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fee045f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d273cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fee045f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d27352d0 0x5636d273d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.789 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fee045f9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 0x5636d273d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fee045f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5636d27427c0 con 0x5636d27352d0 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fee045f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5636d2654310 con 0x5636d27349b0 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fee045f9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5636d2742940 con 0x5636d273fb90 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fee045f9740 10 client.0 _ll_register_callbacks cb 0x5636d2622cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fedfaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d273cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fedfb400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 0x5636d273d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fedfaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d273cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:34748/0 (socket says 172.21.15.45:34748) 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fedfaa00640 1 -- 172.21.15.45:0/2585959628 learned_addr learned my addr 172.21.15.45:0/2585959628 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.784+0000 7fedfa000640 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d27352d0 0x5636d273d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.785+0000 7fedfb400640 1 -- 172.21.15.45:0/2585959628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 msgr2=0x5636d273cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.785+0000 7fedfb400640 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d273cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.790 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.785+0000 7fedfb400640 1 -- 172.21.15.45:0/2585959628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d27352d0 msgr2=0x5636d273d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.785+0000 7fedfb400640 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d27352d0 0x5636d273d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.785+0000 7fedfb400640 1 -- 172.21.15.45:0/2585959628 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d2479620 con 0x5636d273fb90 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.785+0000 7fedfaa00640 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d273cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.785+0000 7fedfb400640 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 0x5636d273d6b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fedf4000f80 tx=0x7fedf4011f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b06e80528b9e1183 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.785+0000 7fede4c00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.786+0000 7fede4c00640 1 -- 172.21.15.45:0/2585959628 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf402b060 con 0x5636d273fb90 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.786+0000 7fede4c00640 1 -- 172.21.15.45:0/2585959628 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fedf4004c50 con 0x5636d273fb90 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.786+0000 7fee045f9740 1 -- 172.21.15.45:0/2585959628 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf4011d50 con 0x5636d273fb90 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.786+0000 7fede4c00640 1 -- 172.21.15.45:0/2585959628 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fedf4004e50 con 0x5636d273fb90 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.786+0000 7fee045f9740 10 client.25906 fetch_fsmap learned FSMap version 32 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.786+0000 7fee045f9740 1 -- 172.21.15.45:0/2585959628 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5636d274eb30 con 0x5636d273fb90 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.786+0000 7fede4c00640 1 -- 172.21.15.45:0/2585959628 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedf4011d50 con 0x5636d273fb90 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.786+0000 7fede4c00640 1 -- 172.21.15.45:0/2585959628 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fedf40053a0 con 0x5636d273fb90 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.786+0000 7fee045f9740 10 client.25906 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.786+0000 7fee045f9740 10 client.25906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.787+0000 7fee045f9740 20 client.25906 populate_metadata read hostname 'smithi045' 2024-04-25T19:06:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.787+0000 7fee045f9740 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5636d274fee0 0x5636d27522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.787+0000 7fee045f9740 4 client.25906 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.787+0000 7fee045f9740 1 -- 172.21.15.45:0/2585959628 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636d26ae8b0 con 0x5636d274fee0 2024-04-25T19:06:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.787+0000 7fedfaa00640 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5636d274fee0 0x5636d27522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.788+0000 7fedfaa00640 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5636d274fee0 0x5636d27522d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6de4a6e7c5ac7cc server_cookie=9d71307aee896198 in_seq=0 out_seq=0 2024-04-25T19:06:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.788+0000 7fede4c00640 10 client.25906 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fede4c00640 1 -- 172.21.15.45:0/2585959628 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636d26ae8b0 con 0x5636d274fee0 2024-04-25T19:06:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fede4c00640 10 client.25906 handle_command_reply: tid=0 2024-04-25T19:06:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 client.25906 shutdown 2024-04-25T19:06:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 -- 172.21.15.45:0/2585959628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 msgr2=0x5636d273d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 0x5636d273d6b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fedf4000f80 tx=0x7fedf4011f00 comp rx=0 tx=0).stop 2024-04-25T19:06:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 -- 172.21.15.45:0/2585959628 shutdown_connections 2024-04-25T19:06:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5636d274fee0 0x5636d27522d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 0x5636d273d6b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d27352d0 0x5636d273d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fede4c00640 0 client.25906 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 --2- 172.21.15.45:0/2585959628 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d273cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 -- 172.21.15.45:0/2585959628 >> 172.21.15.45:0/2585959628 conn(0x5636d2730ec0 msgr2=0x5636d2736bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 -- 172.21.15.45:0/2585959628 shutdown_connections 2024-04-25T19:06:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 1 -- 172.21.15.45:0/2585959628 wait complete. 2024-04-25T19:06:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.789+0000 7fee045f9740 20 client.25906 trim_cache size 0 max 16384 2024-04-25T19:06:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.819+0000 7fee045f9740 1 Processor -- start 2024-04-25T19:06:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.819+0000 7fee045f9740 1 -- start start 2024-04-25T19:06:28.821 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.819+0000 7fee045f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d273bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.819+0000 7fee045f9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27352d0 0x5636d27307a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.819+0000 7fee045f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d273fb90 0x5636d273eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.819+0000 7fee045f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1b25f0 con 0x5636d273fb90 2024-04-25T19:06:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.819+0000 7fee045f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1afec0 con 0x5636d27349b0 2024-04-25T19:06:28.822 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.819+0000 7fee045f9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1add30 con 0x5636d27352d0 2024-04-25T19:06:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfa000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27352d0 0x5636d27307a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfa000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27352d0 0x5636d27307a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:43248/0 (socket says 172.21.15.45:43248) 2024-04-25T19:06:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfa000640 1 -- 172.21.15.45:0/734944169 learned_addr learned my addr 172.21.15.45:0/734944169 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfb400640 1 --2- 172.21.15.45:0/734944169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d273fb90 0x5636d273eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfaa00640 1 --2- 172.21.15.45:0/734944169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d273bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfa000640 1 -- 172.21.15.45:0/734944169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 msgr2=0x5636d273bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfa000640 1 --2- 172.21.15.45:0/734944169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d273bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfa000640 1 -- 172.21.15.45:0/734944169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d273fb90 msgr2=0x5636d273eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:28.823 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfa000640 1 --2- 172.21.15.45:0/734944169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d273fb90 0x5636d273eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfa000640 1 -- 172.21.15.45:0/734944169 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636d27350f0 con 0x5636d27352d0 2024-04-25T19:06:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.820+0000 7fedfa000640 1 --2- 172.21.15.45:0/734944169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27352d0 0x5636d27307a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fede80b9730 tx=0x7fede80f4940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5768120705502b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.821+0000 7fede7400640 1 -- 172.21.15.45:0/734944169 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8049070 con 0x5636d27352d0 2024-04-25T19:06:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.821+0000 7fede7400640 1 -- 172.21.15.45:0/734944169 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede80bd460 con 0x5636d27352d0 2024-04-25T19:06:28.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.821+0000 7fede7400640 1 -- 172.21.15.45:0/734944169 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede80c59d0 con 0x5636d27352d0 2024-04-25T19:06:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.821+0000 7fee045f9740 1 -- 172.21.15.45:0/734944169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27352d0 msgr2=0x5636d27307a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.821+0000 7fee045f9740 1 --2- 172.21.15.45:0/734944169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27352d0 0x5636d27307a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fede80b9730 tx=0x7fede80f4940 comp rx=0 tx=0).stop 2024-04-25T19:06:28.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.822+0000 7fee045f9740 1 -- 172.21.15.45:0/734944169 shutdown_connections 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.822+0000 7fee045f9740 1 --2- 172.21.15.45:0/734944169 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d273fb90 0x5636d273eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.822+0000 7fee045f9740 1 --2- 172.21.15.45:0/734944169 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d27352d0 0x5636d27307a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.822+0000 7fee045f9740 1 --2- 172.21.15.45:0/734944169 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d273bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.822+0000 7fee045f9740 1 -- 172.21.15.45:0/734944169 >> 172.21.15.45:0/734944169 conn(0x5636d2730ec0 msgr2=0x5636d273e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.822+0000 7fee045f9740 1 -- 172.21.15.45:0/734944169 shutdown_connections 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.822+0000 7fee045f9740 1 -- 172.21.15.45:0/734944169 wait complete. 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.822+0000 7fee045f9740 1 -- start start 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.822+0000 7fee045f9740 1 Processor -- start 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fee045f9740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d27ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fee045f9740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d27352d0 0x5636d27aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fee045f9740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 0x5636d27547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fee045f9740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1ae380 con 0x5636d27352d0 2024-04-25T19:06:28.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fee045f9740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1af7f0 con 0x5636d27349b0 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fee045f9740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fedfc1aea50 con 0x5636d273fb90 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fee045f9740 10 client.0 _ll_register_callbacks cb 0x7fedf4009e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fedfaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d27ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fedfaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d27ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:34768/0 (socket says 172.21.15.45:34768) 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fedfaa00640 1 -- 172.21.15.45:0/1792245487 learned_addr learned my addr 172.21.15.45:0/1792245487 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fedfa000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d27352d0 0x5636d27aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.823+0000 7fedfb400640 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 0x5636d27547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fedfaa00640 1 -- 172.21.15.45:0/1792245487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 msgr2=0x5636d27547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fedfaa00640 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 0x5636d27547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fedfaa00640 1 -- 172.21.15.45:0/1792245487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d27352d0 msgr2=0x5636d27aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fedfaa00640 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d27352d0 0x5636d27aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fedfaa00640 1 -- 172.21.15.45:0/1792245487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636d2754ea0 con 0x5636d27349b0 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fedfb400640 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 0x5636d27547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:28.829 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fedfaa00640 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d27ae5d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x5636d2654420 tx=0x7fedec0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ee263172627c7c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fede4c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fede4c00640 1 -- 172.21.15.45:0/1792245487 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedec00fe50 con 0x5636d27349b0 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fee045f9740 1 -- 172.21.15.45:0/1792245487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fede80bd460 con 0x5636d27349b0 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fede4c00640 1 -- 172.21.15.45:0/1792245487 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fedec01f8f0 con 0x5636d27349b0 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fede4c00640 1 -- 172.21.15.45:0/1792245487 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fedec01fb30 con 0x5636d27349b0 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fee045f9740 10 client.25898 fetch_fsmap learned FSMap version 32 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.824+0000 7fee045f9740 1 -- 172.21.15.45:0/1792245487 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5636d27b5e10 con 0x5636d27349b0 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.825+0000 7fede4c00640 1 -- 172.21.15.45:0/1792245487 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fede80bd460 con 0x5636d27349b0 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.825+0000 7fede4c00640 1 -- 172.21.15.45:0/1792245487 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fedec02b730 con 0x5636d27349b0 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.825+0000 7fee045f9740 10 client.25898 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.825+0000 7fee045f9740 10 client.25898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.825+0000 7fee045f9740 20 client.25898 populate_metadata read hostname 'smithi045' 2024-04-25T19:06:28.830 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.825+0000 7fee045f9740 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5636d27b65a0 0x5636d27b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.825+0000 7fee045f9740 4 client.25898 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:06:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.825+0000 7fee045f9740 1 -- 172.21.15.45:0/1792245487 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636d26a8320 con 0x5636d27b65a0 2024-04-25T19:06:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.825+0000 7fedfa000640 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5636d27b65a0 0x5636d27b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.826+0000 7fedfa000640 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5636d27b65a0 0x5636d27b89c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6403333d8c26f71c server_cookie=92eb6a03ef4f1367 in_seq=0 out_seq=0 2024-04-25T19:06:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.826+0000 7fede4c00640 10 client.25898 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.843+0000 7fede4c00640 1 -- 172.21.15.45:0/1792245487 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636d26a8320 con 0x5636d27b65a0 2024-04-25T19:06:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.843+0000 7fede4c00640 10 client.25898 handle_command_reply: tid=0 2024-04-25T19:06:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.843+0000 7fee045f9740 1 client.25898 shutdown 2024-04-25T19:06:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.843+0000 7fee045f9740 1 -- 172.21.15.45:0/1792245487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 msgr2=0x5636d27ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.843+0000 7fee045f9740 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d27ae5d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x5636d2654420 tx=0x7fedec0046c0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.844+0000 7fee045f9740 1 -- 172.21.15.45:0/1792245487 shutdown_connections 2024-04-25T19:06:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.844+0000 7fee045f9740 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5636d27b65a0 0x5636d27b89c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.844+0000 7fee045f9740 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5636d273fb90 0x5636d27547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.844+0000 7fee045f9740 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5636d27352d0 0x5636d27aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.844+0000 7fee045f9740 1 --2- 172.21.15.45:0/1792245487 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5636d27349b0 0x5636d27ae5d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.844+0000 7fee045f9740 1 -- 172.21.15.45:0/1792245487 >> 172.21.15.45:0/1792245487 conn(0x5636d2730ec0 msgr2=0x5636d274ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.844+0000 7fede4c00640 0 client.25898 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.844+0000 7fee045f9740 1 -- 172.21.15.45:0/1792245487 shutdown_connections 2024-04-25T19:06:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.845+0000 7fee045f9740 1 -- 172.21.15.45:0/1792245487 wait complete. 2024-04-25T19:06:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.845+0000 7fee045f9740 20 client.25898 trim_cache size 0 max 16384 2024-04-25T19:06:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:06:28.848 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:06:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:06:28.849 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:06:28.850 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 -- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fede00a3d80 msgr2=0x7fede00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fede00a3d80 0x7fede00a6240 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedec009fd0 tx=0x7fedec005f90 comp rx=0 tx=0).stop 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 -- 172.21.15.45:0/1646211725 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedfc1070a0 msgr2=0x7fedfc1a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedfc1070a0 0x7fedfc1a0ab0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fede8005ab0 tx=0x7fede8003b90 comp rx=0 tx=0).stop 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 -- 172.21.15.45:0/1646211725 shutdown_connections 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fede00a3d80 0x7fede00a6240 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fedfc19a630 0x7fedfc19aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fedfc1070a0 0x7fedfc1a0ab0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 --2- 172.21.15.45:0/1646211725 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fedfc101fd0 0x7fedfc1a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 -- 172.21.15.45:0/1646211725 >> 172.21.15.45:0/1646211725 conn(0x7fedfc0fd890 msgr2=0x7fedfc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 -- 172.21.15.45:0/1646211725 shutdown_connections 2024-04-25T19:06:28.851 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:28.848+0000 7fedfbe00640 1 -- 172.21.15.45:0/1646211725 wait complete. 2024-04-25T19:06:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:30 smithi079 ceph-mon[33002]: pgmap v221: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 910 KiB/s wr, 305 op/s 2024-04-25T19:06:30.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:30 smithi045 ceph-mon[27776]: pgmap v221: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 910 KiB/s wr, 305 op/s 2024-04-25T19:06:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:30 smithi062 ceph-mon[32844]: pgmap v221: 97 pgs: 97 active+clean; 84 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 910 KiB/s wr, 305 op/s 2024-04-25T19:06:32.045 INFO:tasks.workunit.client.0.smithi045.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-04-25T19:06:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:32 smithi079 ceph-mon[33002]: pgmap v222: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 457 op/s 2024-04-25T19:06:32.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:32 smithi045 ceph-mon[27776]: pgmap v222: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 457 op/s 2024-04-25T19:06:32.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:32 smithi062 ceph-mon[32844]: pgmap v222: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.3 MiB/s wr, 457 op/s 2024-04-25T19:06:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:34 smithi079 ceph-mon[33002]: pgmap v223: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 900 KiB/s wr, 307 op/s 2024-04-25T19:06:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:34 smithi045 ceph-mon[27776]: pgmap v223: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 900 KiB/s wr, 307 op/s 2024-04-25T19:06:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:34 smithi062 ceph-mon[32844]: pgmap v223: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 900 KiB/s wr, 307 op/s 2024-04-25T19:06:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:36 smithi079 ceph-mon[33002]: pgmap v224: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.3 MiB/s wr, 456 op/s 2024-04-25T19:06:36.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:36 smithi045 ceph-mon[27776]: pgmap v224: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.3 MiB/s wr, 456 op/s 2024-04-25T19:06:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:36 smithi062 ceph-mon[32844]: pgmap v224: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 1.3 MiB/s wr, 456 op/s 2024-04-25T19:06:37.454 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0x63ed6b9 2024-04-25T19:06:37.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:37 smithi079 ceph-mon[33002]: pgmap v225: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 880 KiB/s wr, 302 op/s 2024-04-25T19:06:37.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:37 smithi045 ceph-mon[27776]: pgmap v225: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 880 KiB/s wr, 302 op/s 2024-04-25T19:06:37.863 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:37 smithi062 ceph-mon[32844]: pgmap v225: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 880 KiB/s wr, 302 op/s 2024-04-25T19:06:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:38 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:06:38.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:38 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:06:38.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:38 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:06:39.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:39 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:06:39.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:39 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:06:39.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:39 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:06:39.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:39 smithi079 ceph-mon[33002]: pgmap v226: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 880 KiB/s wr, 302 op/s 2024-04-25T19:06:39.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:39 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:06:39.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:39 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:06:39.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:39 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:06:39.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:39 smithi045 ceph-mon[27776]: pgmap v226: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 880 KiB/s wr, 302 op/s 2024-04-25T19:06:39.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:39 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:06:39.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:39 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:06:39.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:39 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:06:39.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:39 smithi062 ceph-mon[32844]: pgmap v226: 97 pgs: 97 active+clean; 99 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 880 KiB/s wr, 302 op/s 2024-04-25T19:06:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.003+0000 7f777de00640 1 -- 172.21.15.45:0/2403207208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7778074c40 msgr2=0x7f777810e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.003+0000 7f777de00640 1 --2- 172.21.15.45:0/2403207208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7778074c40 0x7f777810e2c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7760009900 tx=0x7f776002a480 comp rx=0 tx=0).stop 2024-04-25T19:06:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.003+0000 7f777de00640 1 -- 172.21.15.45:0/2403207208 shutdown_connections 2024-04-25T19:06:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.003+0000 7f777de00640 1 --2- 172.21.15.45:0/2403207208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f777810e800 0x7f7778110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.003+0000 7f777de00640 1 --2- 172.21.15.45:0/2403207208 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7778074c40 0x7f777810e2c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.003+0000 7f777de00640 1 --2- 172.21.15.45:0/2403207208 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7778074320 0x7f7778074700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.004+0000 7f777de00640 1 -- 172.21.15.45:0/2403207208 >> 172.21.15.45:0/2403207208 conn(0x7f77780fda30 msgr2=0x7f77780ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.004+0000 7f777de00640 1 -- 172.21.15.45:0/2403207208 shutdown_connections 2024-04-25T19:06:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.004+0000 7f777de00640 1 -- 172.21.15.45:0/2403207208 wait complete. 2024-04-25T19:06:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f777de00640 1 Processor -- start 2024-04-25T19:06:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f777de00640 1 -- start start 2024-04-25T19:06:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f777de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7778074320 0x7f77781a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f777de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7778074c40 0x7f77781a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f777de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f777810e800 0x7f777819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f777de00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7778114480 con 0x7f7778074c40 2024-04-25T19:06:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f777de00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7778114300 con 0x7f777810e800 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f777de00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7778114600 con 0x7f7778074320 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f7777400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f777810e800 0x7f777819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f7776a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7778074320 0x7f77781a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f7777400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f777810e800 0x7f777819a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:56560/0 (socket says 172.21.15.45:56560) 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.005+0000 7f7777400640 1 -- 172.21.15.45:0/362139415 learned_addr learned my addr 172.21.15.45:0/362139415 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.006+0000 7f7776000640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7778074c40 0x7f77781a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.006+0000 7f7776a00640 1 -- 172.21.15.45:0/362139415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f777810e800 msgr2=0x7f777819a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.006+0000 7f7776a00640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f777810e800 0x7f777819a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.006+0000 7f7776a00640 1 -- 172.21.15.45:0/362139415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7778074c40 msgr2=0x7f77781a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.006+0000 7f7776a00640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7778074c40 0x7f77781a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.006+0000 7f7776a00640 1 -- 172.21.15.45:0/362139415 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777819b010 con 0x7f7778074320 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.006+0000 7f7777400640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f777810e800 0x7f777819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.006+0000 7f7776a00640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7778074320 0x7f77781a06d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9f0 tx=0x7f776800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.007+0000 7f777ca00640 1 -- 172.21.15.45:0/362139415 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776800ccc0 con 0x7f7778074320 2024-04-25T19:06:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.007+0000 7f777ca00640 1 -- 172.21.15.45:0/362139415 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7768005020 con 0x7f7778074320 2024-04-25T19:06:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.007+0000 7f777ca00640 1 -- 172.21.15.45:0/362139415 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768004a00 con 0x7f7778074320 2024-04-25T19:06:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.007+0000 7f777de00640 1 -- 172.21.15.45:0/362139415 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777819b300 con 0x7f7778074320 2024-04-25T19:06:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.007+0000 7f777de00640 1 -- 172.21.15.45:0/362139415 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77781a7520 con 0x7f7778074320 2024-04-25T19:06:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.010+0000 7f777ca00640 1 -- 172.21.15.45:0/362139415 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7768004ba0 con 0x7f7778074320 2024-04-25T19:06:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.010+0000 7f7780c2d740 1 Processor -- start 2024-04-25T19:06:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.010+0000 7f7780c2d740 1 -- start start 2024-04-25T19:06:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d8dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 0x559a95d984c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d9adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7780c2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a95cf3900 con 0x559a95d8e140 2024-04-25T19:06:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7780c2d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a95c99ef0 con 0x559a95d98a00 2024-04-25T19:06:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7780c2d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559a95cef8c0 con 0x559a95d8d820 2024-04-25T19:06:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7777400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d8dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7776000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d9adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7776000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d9adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:56568/0 (socket says 172.21.15.45:56568) 2024-04-25T19:06:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7777400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d8dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60380/0 (socket says 172.21.15.45:60380) 2024-04-25T19:06:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7776000640 1 -- 172.21.15.45:0/1643484634 learned_addr learned my addr 172.21.15.45:0/1643484634 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.011+0000 7f7776a00640 1 --2- 172.21.15.45:0/1643484634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 0x559a95d984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.012+0000 7f7776000640 1 -- 172.21.15.45:0/1643484634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 msgr2=0x559a95d8dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.012+0000 7f7776000640 1 --2- 172.21.15.45:0/1643484634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d8dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.012+0000 7f7776000640 1 -- 172.21.15.45:0/1643484634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 msgr2=0x559a95d984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.012+0000 7f7776000640 1 --2- 172.21.15.45:0/1643484634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 0x559a95d984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.012+0000 7f7776000640 1 -- 172.21.15.45:0/1643484634 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a95a06210 con 0x559a95d98a00 2024-04-25T19:06:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.012+0000 7f7776000640 1 --2- 172.21.15.45:0/1643484634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d9adf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7760000c00 tx=0x7f7760004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80e203d9e94fb91d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.012+0000 7f777ca00640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f775c0a3d90 0x7f775c0a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.013+0000 7f7767e00640 1 -- 172.21.15.45:0/1643484634 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760035040 con 0x559a95d98a00 2024-04-25T19:06:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.013+0000 7f7767e00640 1 -- 172.21.15.45:0/1643484634 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7760004430 con 0x559a95d98a00 2024-04-25T19:06:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.013+0000 7f7767e00640 1 -- 172.21.15.45:0/1643484634 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760004e60 con 0x559a95d98a00 2024-04-25T19:06:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.013+0000 7f777ca00640 1 -- 172.21.15.45:0/362139415 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f77680d0780 con 0x7f7778074320 2024-04-25T19:06:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.013+0000 7f7777400640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f775c0a3d90 0x7f775c0a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.014+0000 7f7780c2d740 1 -- 172.21.15.45:0/1643484634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 msgr2=0x559a95d9adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.014+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1643484634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d9adf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7760000c00 tx=0x7f7760004040 comp rx=0 tx=0).stop 2024-04-25T19:06:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.014+0000 7f7777400640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f775c0a3d90 0x7f775c0a6250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7770005f40 tx=0x7f7770009500 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.014+0000 7f7780c2d740 1 -- 172.21.15.45:0/1643484634 shutdown_connections 2024-04-25T19:06:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.014+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1643484634 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d9adf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.014+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1643484634 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 0x559a95d984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.014+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1643484634 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d8dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.014+0000 7f7780c2d740 1 -- 172.21.15.45:0/1643484634 >> 172.21.15.45:0/1643484634 conn(0x559a95d89d30 msgr2=0x559a95d8a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.014+0000 7f7780c2d740 1 -- 172.21.15.45:0/1643484634 shutdown_connections 2024-04-25T19:06:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.014+0000 7f7780c2d740 1 -- 172.21.15.45:0/1643484634 wait complete. 2024-04-25T19:06:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.015+0000 7f7780c2d740 1 -- start start 2024-04-25T19:06:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.015+0000 7f7780c2d740 1 Processor -- start 2024-04-25T19:06:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d95a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 0x559a95d95f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d96480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7780c2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559a95d9b5d0 con 0x559a95d8e140 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7780c2d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559a95cad310 con 0x559a95d98a00 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7780c2d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559a95d9b750 con 0x559a95d8d820 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7780c2d740 10 client.0 _ll_register_callbacks cb 0x559a95c7bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7777400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d96480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7776000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 0x559a95d95f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7777400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d96480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:56578/0 (socket says 172.21.15.45:56578) 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.016+0000 7f7777400640 1 -- 172.21.15.45:0/2280791770 learned_addr learned my addr 172.21.15.45:0/2280791770 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.017+0000 7f7776a00640 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d95a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.017+0000 7f7776000640 1 -- 172.21.15.45:0/2280791770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 msgr2=0x559a95d95a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.017+0000 7f7776000640 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d95a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.017+0000 7f7776000640 1 -- 172.21.15.45:0/2280791770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 msgr2=0x559a95d96480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.017+0000 7f7776000640 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d96480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.017+0000 7f7776000640 1 -- 172.21.15.45:0/2280791770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a95ad2620 con 0x559a95d8e140 2024-04-25T19:06:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.017+0000 7f7776a00640 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d95a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:06:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.017+0000 7f7776000640 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 0x559a95d95f40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7760005ab0 tx=0x7f776002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c92134502184d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.018+0000 7f7764c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:06:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.023+0000 7f7764c00640 1 -- 172.21.15.45:0/2280791770 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760053060 con 0x559a95d8e140 2024-04-25T19:06:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.023+0000 7f7764c00640 1 -- 172.21.15.45:0/2280791770 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f7760003800 con 0x559a95d8e140 2024-04-25T19:06:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.023+0000 7f7780c2d740 1 -- 172.21.15.45:0/2280791770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7760004430 con 0x559a95d8e140 2024-04-25T19:06:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.023+0000 7f7764c00640 1 -- 172.21.15.45:0/2280791770 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7760035050 con 0x559a95d8e140 2024-04-25T19:06:42.025 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.023+0000 7f7780c2d740 10 client.15762 fetch_fsmap learned FSMap version 32 2024-04-25T19:06:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.023+0000 7f7780c2d740 1 -- 172.21.15.45:0/2280791770 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a95db00a0 con 0x559a95d8e140 2024-04-25T19:06:42.026 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.024+0000 7f7764c00640 1 -- 172.21.15.45:0/2280791770 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7760004430 con 0x559a95d8e140 2024-04-25T19:06:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.024+0000 7f7764c00640 1 -- 172.21.15.45:0/2280791770 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f776004e9a0 con 0x559a95d8e140 2024-04-25T19:06:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.024+0000 7f7780c2d740 10 client.15762 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:06:42.027 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.024+0000 7f7780c2d740 10 client.15762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.024+0000 7f7780c2d740 20 client.15762 populate_metadata read hostname 'smithi045' 2024-04-25T19:06:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.024+0000 7f7780c2d740 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559a95db13f0 0x559a95db37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.029 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.024+0000 7f7780c2d740 4 client.15762 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:42.030 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.024+0000 7f7780c2d740 1 -- 172.21.15.45:0/2280791770 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a95d078b0 con 0x559a95db13f0 2024-04-25T19:06:42.031 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.025+0000 7f7776a00640 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559a95db13f0 0x559a95db37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.026+0000 7f7776a00640 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559a95db13f0 0x559a95db37e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b575fc01d412a5e7 server_cookie=7f29ccd525e840e6 in_seq=0 out_seq=0 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.026+0000 7f7764c00640 10 client.15762 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.026+0000 7f7764c00640 1 -- 172.21.15.45:0/2280791770 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a95d078b0 con 0x559a95db13f0 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.026+0000 7f7764c00640 10 client.15762 handle_command_reply: tid=0 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.026+0000 7f7780c2d740 1 client.15762 shutdown 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 1 -- 172.21.15.45:0/2280791770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 msgr2=0x559a95d95f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 0x559a95d95f40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7760005ab0 tx=0x7f776002a480 comp rx=0 tx=0).stop 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 1 -- 172.21.15.45:0/2280791770 shutdown_connections 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559a95db13f0 0x559a95db37e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d98a00 0x559a95d96480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8e140 0x559a95d95f40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7764c00640 0 client.15762 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:42.032 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 1 --2- 172.21.15.45:0/2280791770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d95a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 1 -- 172.21.15.45:0/2280791770 >> 172.21.15.45:0/2280791770 conn(0x559a95d89d30 msgr2=0x559a95d8fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 1 -- 172.21.15.45:0/2280791770 shutdown_connections 2024-04-25T19:06:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 1 -- 172.21.15.45:0/2280791770 wait complete. 2024-04-25T19:06:42.033 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.027+0000 7f7780c2d740 20 client.15762 trim_cache size 0 max 16384 2024-04-25T19:06:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.049+0000 7f7780c2d740 1 Processor -- start 2024-04-25T19:06:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.049+0000 7f7780c2d740 1 -- start start 2024-04-25T19:06:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d89610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95d901e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d98a00 0x559a95d960d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7780c2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77781ae3c0 con 0x559a95d98a00 2024-04-25T19:06:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7780c2d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77781abc90 con 0x559a95d8e140 2024-04-25T19:06:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7780c2d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77781a9b20 con 0x559a95d8d820 2024-04-25T19:06:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7777400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d98a00 0x559a95d960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7776000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95d901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7776a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d89610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7776a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d89610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:60406/0 (socket says 172.21.15.45:60406) 2024-04-25T19:06:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7776000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95d901e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:56580/0 (socket says 172.21.15.45:56580) 2024-04-25T19:06:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.050+0000 7f7776a00640 1 -- 172.21.15.45:0/1973476445 learned_addr learned my addr 172.21.15.45:0/1973476445 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.051+0000 7f7776a00640 1 -- 172.21.15.45:0/1973476445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 msgr2=0x559a95d901e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.051+0000 7f7776a00640 1 --2- 172.21.15.45:0/1973476445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95d901e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.051+0000 7f7776a00640 1 -- 172.21.15.45:0/1973476445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d98a00 msgr2=0x559a95d960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.051+0000 7f7776a00640 1 --2- 172.21.15.45:0/1973476445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d98a00 0x559a95d960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.051+0000 7f7776a00640 1 -- 172.21.15.45:0/1973476445 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a95d9b160 con 0x559a95d8d820 2024-04-25T19:06:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.051+0000 7f7776000640 1 --2- 172.21.15.45:0/1973476445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95d901e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.051+0000 7f7776a00640 1 --2- 172.21.15.45:0/1973476445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d89610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f776809c390 tx=0x7f77680d2bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a17a3c833a4f39b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.052+0000 7f7767400640 1 -- 172.21.15.45:0/1973476445 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776801c040 con 0x559a95d8d820 2024-04-25T19:06:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.052+0000 7f7767400640 1 -- 172.21.15.45:0/1973476445 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776809cd90 con 0x559a95d8d820 2024-04-25T19:06:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.052+0000 7f7767400640 1 -- 172.21.15.45:0/1973476445 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776809a990 con 0x559a95d8d820 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.053+0000 7f7780c2d740 1 -- 172.21.15.45:0/1973476445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 msgr2=0x559a95d89610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.053+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1973476445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d89610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f776809c390 tx=0x7f77680d2bd0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.053+0000 7f7780c2d740 1 -- 172.21.15.45:0/1973476445 shutdown_connections 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.053+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1973476445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d98a00 0x559a95d960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.053+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1973476445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95d901e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.053+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1973476445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d8d820 0x559a95d89610 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.053+0000 7f7780c2d740 1 -- 172.21.15.45:0/1973476445 >> 172.21.15.45:0/1973476445 conn(0x559a95d89d30 msgr2=0x559a95d959c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.053+0000 7f7780c2d740 1 -- 172.21.15.45:0/1973476445 shutdown_connections 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.053+0000 7f7780c2d740 1 -- 172.21.15.45:0/1973476445 wait complete. 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.054+0000 7f7780c2d740 1 -- start start 2024-04-25T19:06:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.054+0000 7f7780c2d740 1 Processor -- start 2024-04-25T19:06:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.054+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8d820 0x559a95e10030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.054+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95e10570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.054+0000 7f7780c2d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d98a00 0x559a95db61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.054+0000 7f7780c2d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f77781aa170 con 0x559a95d8d820 2024-04-25T19:06:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.054+0000 7f7780c2d740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f77781ab5c0 con 0x559a95d8e140 2024-04-25T19:06:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.054+0000 7f7780c2d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77781aa820 con 0x559a95d98a00 2024-04-25T19:06:42.059 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.054+0000 7f7780c2d740 10 client.0 _ll_register_callbacks cb 0x7f7760009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95e10570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95e10570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:56592/0 (socket says 172.21.15.45:56592) 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776000640 1 -- 172.21.15.45:0/1522845416 learned_addr learned my addr 172.21.15.45:0/1522845416 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7777400640 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d98a00 0x559a95db61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776a00640 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8d820 0x559a95e10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776000640 1 -- 172.21.15.45:0/1522845416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d98a00 msgr2=0x559a95db61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776000640 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d98a00 0x559a95db61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776000640 1 -- 172.21.15.45:0/1522845416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8d820 msgr2=0x559a95e10030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776000640 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8d820 0x559a95e10030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776000640 1 -- 172.21.15.45:0/1522845416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a95db68a0 con 0x559a95d8e140 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776a00640 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8d820 0x559a95e10030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7776000640 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95e10570 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f776002aa70 tx=0x7f7760009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6daef56dbc2bb2e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.055+0000 7f7764c00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:06:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.056+0000 7f7764c00640 1 -- 172.21.15.45:0/1522845416 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7760053040 con 0x559a95d8e140 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.056+0000 7f7764c00640 1 -- 172.21.15.45:0/1522845416 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f776004a040 con 0x559a95d8e140 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.056+0000 7f7780c2d740 1 -- 172.21.15.45:0/1522845416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776809cd90 con 0x559a95d8e140 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.056+0000 7f7764c00640 1 -- 172.21.15.45:0/1522845416 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f776003cbb0 con 0x559a95d8e140 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.056+0000 7f7780c2d740 10 client.25916 fetch_fsmap learned FSMap version 32 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.056+0000 7f7780c2d740 1 -- 172.21.15.45:0/1522845416 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a95e17800 con 0x559a95d8e140 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.056+0000 7f7764c00640 1 -- 172.21.15.45:0/1522845416 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776809cd90 con 0x559a95d8e140 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.057+0000 7f7764c00640 1 -- 172.21.15.45:0/1522845416 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f77600038e0 con 0x559a95d8e140 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.057+0000 7f7780c2d740 10 client.25916 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.057+0000 7f7780c2d740 10 client.25916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.057+0000 7f7780c2d740 20 client.25916 populate_metadata read hostname 'smithi045' 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.057+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559a95e17fa0 0x559a95e1a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.057+0000 7f7780c2d740 4 client.25916 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.057+0000 7f7780c2d740 1 -- 172.21.15.45:0/1522845416 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a95d01320 con 0x559a95e17fa0 2024-04-25T19:06:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.057+0000 7f7776a00640 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559a95e17fa0 0x559a95e1a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.057+0000 7f7764c00640 10 client.25916 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.058+0000 7f7776a00640 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559a95e17fa0 0x559a95e1a3c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fcc66a2c99c7cfb server_cookie=4dc006d03cffbe79 in_seq=0 out_seq=0 2024-04-25T19:06:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.074+0000 7f7764c00640 1 -- 172.21.15.45:0/1522845416 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559a95d01320 con 0x559a95e17fa0 2024-04-25T19:06:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.074+0000 7f7764c00640 10 client.25916 handle_command_reply: tid=0 2024-04-25T19:06:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.074+0000 7f7780c2d740 1 client.25916 shutdown 2024-04-25T19:06:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 1 -- 172.21.15.45:0/1522845416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 msgr2=0x559a95e10570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95e10570 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f776002aa70 tx=0x7f7760009640 comp rx=0 tx=0).stop 2024-04-25T19:06:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 1 -- 172.21.15.45:0/1522845416 shutdown_connections 2024-04-25T19:06:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559a95e17fa0 0x559a95e1a3c0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559a95d98a00 0x559a95db61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559a95d8e140 0x559a95e10570 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 1 --2- 172.21.15.45:0/1522845416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559a95d8d820 0x559a95e10030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7764c00640 0 client.25916 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 1 -- 172.21.15.45:0/1522845416 >> 172.21.15.45:0/1522845416 conn(0x559a95d89d30 msgr2=0x559a95da4230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 1 -- 172.21.15.45:0/1522845416 shutdown_connections 2024-04-25T19:06:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 1 -- 172.21.15.45:0/1522845416 wait complete. 2024-04-25T19:06:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.075+0000 7f7780c2d740 20 client.25916 trim_cache size 0 max 16384 2024-04-25T19:06:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:06:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:06:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:06:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:06:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 -- 172.21.15.45:0/362139415 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f775c0a3d90 msgr2=0x7f775c0a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f775c0a3d90 0x7f775c0a6250 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7770005f40 tx=0x7f7770009500 comp rx=0 tx=0).stop 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 -- 172.21.15.45:0/362139415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7778074320 msgr2=0x7f77781a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7778074320 0x7f77781a06d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f776800e9f0 tx=0x7f776800eeb0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 -- 172.21.15.45:0/362139415 shutdown_connections 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f775c0a3d90 0x7f775c0a6250 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f777810e800 0x7f777819a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7778074c40 0x7f77781a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 --2- 172.21.15.45:0/362139415 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7778074320 0x7f77781a06d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 -- 172.21.15.45:0/362139415 >> 172.21.15.45:0/362139415 conn(0x7f77780fda30 msgr2=0x7f77780fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 -- 172.21.15.45:0/362139415 shutdown_connections 2024-04-25T19:06:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:42.076+0000 7f777de00640 1 -- 172.21.15.45:0/362139415 wait complete. 2024-04-25T19:06:42.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:42 smithi062 ceph-mon[32844]: pgmap v227: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.4 MiB/s wr, 465 op/s 2024-04-25T19:06:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:42 smithi079 ceph-mon[33002]: pgmap v227: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.4 MiB/s wr, 465 op/s 2024-04-25T19:06:42.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:42 smithi045 ceph-mon[27776]: pgmap v227: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.4 MiB/s wr, 465 op/s 2024-04-25T19:06:44.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:44 smithi062 ceph-mon[32844]: pgmap v228: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 999 KiB/s wr, 312 op/s 2024-04-25T19:06:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:44 smithi079 ceph-mon[33002]: pgmap v228: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 999 KiB/s wr, 312 op/s 2024-04-25T19:06:44.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:44 smithi045 ceph-mon[27776]: pgmap v228: 97 pgs: 97 active+clean; 87 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 999 KiB/s wr, 312 op/s 2024-04-25T19:06:46.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:46 smithi062 ceph-mon[32844]: pgmap v229: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.4 MiB/s wr, 447 op/s 2024-04-25T19:06:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:46 smithi079 ceph-mon[33002]: pgmap v229: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.4 MiB/s wr, 447 op/s 2024-04-25T19:06:46.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:46 smithi045 ceph-mon[27776]: pgmap v229: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 1.4 MiB/s wr, 447 op/s 2024-04-25T19:06:48.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:48 smithi062 ceph-mon[32844]: pgmap v230: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 298 op/s 2024-04-25T19:06:48.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:48 smithi079 ceph-mon[33002]: pgmap v230: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 298 op/s 2024-04-25T19:06:48.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:48 smithi045 ceph-mon[27776]: pgmap v230: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 298 op/s 2024-04-25T19:06:50.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:50 smithi062 ceph-mon[32844]: pgmap v231: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 298 op/s 2024-04-25T19:06:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:50 smithi079 ceph-mon[33002]: pgmap v231: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 298 op/s 2024-04-25T19:06:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:50 smithi045 ceph-mon[27776]: pgmap v231: 97 pgs: 97 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 298 op/s 2024-04-25T19:06:52.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:52 smithi062 ceph-mon[32844]: pgmap v232: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 449 op/s 2024-04-25T19:06:52.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:52 smithi079 ceph-mon[33002]: pgmap v232: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 449 op/s 2024-04-25T19:06:52.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:52 smithi045 ceph-mon[27776]: pgmap v232: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.4 MiB/s wr, 449 op/s 2024-04-25T19:06:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:54 smithi062 ceph-mon[32844]: pgmap v233: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 876 KiB/s wr, 286 op/s 2024-04-25T19:06:54.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:54 smithi079 ceph-mon[33002]: pgmap v233: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 876 KiB/s wr, 286 op/s 2024-04-25T19:06:54.466 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:54 smithi045 ceph-mon[27776]: pgmap v233: 97 pgs: 97 active+clean; 80 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 876 KiB/s wr, 286 op/s 2024-04-25T19:06:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.239+0000 7f6603a00640 1 -- 172.21.15.45:0/2063646224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65fc104d90 msgr2=0x7f65fc108dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.239+0000 7f6603a00640 1 --2- 172.21.15.45:0/2063646224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65fc104d90 0x7f65fc108dd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65e8009910 tx=0x7f65e802a3f0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.240+0000 7f6603a00640 1 -- 172.21.15.45:0/2063646224 shutdown_connections 2024-04-25T19:06:55.241 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.240+0000 7f6603a00640 1 --2- 172.21.15.45:0/2063646224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65fc1094b0 0x7f65fc111080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.240+0000 7f6603a00640 1 --2- 172.21.15.45:0/2063646224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65fc104d90 0x7f65fc108dd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.240+0000 7f6603a00640 1 --2- 172.21.15.45:0/2063646224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65fc1043e0 0x7f65fc1047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.240+0000 7f6603a00640 1 -- 172.21.15.45:0/2063646224 >> 172.21.15.45:0/2063646224 conn(0x7f65fc0ffc00 msgr2=0x7f65fc102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.241+0000 7f6603a00640 1 -- 172.21.15.45:0/2063646224 shutdown_connections 2024-04-25T19:06:55.242 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.241+0000 7f6603a00640 1 -- 172.21.15.45:0/2063646224 wait complete. 2024-04-25T19:06:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.242+0000 7f6603a00640 1 Processor -- start 2024-04-25T19:06:55.244 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f6603a00640 1 -- start start 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f6603a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65fc1043e0 0x7f65fc111010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f6603a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65fc1094b0 0x7f65fc10e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f6603a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65fc10e7a0 0x7f65fc1ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f6603a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc1143d0 con 0x7f65fc10e7a0 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f6603a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc114250 con 0x7f65fc1043e0 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f6603a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc114550 con 0x7f65fc1094b0 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f65fbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65fc1094b0 0x7f65fc10e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f65fbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65fc1094b0 0x7f65fc10e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:47394/0 (socket says 172.21.15.45:47394) 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f6601000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65fc1043e0 0x7f65fc111010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f65fbe00640 1 -- 172.21.15.45:0/914893510 learned_addr learned my addr 172.21.15.45:0/914893510 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.243+0000 7f6601a00640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65fc10e7a0 0x7f65fc1ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.246 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.244+0000 7f6601000640 1 -- 172.21.15.45:0/914893510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65fc1094b0 msgr2=0x7f65fc10e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.244+0000 7f6601000640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65fc1094b0 0x7f65fc10e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.244+0000 7f6601000640 1 -- 172.21.15.45:0/914893510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65fc10e7a0 msgr2=0x7f65fc1ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.244+0000 7f6601000640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65fc10e7a0 0x7f65fc1ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.244+0000 7f6601000640 1 -- 172.21.15.45:0/914893510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65fc1ab650 con 0x7f65fc1043e0 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.244+0000 7f65fbe00640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65fc1094b0 0x7f65fc10e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.244+0000 7f6601000640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65fc1043e0 0x7f65fc111010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f65f000e9f0 tx=0x7f65f000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.244+0000 7f65f9600640 1 -- 172.21.15.45:0/914893510 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f000cd80 con 0x7f65fc1043e0 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.244+0000 7f65f9600640 1 -- 172.21.15.45:0/914893510 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f00035f0 con 0x7f65fc1043e0 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.244+0000 7f65f9600640 1 -- 172.21.15.45:0/914893510 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0010670 con 0x7f65fc1043e0 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.245+0000 7f6603a00640 1 -- 172.21.15.45:0/914893510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65fc1ab820 con 0x7f65fc1043e0 2024-04-25T19:06:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.245+0000 7f6603a00640 1 -- 172.21.15.45:0/914893510 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65fc1abc80 con 0x7f65fc1043e0 2024-04-25T19:06:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.248+0000 7f65f9600640 1 -- 172.21.15.45:0/914893510 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f65f00037a0 con 0x7f65fc1043e0 2024-04-25T19:06:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.248+0000 7f6606884740 1 Processor -- start 2024-04-25T19:06:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.250+0000 7f6606884740 1 -- start start 2024-04-25T19:06:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.252+0000 7f6606884740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862a820 0x560b2862ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.252+0000 7f6606884740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862b140 0x560b286354c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.252+0000 7f6606884740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b28637df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.252+0000 7f6606884740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560b28590900 con 0x560b2862a820 2024-04-25T19:06:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.252+0000 7f6606884740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560b28536ef0 con 0x560b2862b140 2024-04-25T19:06:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.252+0000 7f6606884740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560b2858c8c0 con 0x560b28635a00 2024-04-25T19:06:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f65f9600640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65e40a3de0 0x7f65e40a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f6601000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862b140 0x560b286354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f6601000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862b140 0x560b286354c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48362/0 (socket says 172.21.15.45:48362) 2024-04-25T19:06:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f6601000640 1 -- 172.21.15.45:0/445866808 learned_addr learned my addr 172.21.15.45:0/445866808 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f6601a00640 1 --2- 172.21.15.45:0/445866808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862a820 0x560b2862ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f65fbe00640 1 --2- 172.21.15.45:0/445866808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b28637df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f65f9600640 1 -- 172.21.15.45:0/914893510 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f65f00d0ad0 con 0x7f65fc1043e0 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f6601000640 1 -- 172.21.15.45:0/445866808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 msgr2=0x560b28637df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f6601000640 1 --2- 172.21.15.45:0/445866808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b28637df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f6601000640 1 -- 172.21.15.45:0/445866808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862a820 msgr2=0x560b2862ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f6601000640 1 --2- 172.21.15.45:0/445866808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862a820 0x560b2862ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.253+0000 7f6601000640 1 -- 172.21.15.45:0/445866808 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b282a3210 con 0x560b2862b140 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.254+0000 7f6601a00640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65e40a3de0 0x7f65e40a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.254+0000 7f6601000640 1 --2- 172.21.15.45:0/445866808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862b140 0x560b286354c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f65f00a5120 tx=0x7f65f00a5650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d48562432fda5c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.254+0000 7f65ee000640 1 -- 172.21.15.45:0/445866808 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f00a0d90 con 0x560b2862b140 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.254+0000 7f65ee000640 1 -- 172.21.15.45:0/445866808 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f009ace0 con 0x560b2862b140 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.254+0000 7f65ee000640 1 -- 172.21.15.45:0/445866808 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f00ce410 con 0x560b2862b140 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.255+0000 7f6606884740 1 -- 172.21.15.45:0/445866808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862b140 msgr2=0x560b286354c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.255+0000 7f6601a00640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65e40a3de0 0x7f65e40a62a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f65f4005fd0 tx=0x7f65f4014040 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.255+0000 7f6606884740 1 --2- 172.21.15.45:0/445866808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862b140 0x560b286354c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f65f00a5120 tx=0x7f65f00a5650 comp rx=0 tx=0).stop 2024-04-25T19:06:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.255+0000 7f6606884740 1 -- 172.21.15.45:0/445866808 shutdown_connections 2024-04-25T19:06:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.255+0000 7f6606884740 1 --2- 172.21.15.45:0/445866808 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b28637df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.255+0000 7f6606884740 1 --2- 172.21.15.45:0/445866808 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862b140 0x560b286354c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.255+0000 7f6606884740 1 --2- 172.21.15.45:0/445866808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862a820 0x560b2862ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.255+0000 7f6606884740 1 -- 172.21.15.45:0/445866808 >> 172.21.15.45:0/445866808 conn(0x560b28626d30 msgr2=0x560b28627110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.256+0000 7f6606884740 1 -- 172.21.15.45:0/445866808 shutdown_connections 2024-04-25T19:06:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.256+0000 7f6606884740 1 -- 172.21.15.45:0/445866808 wait complete. 2024-04-25T19:06:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.256+0000 7f6606884740 1 -- start start 2024-04-25T19:06:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.257+0000 7f6606884740 1 Processor -- start 2024-04-25T19:06:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6606884740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862a820 0x560b28632a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6606884740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b2862b140 0x560b28632f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6606884740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b28635a00 0x560b28633480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6606884740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560b28638630 con 0x560b2862a820 2024-04-25T19:06:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6606884740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560b2854a310 con 0x560b28635a00 2024-04-25T19:06:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6606884740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560b286387b0 con 0x560b2862b140 2024-04-25T19:06:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6606884740 10 client.0 _ll_register_callbacks cb 0x560b28518cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6601a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b28635a00 0x560b28633480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f65fbe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b2862b140 0x560b28632f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6601a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b28635a00 0x560b28633480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48372/0 (socket says 172.21.15.45:48372) 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6601a00640 1 -- 172.21.15.45:0/3443920344 learned_addr learned my addr 172.21.15.45:0/3443920344 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6601000640 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862a820 0x560b28632a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6601a00640 1 -- 172.21.15.45:0/3443920344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b2862b140 msgr2=0x560b28632f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6601a00640 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b2862b140 0x560b28632f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6601a00640 1 -- 172.21.15.45:0/3443920344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862a820 msgr2=0x560b28632a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.258+0000 7f6601a00640 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862a820 0x560b28632a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.259+0000 7f6601a00640 1 -- 172.21.15.45:0/3443920344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b2836f620 con 0x560b28635a00 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.259+0000 7f65fbe00640 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b2862b140 0x560b28632f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.259+0000 7f6601a00640 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b28635a00 0x560b28633480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f65f4005c40 tx=0x7f65f4009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a4de1ea95200292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.259+0000 7f65e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.259+0000 7f65e2a00640 1 -- 172.21.15.45:0/3443920344 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f401c890 con 0x560b28635a00 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.259+0000 7f6606884740 1 -- 172.21.15.45:0/3443920344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65f009ace0 con 0x560b28635a00 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.259+0000 7f65e2a00640 1 -- 172.21.15.45:0/3443920344 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f65f401ca30 con 0x560b28635a00 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f65e2a00640 1 -- 172.21.15.45:0/3443920344 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65f401cc30 con 0x560b28635a00 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f6606884740 10 client.25928 fetch_fsmap learned FSMap version 32 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f6606884740 1 -- 172.21.15.45:0/3443920344 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560b286449a0 con 0x560b28635a00 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f65e2a00640 1 -- 172.21.15.45:0/3443920344 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f009ace0 con 0x560b28635a00 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f65e2a00640 1 -- 172.21.15.45:0/3443920344 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f65f4025020 con 0x560b28635a00 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f6606884740 10 client.25928 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f6606884740 10 client.25928 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f6606884740 20 client.25928 populate_metadata read hostname 'smithi045' 2024-04-25T19:06:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f6606884740 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560b28645e60 0x560b28648250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f6606884740 4 client.25928 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:06:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f6606884740 1 -- 172.21.15.45:0/3443920344 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b285a48b0 con 0x560b28645e60 2024-04-25T19:06:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.260+0000 7f6601000640 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560b28645e60 0x560b28648250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.261+0000 7f6601000640 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560b28645e60 0x560b28648250 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82214a3ca69ebc1 server_cookie=473876a4fb0a0a59 in_seq=0 out_seq=0 2024-04-25T19:06:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.261+0000 7f65e2a00640 10 client.25928 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.261+0000 7f65e2a00640 1 -- 172.21.15.45:0/3443920344 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b285a48b0 con 0x560b28645e60 2024-04-25T19:06:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.261+0000 7f65e2a00640 10 client.25928 handle_command_reply: tid=0 2024-04-25T19:06:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 client.25928 shutdown 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 -- 172.21.15.45:0/3443920344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b28635a00 msgr2=0x560b28633480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b28635a00 0x560b28633480 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f65f4005c40 tx=0x7f65f4009500 comp rx=0 tx=0).stop 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 -- 172.21.15.45:0/3443920344 shutdown_connections 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560b28645e60 0x560b28648250 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b28635a00 0x560b28633480 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b2862b140 0x560b28632f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 --2- 172.21.15.45:0/3443920344 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862a820 0x560b28632a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f65e2a00640 0 client.25928 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 -- 172.21.15.45:0/3443920344 >> 172.21.15.45:0/3443920344 conn(0x560b28626d30 msgr2=0x560b2862cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 -- 172.21.15.45:0/3443920344 shutdown_connections 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 1 -- 172.21.15.45:0/3443920344 wait complete. 2024-04-25T19:06:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.262+0000 7f6606884740 20 client.25928 trim_cache size 0 max 16384 2024-04-25T19:06:55.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.285+0000 7f6606884740 1 Processor -- start 2024-04-25T19:06:55.286 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.285+0000 7f6606884740 1 -- start start 2024-04-25T19:06:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.285+0000 7f6606884740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b286414f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.285+0000 7f6606884740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862b140 0x560b2864db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.286+0000 7f6606884740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b2864e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.286+0000 7f6606884740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc1b6bf0 con 0x560b2862b140 2024-04-25T19:06:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.286+0000 7f6606884740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc1b44c0 con 0x560b2862a820 2024-04-25T19:06:55.287 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.286+0000 7f6606884740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc1b2330 con 0x560b28635a00 2024-04-25T19:06:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.286+0000 7f6601000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b286414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.286+0000 7f6601000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b286414f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48384/0 (socket says 172.21.15.45:48384) 2024-04-25T19:06:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.286+0000 7f6601000640 1 -- 172.21.15.45:0/2260150343 learned_addr learned my addr 172.21.15.45:0/2260150343 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.286+0000 7f6601a00640 1 --2- 172.21.15.45:0/2260150343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b2864e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.286+0000 7f65fbe00640 1 --2- 172.21.15.45:0/2260150343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862b140 0x560b2864db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.287+0000 7f6601000640 1 -- 172.21.15.45:0/2260150343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 msgr2=0x560b2864e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.287+0000 7f6601000640 1 --2- 172.21.15.45:0/2260150343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b2864e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.287+0000 7f6601000640 1 -- 172.21.15.45:0/2260150343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862b140 msgr2=0x560b2864db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.287+0000 7f6601000640 1 --2- 172.21.15.45:0/2260150343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862b140 0x560b2864db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.287+0000 7f6601000640 1 -- 172.21.15.45:0/2260150343 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b28638160 con 0x560b2862a820 2024-04-25T19:06:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.288+0000 7f6601000640 1 --2- 172.21.15.45:0/2260150343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b286414f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f65f0000c00 tx=0x7f65f0005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3ac22d045faf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.288+0000 7f65ed600640 1 -- 172.21.15.45:0/2260150343 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f009ace0 con 0x560b2862a820 2024-04-25T19:06:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.288+0000 7f65ed600640 1 -- 172.21.15.45:0/2260150343 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f0003ac0 con 0x560b2862a820 2024-04-25T19:06:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.288+0000 7f65ed600640 1 -- 172.21.15.45:0/2260150343 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0009480 con 0x560b2862a820 2024-04-25T19:06:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.290+0000 7f6606884740 1 -- 172.21.15.45:0/2260150343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 msgr2=0x560b286414f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.290+0000 7f6606884740 1 --2- 172.21.15.45:0/2260150343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b286414f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f65f0000c00 tx=0x7f65f0005bb0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.290+0000 7f6606884740 1 -- 172.21.15.45:0/2260150343 shutdown_connections 2024-04-25T19:06:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.290+0000 7f6606884740 1 --2- 172.21.15.45:0/2260150343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b2864e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.290+0000 7f6606884740 1 --2- 172.21.15.45:0/2260150343 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862b140 0x560b2864db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.290+0000 7f6606884740 1 --2- 172.21.15.45:0/2260150343 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b286414f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.290+0000 7f6606884740 1 -- 172.21.15.45:0/2260150343 >> 172.21.15.45:0/2260150343 conn(0x560b28626d30 msgr2=0x560b28627110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.290+0000 7f6606884740 1 -- 172.21.15.45:0/2260150343 shutdown_connections 2024-04-25T19:06:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.290+0000 7f6606884740 1 -- 172.21.15.45:0/2260150343 wait complete. 2024-04-25T19:06:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.291+0000 7f6606884740 1 -- start start 2024-04-25T19:06:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.291+0000 7f6606884740 1 Processor -- start 2024-04-25T19:06:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6606884740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b2863b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6606884740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862b140 0x560b2863bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6606884740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b2863c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6606884740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc1b2980 con 0x560b2862b140 2024-04-25T19:06:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6606884740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc1b3df0 con 0x560b2862a820 2024-04-25T19:06:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6606884740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65fc1b3050 con 0x560b28635a00 2024-04-25T19:06:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6606884740 10 client.0 _ll_register_callbacks cb 0x7f65f40104e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:06:55.295 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6601000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b2863b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6601000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b2863b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48392/0 (socket says 172.21.15.45:48392) 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6601000640 1 -- 172.21.15.45:0/3860424049 learned_addr learned my addr 172.21.15.45:0/3860424049 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6601a00640 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b2863c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f65fbe00640 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862b140 0x560b2863bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6601000640 1 -- 172.21.15.45:0/3860424049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 msgr2=0x560b2863c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6601000640 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b2863c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6601000640 1 -- 172.21.15.45:0/3860424049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862b140 msgr2=0x560b2863bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6601000640 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862b140 0x560b2863bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.292+0000 7f6601000640 1 -- 172.21.15.45:0/3860424049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b286ab660 con 0x560b2862a820 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f6601000640 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b2863b6e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f65f0005c20 tx=0x7f65f0098d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b36802641551c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f65e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f65e2a00640 1 -- 172.21.15.45:0/3860424049 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65f0009460 con 0x560b2862a820 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f65e2a00640 1 -- 172.21.15.45:0/3860424049 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f65f00ce410 con 0x560b2862a820 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f6606884740 1 -- 172.21.15.45:0/3860424049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65f0003ac0 con 0x560b2862a820 2024-04-25T19:06:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f65e2a00640 1 -- 172.21.15.45:0/3860424049 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f65f009cc40 con 0x560b2862a820 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f6606884740 10 client.25934 fetch_fsmap learned FSMap version 32 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f6606884740 1 -- 172.21.15.45:0/3860424049 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560b286ac970 con 0x560b2862a820 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f65e2a00640 1 -- 172.21.15.45:0/3860424049 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65f0003ac0 con 0x560b2862a820 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f65e2a00640 1 -- 172.21.15.45:0/3860424049 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f65f00ba070 con 0x560b2862a820 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f6606884740 10 client.25934 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f6606884740 10 client.25934 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.293+0000 7f6606884740 20 client.25934 populate_metadata read hostname 'smithi045' 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.294+0000 7f6606884740 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560b286acba0 0x560b286aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.294+0000 7f6606884740 4 client.25934 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.294+0000 7f6606884740 1 -- 172.21.15.45:0/3860424049 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b2859e320 con 0x560b286acba0 2024-04-25T19:06:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.294+0000 7f65fbe00640 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560b286acba0 0x560b286aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:06:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.294+0000 7f65fbe00640 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560b286acba0 0x560b286aef90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7bde26027bd4900 server_cookie=35c653718807edc4 in_seq=0 out_seq=0 2024-04-25T19:06:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.294+0000 7f65e2a00640 10 client.25934 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.307+0000 7f65e2a00640 1 -- 172.21.15.45:0/3860424049 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b2859e320 con 0x560b286acba0 2024-04-25T19:06:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.307+0000 7f65e2a00640 10 client.25934 handle_command_reply: tid=0 2024-04-25T19:06:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.307+0000 7f6606884740 1 client.25934 shutdown 2024-04-25T19:06:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 1 -- 172.21.15.45:0/3860424049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 msgr2=0x560b2863b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b2863b6e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f65f0005c20 tx=0x7f65f0098d00 comp rx=0 tx=0).stop 2024-04-25T19:06:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 1 -- 172.21.15.45:0/3860424049 shutdown_connections 2024-04-25T19:06:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560b286acba0 0x560b286aef90 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560b28635a00 0x560b2863c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560b2862b140 0x560b2863bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 1 --2- 172.21.15.45:0/3860424049 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560b2862a820 0x560b2863b6e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 1 -- 172.21.15.45:0/3860424049 >> 172.21.15.45:0/3860424049 conn(0x560b28626d30 msgr2=0x560b28625600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f65e2a00640 0 client.25934 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:06:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 1 -- 172.21.15.45:0/3860424049 shutdown_connections 2024-04-25T19:06:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 1 -- 172.21.15.45:0/3860424049 wait complete. 2024-04-25T19:06:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.308+0000 7f6606884740 20 client.25934 trim_cache size 0 max 16384 2024-04-25T19:06:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:06:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:06:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:06:55.311 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.309+0000 7f6603a00640 1 -- 172.21.15.45:0/914893510 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65e40a3de0 msgr2=0x7f65e40a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.309+0000 7f6603a00640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65e40a3de0 0x7f65e40a62a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f65f4005fd0 tx=0x7f65f4014040 comp rx=0 tx=0).stop 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.309+0000 7f6603a00640 1 -- 172.21.15.45:0/914893510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65fc1043e0 msgr2=0x7f65fc111010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.309+0000 7f6603a00640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65fc1043e0 0x7f65fc111010 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f65f000e9f0 tx=0x7f65f000eeb0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.310+0000 7f6603a00640 1 -- 172.21.15.45:0/914893510 shutdown_connections 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.310+0000 7f6603a00640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65e40a3de0 0x7f65e40a62a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.310+0000 7f6603a00640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65fc10e7a0 0x7f65fc1ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.310+0000 7f6603a00640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65fc1094b0 0x7f65fc10e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.310+0000 7f6603a00640 1 --2- 172.21.15.45:0/914893510 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65fc1043e0 0x7f65fc111010 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.310+0000 7f6603a00640 1 -- 172.21.15.45:0/914893510 >> 172.21.15.45:0/914893510 conn(0x7f65fc0ffc00 msgr2=0x7f65fc100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.310+0000 7f6603a00640 1 -- 172.21.15.45:0/914893510 shutdown_connections 2024-04-25T19:06:55.312 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:06:55.310+0000 7f6603a00640 1 -- 172.21.15.45:0/914893510 wait complete. 2024-04-25T19:06:56.366 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:56 smithi062 ceph-mon[32844]: pgmap v234: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-04-25T19:06:56.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:56 smithi079 ceph-mon[33002]: pgmap v234: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-04-25T19:06:56.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:56 smithi045 ceph-mon[27776]: pgmap v234: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.2 MiB/s wr, 435 op/s 2024-04-25T19:06:58.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:06:58 smithi062 ceph-mon[32844]: pgmap v235: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 778 KiB/s wr, 299 op/s 2024-04-25T19:06:58.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:06:58 smithi079 ceph-mon[33002]: pgmap v235: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 778 KiB/s wr, 299 op/s 2024-04-25T19:06:58.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:06:58 smithi045 ceph-mon[27776]: pgmap v235: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 778 KiB/s wr, 299 op/s 2024-04-25T19:07:00.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:00 smithi062 ceph-mon[32844]: pgmap v236: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 778 KiB/s wr, 299 op/s 2024-04-25T19:07:00.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:00 smithi079 ceph-mon[33002]: pgmap v236: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 778 KiB/s wr, 299 op/s 2024-04-25T19:07:00.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:00 smithi045 ceph-mon[27776]: pgmap v236: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 778 KiB/s wr, 299 op/s 2024-04-25T19:07:02.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:02 smithi062 ceph-mon[32844]: pgmap v237: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.3 MiB/s wr, 462 op/s 2024-04-25T19:07:02.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:02 smithi062 ceph-mon[32844]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T19:07:02.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:02 smithi079 ceph-mon[33002]: pgmap v237: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.3 MiB/s wr, 462 op/s 2024-04-25T19:07:02.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:02 smithi079 ceph-mon[33002]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T19:07:02.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:02 smithi045 ceph-mon[27776]: pgmap v237: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.3 MiB/s wr, 462 op/s 2024-04-25T19:07:02.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:02 smithi045 ceph-mon[27776]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T19:07:04.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:04 smithi062 ceph-mon[32844]: pgmap v239: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.0 MiB/s wr, 373 op/s 2024-04-25T19:07:04.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:04 smithi079 ceph-mon[33002]: pgmap v239: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.0 MiB/s wr, 373 op/s 2024-04-25T19:07:04.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:04 smithi045 ceph-mon[27776]: pgmap v239: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.0 MiB/s wr, 373 op/s 2024-04-25T19:07:05.400 INFO:tasks.workunit.client.0.smithi045.stdout:copying to largest ever: 0x63ef9ee 2024-04-25T19:07:06.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:06 smithi062 ceph-mon[32844]: pgmap v240: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-25T19:07:06.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:06 smithi079 ceph-mon[33002]: pgmap v240: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-25T19:07:06.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:06 smithi045 ceph-mon[27776]: pgmap v240: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.2 MiB/s wr, 396 op/s 2024-04-25T19:07:07.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:07 smithi079 ceph-mon[33002]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T19:07:07.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:07 smithi079 ceph-mon[33002]: pgmap v242: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 495 op/s 2024-04-25T19:07:07.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:07 smithi045 ceph-mon[27776]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T19:07:07.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:07 smithi045 ceph-mon[27776]: pgmap v242: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 495 op/s 2024-04-25T19:07:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:07 smithi062 ceph-mon[32844]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T19:07:07.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:07 smithi062 ceph-mon[32844]: pgmap v242: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.5 MiB/s wr, 495 op/s 2024-04-25T19:07:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.467+0000 7f4e31200640 1 -- 172.21.15.45:0/3948974224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e2c0742f0 msgr2=0x7f4e2c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.467+0000 7f4e31200640 1 --2- 172.21.15.45:0/3948974224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e2c0742f0 0x7f4e2c0746d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009860 tx=0x7f4e1c02a3d0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.467+0000 7f4e31200640 1 -- 172.21.15.45:0/3948974224 shutdown_connections 2024-04-25T19:07:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.467+0000 7f4e31200640 1 --2- 172.21.15.45:0/3948974224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c112d80 0x7f4e2c115170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.467+0000 7f4e31200640 1 --2- 172.21.15.45:0/3948974224 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e2c074c10 0x7f4e2c112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.467+0000 7f4e31200640 1 --2- 172.21.15.45:0/3948974224 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e2c0742f0 0x7f4e2c0746d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.467+0000 7f4e31200640 1 -- 172.21.15.45:0/3948974224 >> 172.21.15.45:0/3948974224 conn(0x7f4e2c0fd860 msgr2=0x7f4e2c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.469+0000 7f4e31200640 1 -- 172.21.15.45:0/3948974224 shutdown_connections 2024-04-25T19:07:08.470 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.469+0000 7f4e31200640 1 -- 172.21.15.45:0/3948974224 wait complete. 2024-04-25T19:07:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.471+0000 7f4e31200640 1 Processor -- start 2024-04-25T19:07:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.471+0000 7f4e31200640 1 -- start start 2024-04-25T19:07:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e31200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e2c0742f0 0x7f4e2c1a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e31200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c074c10 0x7f4e2c1a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e31200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e2c112d80 0x7f4e2c19ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e31200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c118670 con 0x7f4e2c074c10 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e31200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1184f0 con 0x7f4e2c112d80 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e31200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1187f0 con 0x7f4e2c0742f0 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e2c0742f0 0x7f4e2c1a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e2be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e2c112d80 0x7f4e2c19ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e2c0742f0 0x7f4e2c1a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39842/0 (socket says 172.21.15.45:39842) 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e2b400640 1 -- 172.21.15.45:0/1665837779 learned_addr learned my addr 172.21.15.45:0/1665837779 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.472+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c074c10 0x7f4e2c1a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.473+0000 7f4e2be00640 1 -- 172.21.15.45:0/1665837779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e2c0742f0 msgr2=0x7f4e2c1a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.473+0000 7f4e2be00640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e2c0742f0 0x7f4e2c1a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.473+0000 7f4e2be00640 1 -- 172.21.15.45:0/1665837779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c074c10 msgr2=0x7f4e2c1a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.473+0000 7f4e2be00640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c074c10 0x7f4e2c1a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.473+0000 7f4e2be00640 1 -- 172.21.15.45:0/1665837779 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c19f1d0 con 0x7f4e2c112d80 2024-04-25T19:07:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.473+0000 7f4e2be00640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e2c112d80 0x7f4e2c19ea50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005fd0 tx=0x7f4e2400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.473+0000 7f4e23e00640 1 -- 172.21.15.45:0/1665837779 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24004d20 con 0x7f4e2c112d80 2024-04-25T19:07:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.473+0000 7f4e23e00640 1 -- 172.21.15.45:0/1665837779 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e24005020 con 0x7f4e2c112d80 2024-04-25T19:07:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.473+0000 7f4e31200640 1 -- 172.21.15.45:0/1665837779 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c19f4c0 con 0x7f4e2c112d80 2024-04-25T19:07:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.474+0000 7f4e31200640 1 -- 172.21.15.45:0/1665837779 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1ab710 con 0x7f4e2c112d80 2024-04-25T19:07:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.474+0000 7f4e23e00640 1 -- 172.21.15.45:0/1665837779 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e240116b0 con 0x7f4e2c112d80 2024-04-25T19:07:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.477+0000 7f4e23e00640 1 -- 172.21.15.45:0/1665837779 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4e2400ca60 con 0x7f4e2c112d80 2024-04-25T19:07:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.477+0000 7f4e35622740 1 Processor -- start 2024-04-25T19:07:08.478 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.477+0000 7f4e35622740 1 -- start start 2024-04-25T19:07:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e35622740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a6ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e35622740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a75650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e35622740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 0x560137a77f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e35622740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5601379d0900 con 0x560137a6a9b0 2024-04-25T19:07:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e35622740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560137976ef0 con 0x560137a75b90 2024-04-25T19:07:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e35622740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5601379cc8c0 con 0x560137a6b2d0 2024-04-25T19:07:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a75650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a75650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39850/0 (socket says 172.21.15.45:39850) 2024-04-25T19:07:08.480 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e2b400640 1 -- 172.21.15.45:0/502200853 learned_addr learned my addr 172.21.15.45:0/502200853 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/502200853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a6ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.478+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/502200853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 0x560137a77f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e2b400640 1 -- 172.21.15.45:0/502200853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 msgr2=0x560137a77f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e2b400640 1 --2- 172.21.15.45:0/502200853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 0x560137a77f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e2b400640 1 -- 172.21.15.45:0/502200853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 msgr2=0x560137a6ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e2b400640 1 --2- 172.21.15.45:0/502200853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a6ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e2b400640 1 -- 172.21.15.45:0/502200853 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601376e3210 con 0x560137a6b2d0 2024-04-25T19:07:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/502200853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 0x560137a77f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:07:08.481 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e2b400640 1 --2- 172.21.15.45:0/502200853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a75650 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009980 tx=0x7f4e1c005b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4c8f9b8f9e490b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e23e00640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4e100a3de0 0x7f4e100a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.484 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e20c00640 1 -- 172.21.15.45:0/502200853 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c030980 con 0x560137a6b2d0 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e20c00640 1 -- 172.21.15.45:0/502200853 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e1c041070 con 0x560137a6b2d0 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.479+0000 7f4e20c00640 1 -- 172.21.15.45:0/502200853 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c004900 con 0x560137a6b2d0 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.480+0000 7f4e23e00640 1 -- 172.21.15.45:0/1665837779 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f4e240d0700 con 0x7f4e2c112d80 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.480+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4e100a3de0 0x7f4e100a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.480+0000 7f4e35622740 1 -- 172.21.15.45:0/502200853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 msgr2=0x560137a75650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.480+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4e100a3de0 0x7f4e100a62a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x560137a682e0 tx=0x7f4e18015040 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.480+0000 7f4e35622740 1 --2- 172.21.15.45:0/502200853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a75650 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c009980 tx=0x7f4e1c005b20 comp rx=0 tx=0).stop 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.481+0000 7f4e35622740 1 -- 172.21.15.45:0/502200853 shutdown_connections 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.481+0000 7f4e35622740 1 --2- 172.21.15.45:0/502200853 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 0x560137a77f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.481+0000 7f4e35622740 1 --2- 172.21.15.45:0/502200853 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a75650 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.481+0000 7f4e35622740 1 --2- 172.21.15.45:0/502200853 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a6ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.485 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.481+0000 7f4e35622740 1 -- 172.21.15.45:0/502200853 >> 172.21.15.45:0/502200853 conn(0x560137a66ec0 msgr2=0x560137a672a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.481+0000 7f4e35622740 1 -- 172.21.15.45:0/502200853 shutdown_connections 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.481+0000 7f4e35622740 1 -- 172.21.15.45:0/502200853 wait complete. 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.481+0000 7f4e35622740 1 -- start start 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.481+0000 7f4e35622740 1 Processor -- start 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e35622740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a72ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e35622740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a73010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e35622740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 0x560137a73550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e35622740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560137a78760 con 0x560137a6a9b0 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e35622740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x56013798a310 con 0x560137a75b90 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e35622740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560137a788e0 con 0x560137a6b2d0 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e35622740 10 client.0 _ll_register_callbacks cb 0x560137958cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a73010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e2be00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 0x560137a73550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a73010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39860/0 (socket says 172.21.15.45:39860) 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e2aa00640 1 -- 172.21.15.45:0/3777876879 learned_addr learned my addr 172.21.15.45:0/3777876879 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.482+0000 7f4e2b400640 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a72ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e2be00640 1 -- 172.21.15.45:0/3777876879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 msgr2=0x560137a73010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e2be00640 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a73010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e2be00640 1 -- 172.21.15.45:0/3777876879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 msgr2=0x560137a72ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e2be00640 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a72ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e2be00640 1 -- 172.21.15.45:0/3777876879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601377af620 con 0x560137a75b90 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e2be00640 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 0x560137a73550 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4e240cddc0 tx=0x7f4e2409bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5be4c61ca1aafe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e15600640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e15600640 1 -- 172.21.15.45:0/3777876879 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2401f040 con 0x560137a75b90 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e15600640 1 -- 172.21.15.45:0/3777876879 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f4e24085b50 con 0x560137a75b90 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e35622740 1 -- 172.21.15.45:0/3777876879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e1c041070 con 0x560137a75b90 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e15600640 1 -- 172.21.15.45:0/3777876879 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4e240a09c0 con 0x560137a75b90 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e35622740 10 client.25946 fetch_fsmap learned FSMap version 32 2024-04-25T19:07:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.483+0000 7f4e35622740 1 -- 172.21.15.45:0/3777876879 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560137a84a10 con 0x560137a75b90 2024-04-25T19:07:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.484+0000 7f4e15600640 1 -- 172.21.15.45:0/3777876879 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e1c041070 con 0x560137a75b90 2024-04-25T19:07:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.484+0000 7f4e15600640 1 -- 172.21.15.45:0/3777876879 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f4e2400f040 con 0x560137a75b90 2024-04-25T19:07:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.484+0000 7f4e35622740 10 client.25946 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:07:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.484+0000 7f4e35622740 10 client.25946 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.484+0000 7f4e35622740 20 client.25946 populate_metadata read hostname 'smithi045' 2024-04-25T19:07:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.484+0000 7f4e35622740 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560137a85f40 0x560137a88370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.484+0000 7f4e35622740 4 client.25946 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.484+0000 7f4e35622740 1 -- 172.21.15.45:0/3777876879 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601379e48b0 con 0x560137a85f40 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.484+0000 7f4e2b400640 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560137a85f40 0x560137a88370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.485+0000 7f4e2b400640 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560137a85f40 0x560137a88370 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c4de5b66b27a9a server_cookie=d3edf8f7893407eb in_seq=0 out_seq=0 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.485+0000 7f4e15600640 10 client.25946 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e15600640 1 -- 172.21.15.45:0/3777876879 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601379e48b0 con 0x560137a85f40 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e15600640 10 client.25946 handle_command_reply: tid=0 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 client.25946 shutdown 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 -- 172.21.15.45:0/3777876879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 msgr2=0x560137a73550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 0x560137a73550 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4e240cddc0 tx=0x7f4e2409bff0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 -- 172.21.15.45:0/3777876879 shutdown_connections 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560137a85f40 0x560137a88370 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a75b90 0x560137a73550 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e15600640 0 client.25946 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6b2d0 0x560137a73010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 --2- 172.21.15.45:0/3777876879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a72ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 -- 172.21.15.45:0/3777876879 >> 172.21.15.45:0/3777876879 conn(0x560137a66ec0 msgr2=0x560137a6cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 -- 172.21.15.45:0/3777876879 shutdown_connections 2024-04-25T19:07:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 1 -- 172.21.15.45:0/3777876879 wait complete. 2024-04-25T19:07:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.486+0000 7f4e35622740 20 client.25946 trim_cache size 0 max 16384 2024-04-25T19:07:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.508+0000 7f4e35622740 1 Processor -- start 2024-04-25T19:07:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.508+0000 7f4e35622740 1 -- start start 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.508+0000 7f4e35622740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a81680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.508+0000 7f4e35622740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a8dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.508+0000 7f4e35622740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a75b90 0x560137a8e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e35622740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c0110d0 con 0x560137a6a9b0 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e35622740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c00e9a0 con 0x560137a6b2d0 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e35622740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c00c810 con 0x560137a75b90 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a81680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a8dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a75b90 0x560137a8e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a75b90 0x560137a8e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39864/0 (socket says 172.21.15.45:39864) 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2be00640 1 -- 172.21.15.45:0/2320837769 learned_addr learned my addr 172.21.15.45:0/2320837769 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2aa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a8dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:53132/0 (socket says 172.21.15.45:53132) 2024-04-25T19:07:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2be00640 1 -- 172.21.15.45:0/2320837769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 msgr2=0x560137a8dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2be00640 1 --2- 172.21.15.45:0/2320837769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a8dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2be00640 1 -- 172.21.15.45:0/2320837769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 msgr2=0x560137a81680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2be00640 1 --2- 172.21.15.45:0/2320837769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a81680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2be00640 1 -- 172.21.15.45:0/2320837769 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560137a667a0 con 0x560137a75b90 2024-04-25T19:07:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.509+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/2320837769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a8dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.510+0000 7f4e2be00640 1 --2- 172.21.15.45:0/2320837769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a75b90 0x560137a8e1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4e24098fe0 tx=0x7f4e2409c200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92659f3451da4463 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.510+0000 7f4e17e00640 1 -- 172.21.15.45:0/2320837769 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2407f030 con 0x560137a75b90 2024-04-25T19:07:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.510+0000 7f4e17e00640 1 -- 172.21.15.45:0/2320837769 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e24098040 con 0x560137a75b90 2024-04-25T19:07:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.510+0000 7f4e17e00640 1 -- 172.21.15.45:0/2320837769 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2401f070 con 0x560137a75b90 2024-04-25T19:07:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.511+0000 7f4e35622740 1 -- 172.21.15.45:0/2320837769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a75b90 msgr2=0x560137a8e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.511+0000 7f4e35622740 1 --2- 172.21.15.45:0/2320837769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a75b90 0x560137a8e1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4e24098fe0 tx=0x7f4e2409c200 comp rx=0 tx=0).stop 2024-04-25T19:07:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.511+0000 7f4e35622740 1 -- 172.21.15.45:0/2320837769 shutdown_connections 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.511+0000 7f4e35622740 1 --2- 172.21.15.45:0/2320837769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a75b90 0x560137a8e1c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.511+0000 7f4e35622740 1 --2- 172.21.15.45:0/2320837769 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a8dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.511+0000 7f4e35622740 1 --2- 172.21.15.45:0/2320837769 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a6a9b0 0x560137a81680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.511+0000 7f4e35622740 1 -- 172.21.15.45:0/2320837769 >> 172.21.15.45:0/2320837769 conn(0x560137a66ec0 msgr2=0x560137a672a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.511+0000 7f4e35622740 1 -- 172.21.15.45:0/2320837769 shutdown_connections 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.511+0000 7f4e35622740 1 -- 172.21.15.45:0/2320837769 wait complete. 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.511+0000 7f4e35622740 1 -- start start 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e35622740 1 Processor -- start 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e35622740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6a9b0 0x560137a7df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e35622740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a7b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e35622740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a75b90 0x560137a7b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e35622740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c00ce60 con 0x560137a75b90 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e35622740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c00e2d0 con 0x560137a6b2d0 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e35622740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c00d530 con 0x560137a6a9b0 2024-04-25T19:07:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e35622740 10 client.0 _ll_register_callbacks cb 0x7f4e2409c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6a9b0 0x560137a7df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e2b400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6a9b0 0x560137a7df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39876/0 (socket says 172.21.15.45:39876) 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.512+0000 7f4e2b400640 1 -- 172.21.15.45:0/4167903120 learned_addr learned my addr 172.21.15.45:0/4167903120 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e2be00640 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a75b90 0x560137a7b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a7b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e2b400640 1 -- 172.21.15.45:0/4167903120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 msgr2=0x560137a7b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e2b400640 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a7b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e2b400640 1 -- 172.21.15.45:0/4167903120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a75b90 msgr2=0x560137a7b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e2b400640 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a75b90 0x560137a7b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e2b400640 1 -- 172.21.15.45:0/4167903120 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560137a7c0a0 con 0x560137a6a9b0 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a7b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e2b400640 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6a9b0 0x560137a7df20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005590 tx=0x7f4e1c009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5866c8e1a683189b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e15600640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.513+0000 7f4e15600640 1 -- 172.21.15.45:0/4167903120 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1c05a060 con 0x560137a6a9b0 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e15600640 1 -- 172.21.15.45:0/4167903120 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f4e1c004240 con 0x560137a6a9b0 2024-04-25T19:07:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e35622740 1 -- 172.21.15.45:0/4167903120 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e24098040 con 0x560137a6a9b0 2024-04-25T19:07:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e15600640 1 -- 172.21.15.45:0/4167903120 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4e1c004440 con 0x560137a6a9b0 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e35622740 10 client.25963 fetch_fsmap learned FSMap version 32 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e35622740 1 -- 172.21.15.45:0/4167903120 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560137aec4e0 con 0x560137a6a9b0 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e15600640 1 -- 172.21.15.45:0/4167903120 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e24098040 con 0x560137a6a9b0 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e15600640 1 -- 172.21.15.45:0/4167903120 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f4e1c04e040 con 0x560137a6a9b0 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e35622740 10 client.25963 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e35622740 10 client.25963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e35622740 20 client.25963 populate_metadata read hostname 'smithi045' 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.514+0000 7f4e35622740 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560137aec770 0x560137aeeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.515+0000 7f4e35622740 4 client.25963 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.515+0000 7f4e35622740 1 -- 172.21.15.45:0/4167903120 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601379de320 con 0x560137aec770 2024-04-25T19:07:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.515+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560137aec770 0x560137aeeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.515+0000 7f4e2aa00640 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560137aec770 0x560137aeeb60 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e91c1bbec45955a server_cookie=67122ceb983f308b in_seq=0 out_seq=0 2024-04-25T19:07:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.516+0000 7f4e15600640 10 client.25963 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:08.532 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e15600640 1 -- 172.21.15.45:0/4167903120 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601379de320 con 0x560137aec770 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e15600640 10 client.25963 handle_command_reply: tid=0 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 client.25963 shutdown 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 -- 172.21.15.45:0/4167903120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6a9b0 msgr2=0x560137a7df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6a9b0 0x560137a7df20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4e1c005590 tx=0x7f4e1c009590 comp rx=0 tx=0).stop 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 -- 172.21.15.45:0/4167903120 shutdown_connections 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560137aec770 0x560137aeeb60 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560137a75b90 0x560137a7b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560137a6b2d0 0x560137a7b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 --2- 172.21.15.45:0/4167903120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560137a6a9b0 0x560137a7df20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e15600640 0 client.25963 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 -- 172.21.15.45:0/4167903120 >> 172.21.15.45:0/4167903120 conn(0x560137a66ec0 msgr2=0x560137a72a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 -- 172.21.15.45:0/4167903120 shutdown_connections 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 1 -- 172.21.15.45:0/4167903120 wait complete. 2024-04-25T19:07:08.533 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.531+0000 7f4e35622740 20 client.25963 trim_cache size 0 max 16384 2024-04-25T19:07:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:07:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:07:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:07:08.534 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 -- 172.21.15.45:0/1665837779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4e100a3de0 msgr2=0x7f4e100a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4e100a3de0 0x7f4e100a62a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x560137a682e0 tx=0x7f4e18015040 comp rx=0 tx=0).stop 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 -- 172.21.15.45:0/1665837779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e2c112d80 msgr2=0x7f4e2c19ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e2c112d80 0x7f4e2c19ea50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005fd0 tx=0x7f4e2400ed10 comp rx=0 tx=0).stop 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 -- 172.21.15.45:0/1665837779 shutdown_connections 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f4e100a3de0 0x7f4e100a62a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4e2c112d80 0x7f4e2c19ea50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4e2c074c10 0x7f4e2c1a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 --2- 172.21.15.45:0/1665837779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e2c0742f0 0x7f4e2c1a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 -- 172.21.15.45:0/1665837779 >> 172.21.15.45:0/1665837779 conn(0x7f4e2c0fd860 msgr2=0x7f4e2c0ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.532+0000 7f4e31200640 1 -- 172.21.15.45:0/1665837779 shutdown_connections 2024-04-25T19:07:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:08.533+0000 7f4e31200640 1 -- 172.21.15.45:0/1665837779 wait complete. 2024-04-25T19:07:10.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:10 smithi062 ceph-mon[32844]: pgmap v243: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 749 KiB/s wr, 251 op/s 2024-04-25T19:07:10.386 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:10 smithi079 ceph-mon[33002]: pgmap v243: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 749 KiB/s wr, 251 op/s 2024-04-25T19:07:10.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:10 smithi045 ceph-mon[27776]: pgmap v243: 97 pgs: 97 active+clean; 110 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 749 KiB/s wr, 251 op/s 2024-04-25T19:07:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:12 smithi062 ceph-mon[32844]: pgmap v244: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.2 MiB/s wr, 411 op/s 2024-04-25T19:07:12.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:12 smithi079 ceph-mon[33002]: pgmap v244: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.2 MiB/s wr, 411 op/s 2024-04-25T19:07:12.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:12 smithi045 ceph-mon[27776]: pgmap v244: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.2 MiB/s wr, 411 op/s 2024-04-25T19:07:14.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:14 smithi062 ceph-mon[32844]: pgmap v245: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 406 op/s 2024-04-25T19:07:14.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:14 smithi079 ceph-mon[33002]: pgmap v245: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 406 op/s 2024-04-25T19:07:14.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:14 smithi045 ceph-mon[27776]: pgmap v245: 97 pgs: 97 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 1.2 MiB/s wr, 406 op/s 2024-04-25T19:07:16.363 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:16 smithi062 ceph-mon[32844]: pgmap v246: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.2 MiB/s wr, 401 op/s 2024-04-25T19:07:16.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:16 smithi079 ceph-mon[33002]: pgmap v246: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.2 MiB/s wr, 401 op/s 2024-04-25T19:07:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:16 smithi045 ceph-mon[27776]: pgmap v246: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 1.2 MiB/s wr, 401 op/s 2024-04-25T19:07:18.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:18 smithi062 ceph-mon[32844]: pgmap v247: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.1 MiB/s wr, 380 op/s 2024-04-25T19:07:18.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:18 smithi079 ceph-mon[33002]: pgmap v247: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.1 MiB/s wr, 380 op/s 2024-04-25T19:07:18.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:18 smithi045 ceph-mon[27776]: pgmap v247: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 1.1 MiB/s wr, 380 op/s 2024-04-25T19:07:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:20 smithi062 ceph-mon[32844]: pgmap v248: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1020 KiB/s wr, 334 op/s 2024-04-25T19:07:20.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:20 smithi079 ceph-mon[33002]: pgmap v248: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1020 KiB/s wr, 334 op/s 2024-04-25T19:07:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:20 smithi045 ceph-mon[27776]: pgmap v248: 97 pgs: 97 active+clean; 84 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1020 KiB/s wr, 334 op/s 2024-04-25T19:07:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.679+0000 7f66df200640 1 -- 172.21.15.45:0/2691613937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66d8074c20 msgr2=0x7f66d810e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.679+0000 7f66df200640 1 --2- 172.21.15.45:0/2691613937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66d8074c20 0x7f66d810e2c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f66c4009900 tx=0x7f66c402a480 comp rx=0 tx=0).stop 2024-04-25T19:07:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.680+0000 7f66df200640 1 -- 172.21.15.45:0/2691613937 shutdown_connections 2024-04-25T19:07:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.680+0000 7f66df200640 1 --2- 172.21.15.45:0/2691613937 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d810e800 0x7f66d8110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.680+0000 7f66df200640 1 --2- 172.21.15.45:0/2691613937 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66d8074c20 0x7f66d810e2c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.680+0000 7f66df200640 1 --2- 172.21.15.45:0/2691613937 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d8074300 0x7f66d80746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.680+0000 7f66df200640 1 -- 172.21.15.45:0/2691613937 >> 172.21.15.45:0/2691613937 conn(0x7f66d80fda10 msgr2=0x7f66d80ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.680+0000 7f66df200640 1 -- 172.21.15.45:0/2691613937 shutdown_connections 2024-04-25T19:07:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.680+0000 7f66df200640 1 -- 172.21.15.45:0/2691613937 wait complete. 2024-04-25T19:07:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.681+0000 7f66df200640 1 Processor -- start 2024-04-25T19:07:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.681+0000 7f66df200640 1 -- start start 2024-04-25T19:07:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66df200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d8074300 0x7f66d81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66df200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66d8074c20 0x7f66d81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66df200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d810e800 0x7f66d819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66df200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66d8114410 con 0x7f66d810e800 2024-04-25T19:07:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66df200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66d8114290 con 0x7f66d8074300 2024-04-25T19:07:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66df200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66d8114590 con 0x7f66d8074c20 2024-04-25T19:07:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66d7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66d8074c20 0x7f66d81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d8074300 0x7f66d81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66d7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66d8074c20 0x7f66d81a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39664/0 (socket says 172.21.15.45:39664) 2024-04-25T19:07:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66d7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d8074300 0x7f66d81a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:52036/0 (socket says 172.21.15.45:52036) 2024-04-25T19:07:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66d7400640 1 -- 172.21.15.45:0/1722903834 learned_addr learned my addr 172.21.15.45:0/1722903834 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66dd200640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d810e800 0x7f66d819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66d7e00640 1 -- 172.21.15.45:0/1722903834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66d8074c20 msgr2=0x7f66d81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66d7e00640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66d8074c20 0x7f66d81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66d7e00640 1 -- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d810e800 msgr2=0x7f66d819a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.682+0000 7f66d7e00640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d810e800 0x7f66d819a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.683+0000 7f66d7e00640 1 -- 172.21.15.45:0/1722903834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66d819afc0 con 0x7f66d8074300 2024-04-25T19:07:21.686 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.683+0000 7f66dd200640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d810e800 0x7f66d819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:07:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.683+0000 7f66d7e00640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d8074300 0x7f66d81a0680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f66c800e9f0 tx=0x7f66c800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.683+0000 7f66d4c00640 1 -- 172.21.15.45:0/1722903834 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c800ccc0 con 0x7f66d8074300 2024-04-25T19:07:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.683+0000 7f66d4c00640 1 -- 172.21.15.45:0/1722903834 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c8005020 con 0x7f66d8074300 2024-04-25T19:07:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.683+0000 7f66df200640 1 -- 172.21.15.45:0/1722903834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66d819b2b0 con 0x7f66d8074300 2024-04-25T19:07:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.683+0000 7f66d4c00640 1 -- 172.21.15.45:0/1722903834 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c8004a00 con 0x7f66d8074300 2024-04-25T19:07:21.687 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.683+0000 7f66df200640 1 -- 172.21.15.45:0/1722903834 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66d81a74d0 con 0x7f66d8074300 2024-04-25T19:07:21.688 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.687+0000 7f66d4c00640 1 -- 172.21.15.45:0/1722903834 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f66c8004ba0 con 0x7f66d8074300 2024-04-25T19:07:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.687+0000 7f66e2000740 1 Processor -- start 2024-04-25T19:07:21.689 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.687+0000 7f66e2000740 1 -- start start 2024-04-25T19:07:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66e2000740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 0x560ff9f93c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66e2000740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f94140 0x560ff9f9e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.691 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66e2000740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 0x560ff9fa0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66e2000740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560ff9ef9900 con 0x560ff9f94140 2024-04-25T19:07:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66e2000740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560ff9e9fef0 con 0x560ff9f93820 2024-04-25T19:07:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66e2000740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560ff9ef58c0 con 0x560ff9f9ea00 2024-04-25T19:07:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66dd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 0x560ff9f93c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66dd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 0x560ff9f93c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:52040/0 (socket says 172.21.15.45:52040) 2024-04-25T19:07:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66d7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 0x560ff9fa0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.692 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66dd200640 1 -- 172.21.15.45:0/2950015636 learned_addr learned my addr 172.21.15.45:0/2950015636 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66d7e00640 1 --2- 172.21.15.45:0/2950015636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f94140 0x560ff9f9e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66dd200640 1 -- 172.21.15.45:0/2950015636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 msgr2=0x560ff9fa0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66dd200640 1 --2- 172.21.15.45:0/2950015636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 0x560ff9fa0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66dd200640 1 -- 172.21.15.45:0/2950015636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f94140 msgr2=0x560ff9f9e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66dd200640 1 --2- 172.21.15.45:0/2950015636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f94140 0x560ff9f9e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66dd200640 1 -- 172.21.15.45:0/2950015636 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff9c0c210 con 0x560ff9f93820 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66d7400640 1 --2- 172.21.15.45:0/2950015636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 0x560ff9fa0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66d4c00640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f66bc0a3d10 0x7f66bc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.688+0000 7f66d7e00640 1 --2- 172.21.15.45:0/2950015636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f94140 0x560ff9f9e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.689+0000 7f66d4c00640 1 -- 172.21.15.45:0/1722903834 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f66c80d08a0 con 0x7f66d8074300 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.689+0000 7f66dd200640 1 --2- 172.21.15.45:0/2950015636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 0x560ff9f93c00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009fd0 tx=0x7f66d000f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7209b635ef4426b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:21.693 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.689+0000 7f66d7e00640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f66bc0a3d10 0x7f66bc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.689+0000 7f66cd600640 1 -- 172.21.15.45:0/2950015636 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0015070 con 0x560ff9f93820 2024-04-25T19:07:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.689+0000 7f66cd600640 1 -- 172.21.15.45:0/2950015636 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d0011bb0 con 0x560ff9f93820 2024-04-25T19:07:21.694 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.689+0000 7f66cd600640 1 -- 172.21.15.45:0/2950015636 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0010be0 con 0x560ff9f93820 2024-04-25T19:07:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.690+0000 7f66e2000740 1 -- 172.21.15.45:0/2950015636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 msgr2=0x560ff9f93c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.690+0000 7f66e2000740 1 --2- 172.21.15.45:0/2950015636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 0x560ff9f93c00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009fd0 tx=0x7f66d000f7f0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.690+0000 7f66d7e00640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f66bc0a3d10 0x7f66bc0a61d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f66c80ce070 tx=0x7f66c8009f60 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.690+0000 7f66e2000740 1 -- 172.21.15.45:0/2950015636 shutdown_connections 2024-04-25T19:07:21.695 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.690+0000 7f66e2000740 1 --2- 172.21.15.45:0/2950015636 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 0x560ff9fa0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.690+0000 7f66e2000740 1 --2- 172.21.15.45:0/2950015636 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f94140 0x560ff9f9e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.690+0000 7f66e2000740 1 --2- 172.21.15.45:0/2950015636 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 0x560ff9f93c00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.690+0000 7f66e2000740 1 -- 172.21.15.45:0/2950015636 >> 172.21.15.45:0/2950015636 conn(0x560ff9f8fd30 msgr2=0x560ff9f90110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.690+0000 7f66e2000740 1 -- 172.21.15.45:0/2950015636 shutdown_connections 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.690+0000 7f66e2000740 1 -- 172.21.15.45:0/2950015636 wait complete. 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.691+0000 7f66e2000740 1 -- start start 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.691+0000 7f66e2000740 1 Processor -- start 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.692+0000 7f66e2000740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 0x560ff9f9b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.692+0000 7f66e2000740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9f9bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.692+0000 7f66e2000740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f9ea00 0x560ff9f9c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.692+0000 7f66e2000740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560ff9fa1630 con 0x560ff9f9ea00 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.692+0000 7f66e2000740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x560ff9eb3310 con 0x560ff9f93820 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.692+0000 7f66e2000740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x560ff9fa17b0 con 0x560ff9f94140 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.692+0000 7f66e2000740 10 client.0 _ll_register_callbacks cb 0x560ff9e81cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:21.696 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.693+0000 7f66d7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9f9bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.693+0000 7f66dd200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 0x560ff9f9b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.693+0000 7f66d7400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9f9bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:39690/0 (socket says 172.21.15.45:39690) 2024-04-25T19:07:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.693+0000 7f66d7400640 1 -- 172.21.15.45:0/1943177267 learned_addr learned my addr 172.21.15.45:0/1943177267 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.693+0000 7f66dd200640 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f9ea00 0x560ff9f9c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.693+0000 7f66d7400640 1 -- 172.21.15.45:0/1943177267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 msgr2=0x560ff9f9b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.693+0000 7f66d7400640 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 0x560ff9f9b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.697 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.693+0000 7f66d7400640 1 -- 172.21.15.45:0/1943177267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f9ea00 msgr2=0x560ff9f9c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.694+0000 7f66d7400640 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f9ea00 0x560ff9f9c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.698 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.694+0000 7f66d7400640 1 -- 172.21.15.45:0/1943177267 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ff9cd8620 con 0x560ff9f94140 2024-04-25T19:07:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.694+0000 7f66dd200640 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f9ea00 0x560ff9f9c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:07:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.694+0000 7f66d7400640 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9f9bf20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f66c4006010 tx=0x7f66c4004510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e83cfae8d609410f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.694+0000 7f66c2000640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:07:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.695+0000 7f66c2000640 1 -- 172.21.15.45:0/1943177267 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4035040 con 0x560ff9f94140 2024-04-25T19:07:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.695+0000 7f66c2000640 1 -- 172.21.15.45:0/1943177267 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f66c4003bc0 con 0x560ff9f94140 2024-04-25T19:07:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.695+0000 7f66e2000740 1 -- 172.21.15.45:0/1943177267 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66d0011bb0 con 0x560ff9f94140 2024-04-25T19:07:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.696+0000 7f66c2000640 1 -- 172.21.15.45:0/1943177267 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f66c4003dc0 con 0x560ff9f94140 2024-04-25T19:07:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.696+0000 7f66e2000740 10 client.25975 fetch_fsmap learned FSMap version 32 2024-04-25T19:07:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.696+0000 7f66e2000740 1 -- 172.21.15.45:0/1943177267 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560ff9fa4880 con 0x560ff9f94140 2024-04-25T19:07:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.696+0000 7f66c2000640 1 -- 172.21.15.45:0/1943177267 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66d0011bb0 con 0x560ff9f94140 2024-04-25T19:07:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.697+0000 7f66c2000640 1 -- 172.21.15.45:0/1943177267 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f66c4004c70 con 0x560ff9f94140 2024-04-25T19:07:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.697+0000 7f66e2000740 10 client.25975 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:07:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.697+0000 7f66e2000740 10 client.25975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.697+0000 7f66e2000740 20 client.25975 populate_metadata read hostname 'smithi045' 2024-04-25T19:07:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.697+0000 7f66e2000740 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560ff9fa5d40 0x560ff9fa81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.697+0000 7f66e2000740 4 client.25975 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:07:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.697+0000 7f66e2000740 1 -- 172.21.15.45:0/1943177267 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ff9f0d8b0 con 0x560ff9fa5d40 2024-04-25T19:07:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.697+0000 7f66dd200640 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560ff9fa5d40 0x560ff9fa81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.698+0000 7f66dd200640 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560ff9fa5d40 0x560ff9fa81a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1440007a9ac1b9e server_cookie=809b869842d6ff66 in_seq=0 out_seq=0 2024-04-25T19:07:21.701 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.698+0000 7f66c2000640 10 client.25975 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66c2000640 1 -- 172.21.15.45:0/1943177267 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ff9f0d8b0 con 0x560ff9fa5d40 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66c2000640 10 client.25975 handle_command_reply: tid=0 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66e2000740 1 client.25975 shutdown 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66e2000740 1 -- 172.21.15.45:0/1943177267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 msgr2=0x560ff9f9bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66e2000740 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9f9bf20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f66c4006010 tx=0x7f66c4004510 comp rx=0 tx=0).stop 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66e2000740 1 -- 172.21.15.45:0/1943177267 shutdown_connections 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66e2000740 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560ff9fa5d40 0x560ff9fa81a0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66e2000740 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f9ea00 0x560ff9f9c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66e2000740 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9f9bf20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66e2000740 1 --2- 172.21.15.45:0/1943177267 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f93820 0x560ff9f9b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66c2000640 0 client.25975 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.699+0000 7f66e2000740 1 -- 172.21.15.45:0/1943177267 >> 172.21.15.45:0/1943177267 conn(0x560ff9f8fd30 msgr2=0x560ff9f95ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.700+0000 7f66e2000740 1 -- 172.21.15.45:0/1943177267 shutdown_connections 2024-04-25T19:07:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.700+0000 7f66e2000740 1 -- 172.21.15.45:0/1943177267 wait complete. 2024-04-25T19:07:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.700+0000 7f66e2000740 20 client.25975 trim_cache size 0 max 16384 2024-04-25T19:07:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.717+0000 7f66e2000740 1 Processor -- start 2024-04-25T19:07:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.717+0000 7f66e2000740 1 -- start start 2024-04-25T19:07:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.717+0000 7f66e2000740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 0x560ff9f961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.717+0000 7f66e2000740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f94140 0x560ff9f9c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.717+0000 7f66e2000740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 0x560ff9fc92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.717+0000 7f66e2000740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81ae420 con 0x560ff9f93820 2024-04-25T19:07:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.717+0000 7f66e2000740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81abcf0 con 0x560ff9f94140 2024-04-25T19:07:21.720 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.717+0000 7f66e2000740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81a9b60 con 0x560ff9f9ea00 2024-04-25T19:07:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66d7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f94140 0x560ff9f9c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66dd200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 0x560ff9f961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66d7400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f94140 0x560ff9f9c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:52058/0 (socket says 172.21.15.45:52058) 2024-04-25T19:07:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66d7400640 1 -- 172.21.15.45:0/3440141174 learned_addr learned my addr 172.21.15.45:0/3440141174 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66dd200640 1 --2- 172.21.15.45:0/3440141174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 0x560ff9fc92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66d7400640 1 -- 172.21.15.45:0/3440141174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 msgr2=0x560ff9fc92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:21.721 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66d7400640 1 --2- 172.21.15.45:0/3440141174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 0x560ff9fc92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66d7400640 1 -- 172.21.15.45:0/3440141174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 msgr2=0x560ff9f961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66d7400640 1 --2- 172.21.15.45:0/3440141174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 0x560ff9f961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66d7400640 1 -- 172.21.15.45:0/3440141174 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ff9fa0df0 con 0x560ff9f94140 2024-04-25T19:07:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66dd200640 1 --2- 172.21.15.45:0/3440141174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 0x560ff9fc92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:07:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.718+0000 7f66d7400640 1 --2- 172.21.15.45:0/3440141174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f94140 0x560ff9f9c0b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f66c4005ab0 tx=0x7f66c402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46f7f2f0e4fedc0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.719+0000 7f66ccc00640 1 -- 172.21.15.45:0/3440141174 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4035070 con 0x560ff9f94140 2024-04-25T19:07:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.719+0000 7f66ccc00640 1 -- 172.21.15.45:0/3440141174 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c40039c0 con 0x560ff9f94140 2024-04-25T19:07:21.722 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.719+0000 7f66ccc00640 1 -- 172.21.15.45:0/3440141174 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4000e60 con 0x560ff9f94140 2024-04-25T19:07:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.720+0000 7f66e2000740 1 -- 172.21.15.45:0/3440141174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f94140 msgr2=0x560ff9f9c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.720+0000 7f66e2000740 1 --2- 172.21.15.45:0/3440141174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f94140 0x560ff9f9c0b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f66c4005ab0 tx=0x7f66c402a480 comp rx=0 tx=0).stop 2024-04-25T19:07:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.720+0000 7f66e2000740 1 -- 172.21.15.45:0/3440141174 shutdown_connections 2024-04-25T19:07:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.720+0000 7f66e2000740 1 --2- 172.21.15.45:0/3440141174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f9ea00 0x560ff9fc92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.720+0000 7f66e2000740 1 --2- 172.21.15.45:0/3440141174 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f94140 0x560ff9f9c0b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.720+0000 7f66e2000740 1 --2- 172.21.15.45:0/3440141174 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 0x560ff9f961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.720+0000 7f66e2000740 1 -- 172.21.15.45:0/3440141174 >> 172.21.15.45:0/3440141174 conn(0x560ff9f8fd30 msgr2=0x560ff9f9b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:21.724 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.720+0000 7f66e2000740 1 -- 172.21.15.45:0/3440141174 shutdown_connections 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.720+0000 7f66e2000740 1 -- 172.21.15.45:0/3440141174 wait complete. 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.720+0000 7f66e2000740 1 -- start start 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.721+0000 7f66e2000740 1 Processor -- start 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.721+0000 7f66e2000740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 0x560ff9fabbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.721+0000 7f66e2000740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9fac110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.721+0000 7f66e2000740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f9ea00 0x560ffa024b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.721+0000 7f66e2000740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81aa1b0 con 0x560ff9f93820 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.721+0000 7f66e2000740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81ab620 con 0x560ff9f9ea00 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.721+0000 7f66e2000740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f66d81aa880 con 0x560ff9f94140 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.721+0000 7f66e2000740 10 client.0 _ll_register_callbacks cb 0x7f66c40041a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.721+0000 7f66dd200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 0x560ff9fabbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66dd200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 0x560ff9fabbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:48846/0 (socket says 172.21.15.45:48846) 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66dd200640 1 -- 172.21.15.45:0/4005081313 learned_addr learned my addr 172.21.15.45:0/4005081313 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66d7400640 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9fac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.725 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66dd200640 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f9ea00 0x560ffa024b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66d7400640 1 -- 172.21.15.45:0/4005081313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f9ea00 msgr2=0x560ffa024b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66d7400640 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f9ea00 0x560ffa024b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66d7400640 1 -- 172.21.15.45:0/4005081313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 msgr2=0x560ff9fabbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66d7400640 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 0x560ff9fabbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.726 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66d7400640 1 -- 172.21.15.45:0/4005081313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ffa025060 con 0x560ff9f94140 2024-04-25T19:07:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66d7400640 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9fac110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x560ff9f8b2a0 tx=0x7f66c4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=871a2c73f2528ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.722+0000 7f66c2000640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:07:21.727 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.723+0000 7f66c2000640 1 -- 172.21.15.45:0/4005081313 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66c4035040 con 0x560ff9f94140 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.723+0000 7f66c2000640 1 -- 172.21.15.45:0/4005081313 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f66c4005220 con 0x560ff9f94140 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.723+0000 7f66e2000740 1 -- 172.21.15.45:0/4005081313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66c40039c0 con 0x560ff9f94140 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.723+0000 7f66c2000640 1 -- 172.21.15.45:0/4005081313 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f66c4005420 con 0x560ff9f94140 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.723+0000 7f66e2000740 10 client.25978 fetch_fsmap learned FSMap version 32 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.723+0000 7f66e2000740 1 -- 172.21.15.45:0/4005081313 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560ffa0262b0 con 0x560ff9f94140 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.723+0000 7f66c2000640 1 -- 172.21.15.45:0/4005081313 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66c40039c0 con 0x560ff9f94140 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.724+0000 7f66c2000640 1 -- 172.21.15.45:0/4005081313 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f66c403c5f0 con 0x560ff9f94140 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.724+0000 7f66e2000740 10 client.25978 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.724+0000 7f66e2000740 10 client.25978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.724+0000 7f66e2000740 20 client.25978 populate_metadata read hostname 'smithi045' 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.724+0000 7f66e2000740 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560ffa0269b0 0x560ffa028dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.724+0000 7f66e2000740 4 client.25978 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.724+0000 7f66e2000740 1 -- 172.21.15.45:0/4005081313 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ff9f07320 con 0x560ffa0269b0 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.724+0000 7f66dd200640 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560ffa0269b0 0x560ffa028dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.724+0000 7f66c2000640 10 client.25978 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:21.728 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.724+0000 7f66dd200640 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560ffa0269b0 0x560ffa028dd0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f01e2d1d59d2eb77 server_cookie=ac9b44c4d9156e30 in_seq=0 out_seq=0 2024-04-25T19:07:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.740+0000 7f66c2000640 1 -- 172.21.15.45:0/4005081313 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ff9f07320 con 0x560ffa0269b0 2024-04-25T19:07:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.740+0000 7f66c2000640 10 client.25978 handle_command_reply: tid=0 2024-04-25T19:07:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.740+0000 7f66e2000740 1 client.25978 shutdown 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.740+0000 7f66e2000740 1 -- 172.21.15.45:0/4005081313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 msgr2=0x560ff9fac110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.740+0000 7f66e2000740 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9fac110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x560ff9f8b2a0 tx=0x7f66c4009260 comp rx=0 tx=0).stop 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.741+0000 7f66e2000740 1 -- 172.21.15.45:0/4005081313 shutdown_connections 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.741+0000 7f66e2000740 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x560ffa0269b0 0x560ffa028dd0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.741+0000 7f66e2000740 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x560ff9f9ea00 0x560ffa024b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.741+0000 7f66e2000740 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x560ff9f94140 0x560ff9fac110 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.741+0000 7f66c2000640 0 client.25978 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.741+0000 7f66e2000740 1 --2- 172.21.15.45:0/4005081313 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560ff9f93820 0x560ff9fabbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.741+0000 7f66e2000740 1 -- 172.21.15.45:0/4005081313 >> 172.21.15.45:0/4005081313 conn(0x560ff9f8fd30 msgr2=0x560ff9f9c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.741+0000 7f66e2000740 1 -- 172.21.15.45:0/4005081313 shutdown_connections 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.741+0000 7f66e2000740 1 -- 172.21.15.45:0/4005081313 wait complete. 2024-04-25T19:07:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.741+0000 7f66e2000740 20 client.25978 trim_cache size 0 max 16384 2024-04-25T19:07:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:07:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:07:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:07:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.742+0000 7f66df200640 1 -- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f66bc0a3d10 msgr2=0x7f66bc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.742+0000 7f66df200640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f66bc0a3d10 0x7f66bc0a61d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f66c80ce070 tx=0x7f66c8009f60 comp rx=0 tx=0).stop 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.742+0000 7f66df200640 1 -- 172.21.15.45:0/1722903834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d8074300 msgr2=0x7f66d81a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.742+0000 7f66df200640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d8074300 0x7f66d81a0680 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f66c800e9f0 tx=0x7f66c800eeb0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.742+0000 7f66df200640 1 -- 172.21.15.45:0/1722903834 shutdown_connections 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.742+0000 7f66df200640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f66bc0a3d10 0x7f66bc0a61d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.742+0000 7f66df200640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66d810e800 0x7f66d819a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.742+0000 7f66df200640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f66d8074c20 0x7f66d81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.742+0000 7f66df200640 1 --2- 172.21.15.45:0/1722903834 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f66d8074300 0x7f66d81a0680 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.742+0000 7f66df200640 1 -- 172.21.15.45:0/1722903834 >> 172.21.15.45:0/1722903834 conn(0x7f66d80fda10 msgr2=0x7f66d80ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.743+0000 7f66df200640 1 -- 172.21.15.45:0/1722903834 shutdown_connections 2024-04-25T19:07:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:21.743+0000 7f66df200640 1 -- 172.21.15.45:0/1722903834 wait complete. 2024-04-25T19:07:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:22 smithi062 ceph-mon[32844]: pgmap v249: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 476 op/s 2024-04-25T19:07:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:22 smithi079 ceph-mon[33002]: pgmap v249: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 476 op/s 2024-04-25T19:07:22.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:22 smithi045 ceph-mon[27776]: pgmap v249: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 476 op/s 2024-04-25T19:07:24.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:24 smithi062 ceph-mon[32844]: pgmap v250: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 931 KiB/s wr, 304 op/s 2024-04-25T19:07:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:24 smithi079 ceph-mon[33002]: pgmap v250: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 931 KiB/s wr, 304 op/s 2024-04-25T19:07:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:24 smithi045 ceph-mon[27776]: pgmap v250: 97 pgs: 97 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 931 KiB/s wr, 304 op/s 2024-04-25T19:07:26.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:26 smithi062 ceph-mon[32844]: pgmap v251: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.3 MiB/s wr, 453 op/s 2024-04-25T19:07:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:26 smithi079 ceph-mon[33002]: pgmap v251: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.3 MiB/s wr, 453 op/s 2024-04-25T19:07:26.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:26 smithi045 ceph-mon[27776]: pgmap v251: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.3 MiB/s wr, 453 op/s 2024-04-25T19:07:28.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:28 smithi062 ceph-mon[32844]: pgmap v252: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 908 KiB/s wr, 289 op/s 2024-04-25T19:07:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:28 smithi079 ceph-mon[33002]: pgmap v252: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 908 KiB/s wr, 289 op/s 2024-04-25T19:07:28.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:28 smithi045 ceph-mon[27776]: pgmap v252: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 908 KiB/s wr, 289 op/s 2024-04-25T19:07:30.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:30 smithi062 ceph-mon[32844]: pgmap v253: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 908 KiB/s wr, 289 op/s 2024-04-25T19:07:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:30 smithi079 ceph-mon[33002]: pgmap v253: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 908 KiB/s wr, 289 op/s 2024-04-25T19:07:30.468 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:30 smithi045 ceph-mon[27776]: pgmap v253: 97 pgs: 97 active+clean; 95 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 908 KiB/s wr, 289 op/s 2024-04-25T19:07:32.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:32 smithi062 ceph-mon[32844]: pgmap v254: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.3 MiB/s wr, 430 op/s 2024-04-25T19:07:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:32 smithi079 ceph-mon[33002]: pgmap v254: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.3 MiB/s wr, 430 op/s 2024-04-25T19:07:32.466 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:32 smithi045 ceph-mon[27776]: pgmap v254: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 1.3 MiB/s wr, 430 op/s 2024-04-25T19:07:34.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:34 smithi062 ceph-mon[32844]: pgmap v255: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 877 KiB/s wr, 288 op/s 2024-04-25T19:07:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:34 smithi079 ceph-mon[33002]: pgmap v255: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 877 KiB/s wr, 288 op/s 2024-04-25T19:07:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:34 smithi045 ceph-mon[27776]: pgmap v255: 97 pgs: 97 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 877 KiB/s wr, 288 op/s 2024-04-25T19:07:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.901+0000 7fd0ff600640 1 -- 172.21.15.45:0/996687787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0f810e7a0 msgr2=0x7fd0f8110bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.901+0000 7fd0ff600640 1 --2- 172.21.15.45:0/996687787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0f810e7a0 0x7fd0f8110bf0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0009ca0 tx=0x7fd0f002a950 comp rx=0 tx=0).stop 2024-04-25T19:07:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.901+0000 7fd0ff600640 1 -- 172.21.15.45:0/996687787 shutdown_connections 2024-04-25T19:07:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.901+0000 7fd0ff600640 1 --2- 172.21.15.45:0/996687787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0f810e7a0 0x7fd0f8110bf0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.901+0000 7fd0ff600640 1 --2- 172.21.15.45:0/996687787 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8074ca0 0x7fd0f810e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.901+0000 7fd0ff600640 1 --2- 172.21.15.45:0/996687787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f8074380 0x7fd0f8074760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.901+0000 7fd0ff600640 1 -- 172.21.15.45:0/996687787 >> 172.21.15.45:0/996687787 conn(0x7fd0f80fda10 msgr2=0x7fd0f80ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:34.904 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.902+0000 7fd0ff600640 1 -- 172.21.15.45:0/996687787 shutdown_connections 2024-04-25T19:07:34.905 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.902+0000 7fd0ff600640 1 -- 172.21.15.45:0/996687787 wait complete. 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.903+0000 7fd0ff600640 1 Processor -- start 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.903+0000 7fd0ff600640 1 -- start start 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.903+0000 7fd0ff600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f8074380 0x7fd0f81a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.903+0000 7fd0ff600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8074ca0 0x7fd0f81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.903+0000 7fd0ff600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0f810e7a0 0x7fd0f819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.903+0000 7fd0ff600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81143d0 con 0x7fd0f8074380 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.903+0000 7fd0ff600640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8114250 con 0x7fd0f8074ca0 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.903+0000 7fd0ff600640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f8114550 con 0x7fd0f810e7a0 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8074ca0 0x7fd0f81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8074ca0 0x7fd0f81a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41378/0 (socket says 172.21.15.45:41378) 2024-04-25T19:07:34.907 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0f7e00640 1 -- 172.21.15.45:0/1815729940 learned_addr learned my addr 172.21.15.45:0/1815729940 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0fcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f8074380 0x7fd0f81a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0fd600640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0f810e7a0 0x7fd0f819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0f7e00640 1 -- 172.21.15.45:0/1815729940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0f810e7a0 msgr2=0x7fd0f819a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0f7e00640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0f810e7a0 0x7fd0f819a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0f7e00640 1 -- 172.21.15.45:0/1815729940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f8074380 msgr2=0x7fd0f81a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0f7e00640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f8074380 0x7fd0f81a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0f7e00640 1 -- 172.21.15.45:0/1815729940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f819afd0 con 0x7fd0f8074ca0 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0f7e00640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8074ca0 0x7fd0f81a0b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400e9f0 tx=0x7fd0e400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.904+0000 7fd0f5600640 1 -- 172.21.15.45:0/1815729940 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e400cd80 con 0x7fd0f8074ca0 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.905+0000 7fd0ff600640 1 -- 172.21.15.45:0/1815729940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f819b2c0 con 0x7fd0f8074ca0 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.905+0000 7fd0f5600640 1 -- 172.21.15.45:0/1815729940 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0e40054d0 con 0x7fd0f8074ca0 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.905+0000 7fd0f5600640 1 -- 172.21.15.45:0/1815729940 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4010690 con 0x7fd0f8074ca0 2024-04-25T19:07:34.908 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.905+0000 7fd0ff600640 1 -- 172.21.15.45:0/1815729940 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f81a75d0 con 0x7fd0f8074ca0 2024-04-25T19:07:34.910 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.908+0000 7fd0f5600640 1 -- 172.21.15.45:0/1815729940 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fd0e4004e90 con 0x7fd0f8074ca0 2024-04-25T19:07:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.909+0000 7fd10245f740 1 Processor -- start 2024-04-25T19:07:34.911 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.909+0000 7fd10245f740 1 -- start start 2024-04-25T19:07:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.909+0000 7fd10245f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc25c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.912 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.909+0000 7fd10245f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc26140 0x557c6cc304c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.909+0000 7fd10245f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 0x557c6cc32df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.909+0000 7fd10245f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557c6cb8b900 con 0x557c6cc26140 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.909+0000 7fd10245f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557c6cb31ef0 con 0x557c6cc25820 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.909+0000 7fd10245f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557c6cb878c0 con 0x557c6cc30a00 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0fd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc25c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0fd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc25c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41384/0 (socket says 172.21.15.45:41384) 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0fd600640 1 -- 172.21.15.45:0/1406982204 learned_addr learned my addr 172.21.15.45:0/1406982204 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0f7e00640 1 --2- 172.21.15.45:0/1406982204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 0x557c6cc32df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0fcc00640 1 --2- 172.21.15.45:0/1406982204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc26140 0x557c6cc304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0fd600640 1 -- 172.21.15.45:0/1406982204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 msgr2=0x557c6cc32df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0fd600640 1 --2- 172.21.15.45:0/1406982204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 0x557c6cc32df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0fd600640 1 -- 172.21.15.45:0/1406982204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc26140 msgr2=0x557c6cc304c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0fd600640 1 --2- 172.21.15.45:0/1406982204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc26140 0x557c6cc304c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.913 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0fd600640 1 -- 172.21.15.45:0/1406982204 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6c89e210 con 0x557c6cc25820 2024-04-25T19:07:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0f5600640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd0e00a3d10 0x7fd0e00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.910+0000 7fd0fd600640 1 --2- 172.21.15.45:0/1406982204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc25c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0035000 tx=0x7fd0f00093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=848fa5975344762a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.911+0000 7fd0f5600640 1 -- 172.21.15.45:0/1815729940 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fd0e40d0c10 con 0x7fd0f8074ca0 2024-04-25T19:07:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.911+0000 7fd0fcc00640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd0e00a3d10 0x7fd0e00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.911+0000 7fd0e9c00640 1 -- 172.21.15.45:0/1406982204 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0048070 con 0x557c6cc25820 2024-04-25T19:07:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.911+0000 7fd0e9c00640 1 -- 172.21.15.45:0/1406982204 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f0003e20 con 0x557c6cc25820 2024-04-25T19:07:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.911+0000 7fd0e9c00640 1 -- 172.21.15.45:0/1406982204 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0034040 con 0x557c6cc25820 2024-04-25T19:07:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd10245f740 1 -- 172.21.15.45:0/1406982204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 msgr2=0x557c6cc25c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd0fcc00640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd0e00a3d10 0x7fd0e00a61d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x557c6cc26dc0 tx=0x7fd0ec005f90 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd10245f740 1 --2- 172.21.15.45:0/1406982204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc25c00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0035000 tx=0x7fd0f00093e0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd10245f740 1 -- 172.21.15.45:0/1406982204 shutdown_connections 2024-04-25T19:07:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd10245f740 1 --2- 172.21.15.45:0/1406982204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 0x557c6cc32df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd10245f740 1 --2- 172.21.15.45:0/1406982204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc26140 0x557c6cc304c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd10245f740 1 --2- 172.21.15.45:0/1406982204 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc25c00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd10245f740 1 -- 172.21.15.45:0/1406982204 >> 172.21.15.45:0/1406982204 conn(0x557c6cc21d30 msgr2=0x557c6cc22110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd10245f740 1 -- 172.21.15.45:0/1406982204 shutdown_connections 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd10245f740 1 -- 172.21.15.45:0/1406982204 wait complete. 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.912+0000 7fd10245f740 1 -- start start 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd10245f740 1 Processor -- start 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd10245f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc25820 0x557c6cc2d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd10245f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 0x557c6cc2de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd10245f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc30a00 0x557c6cc2e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd10245f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x557c6cc335d0 con 0x557c6cc25820 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd10245f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x557c6cb45310 con 0x557c6cc30a00 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd10245f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x557c6cc33750 con 0x557c6cc26140 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd10245f740 10 client.0 _ll_register_callbacks cb 0x557c6cb13cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd0fd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc30a00 0x557c6cc2e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd0f7e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 0x557c6cc2de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd0fcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc25820 0x557c6cc2d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd0fd600640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc30a00 0x557c6cc2e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41396/0 (socket says 172.21.15.45:41396) 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.913+0000 7fd0fd600640 1 -- 172.21.15.45:0/139344468 learned_addr learned my addr 172.21.15.45:0/139344468 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd0fd600640 1 -- 172.21.15.45:0/139344468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 msgr2=0x557c6cc2de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd0fd600640 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 0x557c6cc2de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd0fd600640 1 -- 172.21.15.45:0/139344468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc25820 msgr2=0x557c6cc2d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd0fd600640 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc25820 0x557c6cc2d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd0fd600640 1 -- 172.21.15.45:0/139344468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6c96a620 con 0x557c6cc30a00 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd0f7e00640 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 0x557c6cc2de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd0fd600640 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc30a00 0x557c6cc2e370 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0006010 tx=0x7fd0f0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f78af45f293b5229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd0de000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd0de000640 1 -- 172.21.15.45:0/139344468 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0051060 con 0x557c6cc30a00 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd10245f740 1 -- 172.21.15.45:0/139344468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0f0003e20 con 0x557c6cc30a00 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.914+0000 7fd0de000640 1 -- 172.21.15.45:0/139344468 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fd0f0004210 con 0x557c6cc30a00 2024-04-25T19:07:34.920 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd0de000640 1 -- 172.21.15.45:0/139344468 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd0f0034070 con 0x557c6cc30a00 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd10245f740 10 client.25982 fetch_fsmap learned FSMap version 32 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd10245f740 1 -- 172.21.15.45:0/139344468 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c6cc3f7d0 con 0x557c6cc30a00 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd0de000640 1 -- 172.21.15.45:0/139344468 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0f0003e20 con 0x557c6cc30a00 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd0de000640 1 -- 172.21.15.45:0/139344468 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fd0f0030440 con 0x557c6cc30a00 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd10245f740 10 client.25982 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd10245f740 10 client.25982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd10245f740 20 client.25982 populate_metadata read hostname 'smithi045' 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd10245f740 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557c6cc40b20 0x557c6cc42f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd10245f740 4 client.25982 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.915+0000 7fd10245f740 1 -- 172.21.15.45:0/139344468 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c6cb9f8b0 con 0x557c6cc40b20 2024-04-25T19:07:34.921 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.916+0000 7fd0fcc00640 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557c6cc40b20 0x557c6cc42f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.916+0000 7fd0fcc00640 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557c6cc40b20 0x557c6cc42f10 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f21bb1b0281f9c server_cookie=6fef7d06e8367416 in_seq=0 out_seq=0 2024-04-25T19:07:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.916+0000 7fd0de000640 10 client.25982 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:34.922 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.917+0000 7fd0de000640 1 -- 172.21.15.45:0/139344468 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c6cb9f8b0 con 0x557c6cc40b20 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.917+0000 7fd0de000640 10 client.25982 handle_command_reply: tid=0 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.919+0000 7fd10245f740 1 client.25982 shutdown 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.919+0000 7fd10245f740 1 -- 172.21.15.45:0/139344468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc30a00 msgr2=0x557c6cc2e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.919+0000 7fd10245f740 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc30a00 0x557c6cc2e370 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0006010 tx=0x7fd0f0004a40 comp rx=0 tx=0).stop 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.919+0000 7fd10245f740 1 -- 172.21.15.45:0/139344468 shutdown_connections 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.919+0000 7fd10245f740 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557c6cc40b20 0x557c6cc42f10 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.919+0000 7fd10245f740 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc30a00 0x557c6cc2e370 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.919+0000 7fd10245f740 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 0x557c6cc2de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.919+0000 7fd10245f740 1 --2- 172.21.15.45:0/139344468 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc25820 0x557c6cc2d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.919+0000 7fd0de000640 0 client.25982 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.919+0000 7fd10245f740 1 -- 172.21.15.45:0/139344468 >> 172.21.15.45:0/139344468 conn(0x557c6cc21d30 msgr2=0x557c6cc27ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.920+0000 7fd10245f740 1 -- 172.21.15.45:0/139344468 shutdown_connections 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.920+0000 7fd10245f740 1 -- 172.21.15.45:0/139344468 wait complete. 2024-04-25T19:07:34.923 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.920+0000 7fd10245f740 20 client.25982 trim_cache size 0 max 16384 2024-04-25T19:07:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.938+0000 7fd10245f740 1 Processor -- start 2024-04-25T19:07:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.938+0000 7fd10245f740 1 -- start start 2024-04-25T19:07:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.938+0000 7fd10245f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc3c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.938+0000 7fd10245f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 0x557c6cc20600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.938+0000 7fd10245f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc30a00 0x557c6cc2e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.940 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.938+0000 7fd10245f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81b2660 con 0x557c6cc30a00 2024-04-25T19:07:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.938+0000 7fd10245f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81aff30 con 0x557c6cc25820 2024-04-25T19:07:34.941 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.938+0000 7fd10245f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81adda0 con 0x557c6cc26140 2024-04-25T19:07:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fcc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc3c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0f7e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 0x557c6cc20600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fcc00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc3c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41408/0 (socket says 172.21.15.45:41408) 2024-04-25T19:07:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fcc00640 1 -- 172.21.15.45:0/274931279 learned_addr learned my addr 172.21.15.45:0/274931279 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fd600640 1 --2- 172.21.15.45:0/274931279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc30a00 0x557c6cc2e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fcc00640 1 -- 172.21.15.45:0/274931279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 msgr2=0x557c6cc20600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fcc00640 1 --2- 172.21.15.45:0/274931279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 0x557c6cc20600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fcc00640 1 -- 172.21.15.45:0/274931279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc30a00 msgr2=0x557c6cc2e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:34.942 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fcc00640 1 --2- 172.21.15.45:0/274931279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc30a00 0x557c6cc2e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fcc00640 1 -- 172.21.15.45:0/274931279 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c6cc25f70 con 0x557c6cc25820 2024-04-25T19:07:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fd600640 1 --2- 172.21.15.45:0/274931279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc30a00 0x557c6cc2e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:07:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0f7e00640 1 --2- 172.21.15.45:0/274931279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 0x557c6cc20600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.939+0000 7fd0fcc00640 1 --2- 172.21.15.45:0/274931279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc3c490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x557c6cc43450 tx=0x7fd0ec01b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afb660bc1e3aebc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.940+0000 7fd0e9200640 1 -- 172.21.15.45:0/274931279 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec0043a0 con 0x557c6cc25820 2024-04-25T19:07:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.940+0000 7fd0e9200640 1 -- 172.21.15.45:0/274931279 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ec02b8d0 con 0x557c6cc25820 2024-04-25T19:07:34.943 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.940+0000 7fd0e9200640 1 -- 172.21.15.45:0/274931279 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec0047a0 con 0x557c6cc25820 2024-04-25T19:07:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.940+0000 7fd10245f740 1 -- 172.21.15.45:0/274931279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 msgr2=0x557c6cc3c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.940+0000 7fd10245f740 1 --2- 172.21.15.45:0/274931279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc3c490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x557c6cc43450 tx=0x7fd0ec01b000 comp rx=0 tx=0).stop 2024-04-25T19:07:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.941+0000 7fd10245f740 1 -- 172.21.15.45:0/274931279 shutdown_connections 2024-04-25T19:07:34.946 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.941+0000 7fd10245f740 1 --2- 172.21.15.45:0/274931279 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc30a00 0x557c6cc2e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.941+0000 7fd10245f740 1 --2- 172.21.15.45:0/274931279 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc26140 0x557c6cc20600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.941+0000 7fd10245f740 1 --2- 172.21.15.45:0/274931279 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc25820 0x557c6cc3c490 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.941+0000 7fd10245f740 1 -- 172.21.15.45:0/274931279 >> 172.21.15.45:0/274931279 conn(0x557c6cc21d30 msgr2=0x557c6cc2c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.941+0000 7fd10245f740 1 -- 172.21.15.45:0/274931279 shutdown_connections 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.941+0000 7fd10245f740 1 -- 172.21.15.45:0/274931279 wait complete. 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.941+0000 7fd10245f740 1 -- start start 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.941+0000 7fd10245f740 1 Processor -- start 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd10245f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc25820 0x557c6cc38aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd10245f740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc26140 0x557c6cc35f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd10245f740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 0x557c6cc364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd10245f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81ae3f0 con 0x557c6cc25820 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd10245f740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81af860 con 0x557c6cc26140 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd10245f740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f81aeac0 con 0x557c6cc30a00 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd10245f740 10 client.0 _ll_register_callbacks cb 0x7fd0e40d4590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd0f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc26140 0x557c6cc35f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.947 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd0fcc00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc25820 0x557c6cc38aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd0fd600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 0x557c6cc364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd0f7e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc26140 0x557c6cc35f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41416/0 (socket says 172.21.15.45:41416) 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd0f7e00640 1 -- 172.21.15.45:0/4068799157 learned_addr learned my addr 172.21.15.45:0/4068799157 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.942+0000 7fd0fd600640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 0x557c6cc364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:51644/0 (socket says 172.21.15.45:51644) 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0f7e00640 1 -- 172.21.15.45:0/4068799157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 msgr2=0x557c6cc364b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0f7e00640 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 0x557c6cc364b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0f7e00640 1 -- 172.21.15.45:0/4068799157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc25820 msgr2=0x557c6cc38aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0f7e00640 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc25820 0x557c6cc38aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0f7e00640 1 -- 172.21.15.45:0/4068799157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c6cc369f0 con 0x557c6cc26140 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0fd600640 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 0x557c6cc364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0f7e00640 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc26140 0x557c6cc35f70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40d0eb0 tx=0x7fd0e4098e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6d3f1f24e16d6eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0de000640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0de000640 1 -- 172.21.15.45:0/4068799157 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e409c480 con 0x557c6cc26140 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0de000640 1 -- 172.21.15.45:0/4068799157 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fd0e40a5300 con 0x557c6cc26140 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd10245f740 1 -- 172.21.15.45:0/4068799157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0ec02b8d0 con 0x557c6cc26140 2024-04-25T19:07:34.948 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.943+0000 7fd0de000640 1 -- 172.21.15.45:0/4068799157 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd0e4005180 con 0x557c6cc26140 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd10245f740 10 client.25988 fetch_fsmap learned FSMap version 32 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd10245f740 1 -- 172.21.15.45:0/4068799157 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557c6cca6a40 con 0x557c6cc26140 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd0de000640 1 -- 172.21.15.45:0/4068799157 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0ec02b8d0 con 0x557c6cc26140 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd0de000640 1 -- 172.21.15.45:0/4068799157 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fd0e40d9040 con 0x557c6cc26140 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd10245f740 10 client.25988 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd10245f740 10 client.25988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd10245f740 20 client.25988 populate_metadata read hostname 'smithi045' 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd10245f740 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557c6cca7170 0x557c6cca9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd10245f740 4 client.25988 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd10245f740 1 -- 172.21.15.45:0/4068799157 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c6cb99320 con 0x557c6cca7170 2024-04-25T19:07:34.949 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.944+0000 7fd0fcc00640 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557c6cca7170 0x557c6cca9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.945+0000 7fd0fcc00640 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557c6cca7170 0x557c6cca9590 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4363b17ffea71ab9 server_cookie=50621c5ef0e22983 in_seq=0 out_seq=0 2024-04-25T19:07:34.950 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.945+0000 7fd0de000640 10 client.25988 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd0de000640 1 -- 172.21.15.45:0/4068799157 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c6cb99320 con 0x557c6cca7170 2024-04-25T19:07:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd0de000640 10 client.25988 handle_command_reply: tid=0 2024-04-25T19:07:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 client.25988 shutdown 2024-04-25T19:07:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 -- 172.21.15.45:0/4068799157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc26140 msgr2=0x557c6cc35f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc26140 0x557c6cc35f70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40d0eb0 tx=0x7fd0e4098e80 comp rx=0 tx=0).stop 2024-04-25T19:07:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 -- 172.21.15.45:0/4068799157 shutdown_connections 2024-04-25T19:07:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x557c6cca7170 0x557c6cca9590 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x557c6cc30a00 0x557c6cc364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x557c6cc26140 0x557c6cc35f70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.963 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 --2- 172.21.15.45:0/4068799157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x557c6cc25820 0x557c6cc38aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd0de000640 0 client.25988 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 -- 172.21.15.45:0/4068799157 >> 172.21.15.45:0/4068799157 conn(0x557c6cc21d30 msgr2=0x557c6cc27140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 -- 172.21.15.45:0/4068799157 shutdown_connections 2024-04-25T19:07:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.960+0000 7fd10245f740 1 -- 172.21.15.45:0/4068799157 wait complete. 2024-04-25T19:07:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.961+0000 7fd10245f740 20 client.25988 trim_cache size 0 max 16384 2024-04-25T19:07:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:07:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:07:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:07:34.964 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:07:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 -- 172.21.15.45:0/1815729940 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd0e00a3d10 msgr2=0x7fd0e00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd0e00a3d10 0x7fd0e00a61d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x557c6cc26dc0 tx=0x7fd0ec005f90 comp rx=0 tx=0).stop 2024-04-25T19:07:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 -- 172.21.15.45:0/1815729940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8074ca0 msgr2=0x7fd0f81a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8074ca0 0x7fd0f81a0b90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400e9f0 tx=0x7fd0e400edf0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 -- 172.21.15.45:0/1815729940 shutdown_connections 2024-04-25T19:07:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fd0e00a3d10 0x7fd0e00a61d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.965 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd0f810e7a0 0x7fd0f819a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fd0f8074ca0 0x7fd0f81a0b90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 --2- 172.21.15.45:0/1815729940 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd0f8074380 0x7fd0f81a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 -- 172.21.15.45:0/1815729940 >> 172.21.15.45:0/1815729940 conn(0x7fd0f80fda10 msgr2=0x7fd0f80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 -- 172.21.15.45:0/1815729940 shutdown_connections 2024-04-25T19:07:34.966 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:34.962+0000 7fd0ff600640 1 -- 172.21.15.45:0/1815729940 wait complete. 2024-04-25T19:07:36.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:36 smithi062 ceph-mon[32844]: pgmap v256: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.3 MiB/s wr, 440 op/s 2024-04-25T19:07:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:36 smithi079 ceph-mon[33002]: pgmap v256: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.3 MiB/s wr, 440 op/s 2024-04-25T19:07:36.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:36 smithi045 ceph-mon[27776]: pgmap v256: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.3 MiB/s wr, 440 op/s 2024-04-25T19:07:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:38 smithi079 ceph-mon[33002]: pgmap v257: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 862 KiB/s wr, 292 op/s 2024-04-25T19:07:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:38 smithi045 ceph-mon[27776]: pgmap v257: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 862 KiB/s wr, 292 op/s 2024-04-25T19:07:38.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:38 smithi062 ceph-mon[32844]: pgmap v257: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 862 KiB/s wr, 292 op/s 2024-04-25T19:07:40.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:40 smithi079 ceph-mon[33002]: pgmap v258: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 862 KiB/s wr, 292 op/s 2024-04-25T19:07:40.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:40 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:07:40.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:40 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:07:40.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:40 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:07:40.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:40 smithi045 ceph-mon[27776]: pgmap v258: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 862 KiB/s wr, 292 op/s 2024-04-25T19:07:40.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:40 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:07:40.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:40 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:07:40.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:40 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:07:40.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:40 smithi062 ceph-mon[32844]: pgmap v258: 97 pgs: 97 active+clean; 92 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 862 KiB/s wr, 292 op/s 2024-04-25T19:07:40.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:40 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:07:40.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:40 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:07:40.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:40 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:07:42.415 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:42 smithi062 ceph-mon[32844]: pgmap v259: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 465 op/s 2024-04-25T19:07:42.416 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:42 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:07:42.416 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:42 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:07:42.416 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:42 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:07:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:42 smithi079 ceph-mon[33002]: pgmap v259: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 465 op/s 2024-04-25T19:07:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:42 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:07:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:42 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:07:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:42 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:07:42.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:42 smithi045 ceph-mon[27776]: pgmap v259: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.4 MiB/s wr, 465 op/s 2024-04-25T19:07:42.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:42 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:07:42.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:42 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:07:42.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:42 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:07:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:44 smithi079 ceph-mon[33002]: pgmap v260: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 988 KiB/s wr, 324 op/s 2024-04-25T19:07:44.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:44 smithi045 ceph-mon[27776]: pgmap v260: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 988 KiB/s wr, 324 op/s 2024-04-25T19:07:44.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:44 smithi062 ceph-mon[32844]: pgmap v260: 97 pgs: 97 active+clean; 94 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 988 KiB/s wr, 324 op/s 2024-04-25T19:07:46.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:46 smithi079 ceph-mon[33002]: pgmap v261: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.4 MiB/s wr, 475 op/s 2024-04-25T19:07:46.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:46 smithi045 ceph-mon[27776]: pgmap v261: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.4 MiB/s wr, 475 op/s 2024-04-25T19:07:46.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:46 smithi062 ceph-mon[32844]: pgmap v261: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.4 MiB/s wr, 475 op/s 2024-04-25T19:07:48.127 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.125+0000 7fefebe00640 1 -- 172.21.15.45:0/1969371226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefec102970 msgr2=0x7fefec1069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.125+0000 7fefebe00640 1 --2- 172.21.15.45:0/1969371226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefec102970 0x7fefec1069b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009860 tx=0x7fefd802a340 comp rx=0 tx=0).stop 2024-04-25T19:07:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.125+0000 7fefebe00640 1 -- 172.21.15.45:0/1969371226 shutdown_connections 2024-04-25T19:07:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.125+0000 7fefebe00640 1 --2- 172.21.15.45:0/1969371226 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec107090 0x7fefec10ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.125+0000 7fefebe00640 1 --2- 172.21.15.45:0/1969371226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefec102970 0x7fefec1069b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.125+0000 7fefebe00640 1 --2- 172.21.15.45:0/1969371226 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefec101fc0 0x7fefec1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.125+0000 7fefebe00640 1 -- 172.21.15.45:0/1969371226 >> 172.21.15.45:0/1969371226 conn(0x7fefec0fd880 msgr2=0x7fefec0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.125+0000 7fefebe00640 1 -- 172.21.15.45:0/1969371226 shutdown_connections 2024-04-25T19:07:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.125+0000 7fefebe00640 1 -- 172.21.15.45:0/1969371226 wait complete. 2024-04-25T19:07:48.128 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.126+0000 7fefebe00640 1 Processor -- start 2024-04-25T19:07:48.129 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.127+0000 7fefebe00640 1 -- start start 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.127+0000 7fefebe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefec101fc0 0x7fefec1a0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.127+0000 7fefebe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec107090 0x7fefec1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.127+0000 7fefebe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefec19a670 0x7fefec19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.127+0000 7fefebe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefec111fb0 con 0x7fefec19a670 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.127+0000 7fefebe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fefec111e30 con 0x7fefec107090 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.127+0000 7fefebe00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fefec112130 con 0x7fefec101fc0 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.127+0000 7fefeaa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefec101fc0 0x7fefec1a0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.128+0000 7fefeaa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefec101fc0 0x7fefec1a0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48644/0 (socket says 172.21.15.45:48644) 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.128+0000 7fefeaa00640 1 -- 172.21.15.45:0/354083747 learned_addr learned my addr 172.21.15.45:0/354083747 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.128+0000 7fefea000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec107090 0x7fefec1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.130 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.128+0000 7fefeb400640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefec19a670 0x7fefec19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.129+0000 7fefeb400640 1 -- 172.21.15.45:0/354083747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefec101fc0 msgr2=0x7fefec1a0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.129+0000 7fefeb400640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefec101fc0 0x7fefec1a0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.129+0000 7fefeb400640 1 -- 172.21.15.45:0/354083747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec107090 msgr2=0x7fefec1a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.129+0000 7fefeb400640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec107090 0x7fefec1a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.129+0000 7fefeb400640 1 -- 172.21.15.45:0/354083747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefec19b190 con 0x7fefec19a670 2024-04-25T19:07:48.131 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.129+0000 7fefeb400640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefec19a670 0x7fefec19ab20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fefe400efc0 tx=0x7fefe400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.130+0000 7fefe3400640 1 -- 172.21.15.45:0/354083747 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4003970 con 0x7fefec19a670 2024-04-25T19:07:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.130+0000 7fefe3400640 1 -- 172.21.15.45:0/354083747 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefe4004020 con 0x7fefec19a670 2024-04-25T19:07:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.130+0000 7fefe3400640 1 -- 172.21.15.45:0/354083747 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe400b940 con 0x7fefec19a670 2024-04-25T19:07:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.130+0000 7fefebe00640 1 -- 172.21.15.45:0/354083747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefec19b480 con 0x7fefec19a670 2024-04-25T19:07:48.132 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.130+0000 7fefebe00640 1 -- 172.21.15.45:0/354083747 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefec1a7630 con 0x7fefec19a670 2024-04-25T19:07:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.135+0000 7fefe3400640 1 -- 172.21.15.45:0/354083747 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fefe401d020 con 0x7fefec19a670 2024-04-25T19:07:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.135+0000 7feff440e740 1 Processor -- start 2024-04-25T19:07:48.137 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.135+0000 7feff440e740 1 -- start start 2024-04-25T19:07:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.136+0000 7feff440e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdea6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.136+0000 7feff440e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea72d0 0x5573bdeb1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.136+0000 7feff440e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdeb1b90 0x5573bdeb3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.136+0000 7feff440e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5573bde0c900 con 0x5573bdeb1b90 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.136+0000 7feff440e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5573bddb2ef0 con 0x5573bdea72d0 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.136+0000 7feff440e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5573bde088c0 con 0x5573bdea69b0 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.136+0000 7fefea000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdea6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.136+0000 7fefea000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdea6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48648/0 (socket says 172.21.15.45:48648) 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.136+0000 7fefea000640 1 -- 172.21.15.45:0/3517748571 learned_addr learned my addr 172.21.15.45:0/3517748571 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.136+0000 7fefeaa00640 1 --2- 172.21.15.45:0/3517748571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea72d0 0x5573bdeb1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.137+0000 7fefea000640 1 --2- 172.21.15.45:0/3517748571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdeb1b90 0x5573bdeb3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.137+0000 7fefeaa00640 1 -- 172.21.15.45:0/3517748571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 msgr2=0x5573bdea6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.137+0000 7fefeaa00640 1 --2- 172.21.15.45:0/3517748571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdea6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.137+0000 7fefeaa00640 1 -- 172.21.15.45:0/3517748571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdeb1b90 msgr2=0x5573bdeb3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.137+0000 7fefeaa00640 1 --2- 172.21.15.45:0/3517748571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdeb1b90 0x5573bdeb3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.137+0000 7fefeaa00640 1 -- 172.21.15.45:0/3517748571 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573bdb1f210 con 0x5573bdea72d0 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.137+0000 7fefe3400640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fefd00a3d80 0x7fefd00a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.137+0000 7fefe3400640 1 -- 172.21.15.45:0/354083747 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fefe40d0680 con 0x7fefec19a670 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.137+0000 7fefeaa00640 1 --2- 172.21.15.45:0/3517748571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea72d0 0x5573bdeb1650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009c80 tx=0x7fefdc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4600db1fac61c402 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.137+0000 7fefea000640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fefd00a3d80 0x7fefd00a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.138+0000 7fefea000640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fefd00a3d80 0x7fefd00a6240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5573bdea4050 tx=0x7fefd802c040 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.139+0000 7fefd7400640 1 -- 172.21.15.45:0/3517748571 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc004230 con 0x5573bdea72d0 2024-04-25T19:07:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.139+0000 7fefd7400640 1 -- 172.21.15.45:0/3517748571 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefdc0043d0 con 0x5573bdea72d0 2024-04-25T19:07:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.139+0000 7fefd7400640 1 -- 172.21.15.45:0/3517748571 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc004d80 con 0x5573bdea72d0 2024-04-25T19:07:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.140+0000 7feff440e740 1 -- 172.21.15.45:0/3517748571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea72d0 msgr2=0x5573bdeb1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.140+0000 7feff440e740 1 --2- 172.21.15.45:0/3517748571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea72d0 0x5573bdeb1650 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009c80 tx=0x7fefdc011810 comp rx=0 tx=0).stop 2024-04-25T19:07:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.141+0000 7feff440e740 1 -- 172.21.15.45:0/3517748571 shutdown_connections 2024-04-25T19:07:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.141+0000 7feff440e740 1 --2- 172.21.15.45:0/3517748571 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdeb1b90 0x5573bdeb3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.141+0000 7feff440e740 1 --2- 172.21.15.45:0/3517748571 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea72d0 0x5573bdeb1650 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.141+0000 7feff440e740 1 --2- 172.21.15.45:0/3517748571 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdea6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.141+0000 7feff440e740 1 -- 172.21.15.45:0/3517748571 >> 172.21.15.45:0/3517748571 conn(0x5573bdea2ec0 msgr2=0x5573bdea32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.142+0000 7feff440e740 1 -- 172.21.15.45:0/3517748571 shutdown_connections 2024-04-25T19:07:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.142+0000 7feff440e740 1 -- 172.21.15.45:0/3517748571 wait complete. 2024-04-25T19:07:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.142+0000 7feff440e740 1 -- start start 2024-04-25T19:07:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.142+0000 7feff440e740 1 Processor -- start 2024-04-25T19:07:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7feff440e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdeaeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7feff440e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea72d0 0x5573bdeaf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7feff440e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdeaf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7feff440e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5573bdeb47c0 con 0x5573bdea72d0 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7feff440e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5573bddc6310 con 0x5573bdeb1b90 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7feff440e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5573bdeb4940 con 0x5573bdea69b0 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7feff440e740 10 client.0 _ll_register_callbacks cb 0x5573bdd94cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7fefeaa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdeaeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7fefea000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea72d0 0x5573bdeaf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7fefeaa00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdeaeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48650/0 (socket says 172.21.15.45:48650) 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7fefeb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdeaf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.143+0000 7fefeaa00640 1 -- 172.21.15.45:0/267913094 learned_addr learned my addr 172.21.15.45:0/267913094 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.144+0000 7fefeb400640 1 -- 172.21.15.45:0/267913094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 msgr2=0x5573bdeaeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.144+0000 7fefeb400640 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdeaeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.144+0000 7fefeb400640 1 -- 172.21.15.45:0/267913094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea72d0 msgr2=0x5573bdeaf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.144+0000 7fefeb400640 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea72d0 0x5573bdeaf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.148 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.144+0000 7fefeb400640 1 -- 172.21.15.45:0/267913094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573bdbeb620 con 0x5573bdeb1b90 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.144+0000 7fefeaa00640 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdeaeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.144+0000 7fefeb400640 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdeaf5c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefe40cdda0 tx=0x7fefe409c010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e6eb7d97415d23d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.144+0000 7fefcbe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.144+0000 7fefcbe00640 1 -- 172.21.15.45:0/267913094 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe40a09a0 con 0x5573bdeb1b90 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.145+0000 7fefcbe00640 1 -- 172.21.15.45:0/267913094 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fefe4098c60 con 0x5573bdeb1b90 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.145+0000 7feff440e740 1 -- 172.21.15.45:0/267913094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefdc0043d0 con 0x5573bdeb1b90 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.145+0000 7fefcbe00640 1 -- 172.21.15.45:0/267913094 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fefe40a5260 con 0x5573bdeb1b90 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.145+0000 7feff440e740 10 client.26000 fetch_fsmap learned FSMap version 32 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.145+0000 7feff440e740 1 -- 172.21.15.45:0/267913094 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573bdec0aa0 con 0x5573bdeb1b90 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.145+0000 7fefcbe00640 1 -- 172.21.15.45:0/267913094 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefdc0043d0 con 0x5573bdeb1b90 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.145+0000 7fefcbe00640 1 -- 172.21.15.45:0/267913094 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fefe400b040 con 0x5573bdeb1b90 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.146+0000 7feff440e740 10 client.26000 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.146+0000 7feff440e740 10 client.26000 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.146+0000 7feff440e740 20 client.26000 populate_metadata read hostname 'smithi045' 2024-04-25T19:07:48.149 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.146+0000 7feff440e740 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5573bdec1f10 0x5573bdec4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.146+0000 7feff440e740 4 client.26000 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:07:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.146+0000 7feff440e740 1 -- 172.21.15.45:0/267913094 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573bde208b0 con 0x5573bdec1f10 2024-04-25T19:07:48.150 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.146+0000 7fefeaa00640 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5573bdec1f10 0x5573bdec4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.147+0000 7fefeaa00640 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5573bdec1f10 0x5573bdec4320 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8762a8c81ee5783a server_cookie=d632d6af6e91496f in_seq=0 out_seq=0 2024-04-25T19:07:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.147+0000 7fefcbe00640 10 client.26000 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7fefcbe00640 1 -- 172.21.15.45:0/267913094 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573bde208b0 con 0x5573bdec1f10 2024-04-25T19:07:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7fefcbe00640 10 client.26000 handle_command_reply: tid=0 2024-04-25T19:07:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 client.26000 shutdown 2024-04-25T19:07:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 -- 172.21.15.45:0/267913094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 msgr2=0x5573bdeaf5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdeaf5c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fefe40cdda0 tx=0x7fefe409c010 comp rx=0 tx=0).stop 2024-04-25T19:07:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 -- 172.21.15.45:0/267913094 shutdown_connections 2024-04-25T19:07:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5573bdec1f10 0x5573bdec4320 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.151 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdeaf5c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea72d0 0x5573bdeaf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 --2- 172.21.15.45:0/267913094 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea69b0 0x5573bdeaeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7fefcbe00640 0 client.26000 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 -- 172.21.15.45:0/267913094 >> 172.21.15.45:0/267913094 conn(0x5573bdea2ec0 msgr2=0x5573bdea8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 -- 172.21.15.45:0/267913094 shutdown_connections 2024-04-25T19:07:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 1 -- 172.21.15.45:0/267913094 wait complete. 2024-04-25T19:07:48.152 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.149+0000 7feff440e740 20 client.26000 trim_cache size 0 max 16384 2024-04-25T19:07:48.170 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.166+0000 7feff440e740 1 Processor -- start 2024-04-25T19:07:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.166+0000 7feff440e740 1 -- start start 2024-04-25T19:07:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.166+0000 7feff440e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea69b0 0x5573bdebd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.166+0000 7feff440e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea72d0 0x5573bdea27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.166+0000 7feff440e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdea8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.166+0000 7feff440e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1b2590 con 0x5573bdea69b0 2024-04-25T19:07:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.166+0000 7feff440e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1afe60 con 0x5573bdeb1b90 2024-04-25T19:07:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.166+0000 7feff440e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1adcd0 con 0x5573bdea72d0 2024-04-25T19:07:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefeb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdea8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefeb400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdea8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48636/0 (socket says 172.21.15.45:48636) 2024-04-25T19:07:48.171 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefeb400640 1 -- 172.21.15.45:0/2482673649 learned_addr learned my addr 172.21.15.45:0/2482673649 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefeaa00640 1 --2- 172.21.15.45:0/2482673649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea69b0 0x5573bdebd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefea000640 1 --2- 172.21.15.45:0/2482673649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea72d0 0x5573bdea27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefeb400640 1 -- 172.21.15.45:0/2482673649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea72d0 msgr2=0x5573bdea27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefeb400640 1 --2- 172.21.15.45:0/2482673649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea72d0 0x5573bdea27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefeb400640 1 -- 172.21.15.45:0/2482673649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea69b0 msgr2=0x5573bdebd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefeb400640 1 --2- 172.21.15.45:0/2482673649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea69b0 0x5573bdebd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefeb400640 1 -- 172.21.15.45:0/2482673649 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573bdea2ce0 con 0x5573bdeb1b90 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.167+0000 7fefea000640 1 --2- 172.21.15.45:0/2482673649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea72d0 0x5573bdea27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.168+0000 7fefeb400640 1 --2- 172.21.15.45:0/2482673649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdea8a70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fefe40cde50 tx=0x7fefe4098290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cd51fbaa4d89ec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.168+0000 7fefd6a00640 1 -- 172.21.15.45:0/2482673649 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe4098c60 con 0x5573bdeb1b90 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.168+0000 7fefd6a00640 1 -- 172.21.15.45:0/2482673649 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefe40041d0 con 0x5573bdeb1b90 2024-04-25T19:07:48.172 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.168+0000 7fefd6a00640 1 -- 172.21.15.45:0/2482673649 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe400e0c0 con 0x5573bdeb1b90 2024-04-25T19:07:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 -- 172.21.15.45:0/2482673649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 msgr2=0x5573bdea8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 --2- 172.21.15.45:0/2482673649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdea8a70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fefe40cde50 tx=0x7fefe4098290 comp rx=0 tx=0).stop 2024-04-25T19:07:48.176 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 -- 172.21.15.45:0/2482673649 shutdown_connections 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 --2- 172.21.15.45:0/2482673649 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdeb1b90 0x5573bdea8a70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 --2- 172.21.15.45:0/2482673649 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdea72d0 0x5573bdea27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 --2- 172.21.15.45:0/2482673649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea69b0 0x5573bdebd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 -- 172.21.15.45:0/2482673649 >> 172.21.15.45:0/2482673649 conn(0x5573bdea2ec0 msgr2=0x5573bdea8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 -- 172.21.15.45:0/2482673649 shutdown_connections 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 -- 172.21.15.45:0/2482673649 wait complete. 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 -- start start 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.169+0000 7feff440e740 1 Processor -- start 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7feff440e740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea69b0 0x5573bdeba030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7feff440e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea72d0 0x5573bdeb7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7feff440e740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdeb1b90 0x5573bdeb7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7feff440e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1ae320 con 0x5573bdea72d0 2024-04-25T19:07:48.177 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7feff440e740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1af790 con 0x5573bdea69b0 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7feff440e740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fefec1ae9f0 con 0x5573bdeb1b90 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7feff440e740 10 client.0 _ll_register_callbacks cb 0x7fefe409c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7fefeaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea69b0 0x5573bdeba030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7fefeaa00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea69b0 0x5573bdeba030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48652/0 (socket says 172.21.15.45:48652) 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7fefeaa00640 1 -- 172.21.15.45:0/4035743100 learned_addr learned my addr 172.21.15.45:0/4035743100 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7fefeb400640 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdeb1b90 0x5573bdeb7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.170+0000 7fefea000640 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea72d0 0x5573bdeb7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefeaa00640 1 -- 172.21.15.45:0/4035743100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdeb1b90 msgr2=0x5573bdeb7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefeaa00640 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdeb1b90 0x5573bdeb7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefeaa00640 1 -- 172.21.15.45:0/4035743100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea72d0 msgr2=0x5573bdeb7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefeaa00640 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea72d0 0x5573bdeb7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefeaa00640 1 -- 172.21.15.45:0/4035743100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573bdeb8240 con 0x5573bdea69b0 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefeb400640 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdeb1b90 0x5573bdeb7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:07:48.178 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefeaa00640 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea69b0 0x5573bdeba030 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009c80 tx=0x7fefdc009720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3df18a7401ac245e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefcbe00640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefcbe00640 1 -- 172.21.15.45:0/4035743100 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefdc01e040 con 0x5573bdea69b0 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefcbe00640 1 -- 172.21.15.45:0/4035743100 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7fefdc02d360 con 0x5573bdea69b0 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7feff440e740 1 -- 172.21.15.45:0/4035743100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefe40041d0 con 0x5573bdea69b0 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7fefcbe00640 1 -- 172.21.15.45:0/4035743100 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fefdc02d580 con 0x5573bdea69b0 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7feff440e740 10 client.26006 fetch_fsmap learned FSMap version 32 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.171+0000 7feff440e740 1 -- 172.21.15.45:0/4035743100 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573bdf285e0 con 0x5573bdea69b0 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.172+0000 7fefcbe00640 1 -- 172.21.15.45:0/4035743100 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefe40041d0 con 0x5573bdea69b0 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.172+0000 7fefcbe00640 1 -- 172.21.15.45:0/4035743100 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7fefdc024040 con 0x5573bdea69b0 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.172+0000 7feff440e740 10 client.26006 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.172+0000 7feff440e740 10 client.26006 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.172+0000 7feff440e740 20 client.26006 populate_metadata read hostname 'smithi045' 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.172+0000 7feff440e740 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5573bdf28b00 0x5573bdf2aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:07:48.179 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.172+0000 7feff440e740 4 client.26006 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:07:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.172+0000 7feff440e740 1 -- 172.21.15.45:0/4035743100 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573bde1a320 con 0x5573bdf28b00 2024-04-25T19:07:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.172+0000 7fefea000640 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5573bdf28b00 0x5573bdf2aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:07:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.173+0000 7fefea000640 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5573bdf28b00 0x5573bdf2aef0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1fa9180aae59465 server_cookie=63a43f71d9f5270 in_seq=0 out_seq=0 2024-04-25T19:07:48.180 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.173+0000 7fefcbe00640 10 client.26006 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:48.194 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.192+0000 7fefcbe00640 1 -- 172.21.15.45:0/4035743100 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5573bde1a320 con 0x5573bdf28b00 2024-04-25T19:07:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.192+0000 7fefcbe00640 10 client.26006 handle_command_reply: tid=0 2024-04-25T19:07:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.192+0000 7feff440e740 1 client.26006 shutdown 2024-04-25T19:07:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.192+0000 7feff440e740 1 -- 172.21.15.45:0/4035743100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea69b0 msgr2=0x5573bdeba030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:48.195 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.192+0000 7feff440e740 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea69b0 0x5573bdeba030 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fefdc009c80 tx=0x7fefdc009720 comp rx=0 tx=0).stop 2024-04-25T19:07:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.199+0000 7feff440e740 1 -- 172.21.15.45:0/4035743100 shutdown_connections 2024-04-25T19:07:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.199+0000 7feff440e740 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5573bdf28b00 0x5573bdf2aef0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.199+0000 7feff440e740 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5573bdeb1b90 0x5573bdeb7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.199+0000 7feff440e740 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5573bdea72d0 0x5573bdeb7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.199+0000 7feff440e740 1 --2- 172.21.15.45:0/4035743100 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5573bdea69b0 0x5573bdeba030 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.201 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.199+0000 7feff440e740 1 -- 172.21.15.45:0/4035743100 >> 172.21.15.45:0/4035743100 conn(0x5573bdea2ec0 msgr2=0x5573bdea8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.199+0000 7fefcbe00640 0 client.26006 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:07:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.200+0000 7feff440e740 1 -- 172.21.15.45:0/4035743100 shutdown_connections 2024-04-25T19:07:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.200+0000 7feff440e740 1 -- 172.21.15.45:0/4035743100 wait complete. 2024-04-25T19:07:48.202 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.200+0000 7feff440e740 20 client.26006 trim_cache size 0 max 16384 2024-04-25T19:07:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:07:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:07:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:07:48.203 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.200+0000 7fefebe00640 1 -- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fefd00a3d80 msgr2=0x7fefd00a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.200+0000 7fefebe00640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fefd00a3d80 0x7fefd00a6240 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5573bdea4050 tx=0x7fefd802c040 comp rx=0 tx=0).stop 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.200+0000 7fefebe00640 1 -- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefec19a670 msgr2=0x7fefec19ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.200+0000 7fefebe00640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefec19a670 0x7fefec19ab20 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fefe400efc0 tx=0x7fefe400c800 comp rx=0 tx=0).stop 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.201+0000 7fefebe00640 1 -- 172.21.15.45:0/354083747 shutdown_connections 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.201+0000 7fefebe00640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fefd00a3d80 0x7fefd00a6240 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.201+0000 7fefebe00640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fefec19a670 0x7fefec19ab20 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.201+0000 7fefebe00640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fefec107090 0x7fefec1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.201+0000 7fefebe00640 1 --2- 172.21.15.45:0/354083747 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fefec101fc0 0x7fefec1a0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.201+0000 7fefebe00640 1 -- 172.21.15.45:0/354083747 >> 172.21.15.45:0/354083747 conn(0x7fefec0fd880 msgr2=0x7fefec0febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.201+0000 7fefebe00640 1 -- 172.21.15.45:0/354083747 shutdown_connections 2024-04-25T19:07:48.204 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:07:48.201+0000 7fefebe00640 1 -- 172.21.15.45:0/354083747 wait complete. 2024-04-25T19:07:48.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:48 smithi079 ceph-mon[33002]: pgmap v262: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 323 op/s 2024-04-25T19:07:48.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:48 smithi045 ceph-mon[27776]: pgmap v262: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 323 op/s 2024-04-25T19:07:48.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:48 smithi062 ceph-mon[32844]: pgmap v262: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 323 op/s 2024-04-25T19:07:50.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:50 smithi079 ceph-mon[33002]: pgmap v263: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 323 op/s 2024-04-25T19:07:50.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:50 smithi045 ceph-mon[27776]: pgmap v263: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 323 op/s 2024-04-25T19:07:50.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:50 smithi062 ceph-mon[32844]: pgmap v263: 97 pgs: 97 active+clean; 98 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.0 MiB/s wr, 323 op/s 2024-04-25T19:07:52.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:52 smithi079 ceph-mon[33002]: pgmap v264: 97 pgs: 97 active+clean; 112 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.5 MiB/s wr, 473 op/s 2024-04-25T19:07:52.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:52 smithi045 ceph-mon[27776]: pgmap v264: 97 pgs: 97 active+clean; 112 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.5 MiB/s wr, 473 op/s 2024-04-25T19:07:52.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:52 smithi062 ceph-mon[32844]: pgmap v264: 97 pgs: 97 active+clean; 112 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 1.5 MiB/s wr, 473 op/s 2024-04-25T19:07:54.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:54 smithi079 ceph-mon[33002]: pgmap v265: 97 pgs: 97 active+clean; 112 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1016 KiB/s wr, 300 op/s 2024-04-25T19:07:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:54 smithi045 ceph-mon[27776]: pgmap v265: 97 pgs: 97 active+clean; 112 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1016 KiB/s wr, 300 op/s 2024-04-25T19:07:54.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:54 smithi062 ceph-mon[32844]: pgmap v265: 97 pgs: 97 active+clean; 112 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1016 KiB/s wr, 300 op/s 2024-04-25T19:07:56.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:56 smithi079 ceph-mon[33002]: pgmap v266: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.4 MiB/s wr, 434 op/s 2024-04-25T19:07:56.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:56 smithi045 ceph-mon[27776]: pgmap v266: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.4 MiB/s wr, 434 op/s 2024-04-25T19:07:56.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:56 smithi062 ceph-mon[32844]: pgmap v266: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 1.4 MiB/s wr, 434 op/s 2024-04-25T19:07:58.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:07:58 smithi079 ceph-mon[33002]: pgmap v267: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 916 KiB/s wr, 283 op/s 2024-04-25T19:07:58.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:07:58 smithi045 ceph-mon[27776]: pgmap v267: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 916 KiB/s wr, 283 op/s 2024-04-25T19:07:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:07:58 smithi062 ceph-mon[32844]: pgmap v267: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 916 KiB/s wr, 283 op/s 2024-04-25T19:08:00.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:00 smithi079 ceph-mon[33002]: pgmap v268: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 916 KiB/s wr, 283 op/s 2024-04-25T19:08:00.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:00 smithi045 ceph-mon[27776]: pgmap v268: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 916 KiB/s wr, 283 op/s 2024-04-25T19:08:00.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:00 smithi062 ceph-mon[32844]: pgmap v268: 97 pgs: 97 active+clean; 88 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 916 KiB/s wr, 283 op/s 2024-04-25T19:08:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.349+0000 7f620f200640 1 -- 172.21.15.45:0/1805151675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6210102990 msgr2=0x7f62101069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.349+0000 7f620f200640 1 --2- 172.21.15.45:0/1805151675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6210102990 0x7f62101069d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f61f8009900 tx=0x7f61f802a3f0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.350+0000 7f620f200640 1 -- 172.21.15.45:0/1805151675 shutdown_connections 2024-04-25T19:08:01.351 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.350+0000 7f620f200640 1 --2- 172.21.15.45:0/1805151675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62101070b0 0x7f621010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.350+0000 7f620f200640 1 --2- 172.21.15.45:0/1805151675 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f6210102990 0x7f62101069d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.350+0000 7f620f200640 1 --2- 172.21.15.45:0/1805151675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6210101fe0 0x7f62101023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.350+0000 7f620f200640 1 -- 172.21.15.45:0/1805151675 >> 172.21.15.45:0/1805151675 conn(0x7f62100fd860 msgr2=0x7f62100ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.350+0000 7f620f200640 1 -- 172.21.15.45:0/1805151675 shutdown_connections 2024-04-25T19:08:01.352 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.350+0000 7f620f200640 1 -- 172.21.15.45:0/1805151675 wait complete. 2024-04-25T19:08:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.351+0000 7f620f200640 1 Processor -- start 2024-04-25T19:08:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.351+0000 7f620f200640 1 -- start start 2024-04-25T19:08:01.354 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6210101fe0 0x7f62101a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62101070b0 0x7f62101a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620f200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621019a5e0 0x7f621019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6210112040 con 0x7f6210101fe0 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f6210111ec0 con 0x7f62101070b0 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620f200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62101121c0 con 0x7f621019a5e0 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6210101fe0 0x7f62101a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62101070b0 0x7f62101a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621019a5e0 0x7f621019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62101070b0 0x7f62101a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36896/0 (socket says 172.21.15.45:36896) 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6210101fe0 0x7f62101a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:40690/0 (socket says 172.21.15.45:40690) 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620d400640 1 -- 172.21.15.45:0/3575987784 learned_addr learned my addr 172.21.15.45:0/3575987784 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:01.355 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620e800640 1 -- 172.21.15.45:0/3575987784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62101070b0 msgr2=0x7f62101a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620e800640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62101070b0 0x7f62101a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620e800640 1 -- 172.21.15.45:0/3575987784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6210101fe0 msgr2=0x7f62101a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620e800640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6210101fe0 0x7f62101a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620e800640 1 -- 172.21.15.45:0/3575987784 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621019b100 con 0x7f621019a5e0 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.352+0000 7f620d400640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62101070b0 0x7f62101a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.353+0000 7f620de00640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6210101fe0 0x7f62101a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.353+0000 7f620e800640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621019a5e0 0x7f621019aa90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6208009fd0 tx=0x7f620800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.353+0000 7f6206a00640 1 -- 172.21.15.45:0/3575987784 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620800ce20 con 0x7f621019a5e0 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.353+0000 7f6206a00640 1 -- 172.21.15.45:0/3575987784 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6208005020 con 0x7f621019a5e0 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.353+0000 7f620f200640 1 -- 172.21.15.45:0/3575987784 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621019b3f0 con 0x7f621019a5e0 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.353+0000 7f6206a00640 1 -- 172.21.15.45:0/3575987784 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6208010690 con 0x7f621019a5e0 2024-04-25T19:08:01.356 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.353+0000 7f620f200640 1 -- 172.21.15.45:0/3575987784 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62101a7690 con 0x7f621019a5e0 2024-04-25T19:08:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.357+0000 7f6217765740 1 Processor -- start 2024-04-25T19:08:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.357+0000 7f6206a00640 1 -- 172.21.15.45:0/3575987784 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f62080054d0 con 0x7f621019a5e0 2024-04-25T19:08:01.358 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f6217765740 1 -- start start 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f6217765740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c39b0 0x5562ff5c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f6217765740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f6217765740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5ceb90 0x5562ff5d0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f6217765740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5562ff529900 con 0x5562ff5c39b0 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f6217765740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5562ff4cfef0 con 0x5562ff5ceb90 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f6217765740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5562ff5258c0 con 0x5562ff5c42d0 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f620de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f620d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c39b0 0x5562ff5c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f620de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:59484/0 (socket says 172.21.15.45:59484) 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.358+0000 7f620de00640 1 -- 172.21.15.45:0/1990486902 learned_addr learned my addr 172.21.15.45:0/1990486902 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f620d400640 1 --2- 172.21.15.45:0/1990486902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5ceb90 0x5562ff5d0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f620de00640 1 -- 172.21.15.45:0/1990486902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5ceb90 msgr2=0x5562ff5d0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f620de00640 1 --2- 172.21.15.45:0/1990486902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5ceb90 0x5562ff5d0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.361 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f620de00640 1 -- 172.21.15.45:0/1990486902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c39b0 msgr2=0x5562ff5c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f620de00640 1 --2- 172.21.15.45:0/1990486902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c39b0 0x5562ff5c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f620de00640 1 -- 172.21.15.45:0/1990486902 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ff23c210 con 0x5562ff5c42d0 2024-04-25T19:08:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f620d400640 1 --2- 172.21.15.45:0/1990486902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5ceb90 0x5562ff5d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f6206a00640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f61f40a3d80 0x7f61f40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f620de00640 1 --2- 172.21.15.45:0/1990486902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5ce650 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f62000115f0 tx=0x7f6200011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d8519264495715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f6206a00640 1 -- 172.21.15.45:0/3575987784 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f62080d0e30 con 0x7f621019a5e0 2024-04-25T19:08:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.359+0000 7f620d400640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f61f40a3d80 0x7f61f40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.360+0000 7f61ff000640 1 -- 172.21.15.45:0/1990486902 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620001e070 con 0x5562ff5c42d0 2024-04-25T19:08:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.360+0000 7f61ff000640 1 -- 172.21.15.45:0/1990486902 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f620000faf0 con 0x5562ff5c42d0 2024-04-25T19:08:01.362 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.360+0000 7f61ff000640 1 -- 172.21.15.45:0/1990486902 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200019bc0 con 0x5562ff5c42d0 2024-04-25T19:08:01.364 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.361+0000 7f6217765740 1 -- 172.21.15.45:0/1990486902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 msgr2=0x5562ff5ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.361+0000 7f6217765740 1 --2- 172.21.15.45:0/1990486902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5ce650 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f62000115f0 tx=0x7f6200011ab0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.361+0000 7f620d400640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f61f40a3d80 0x7f61f40a6240 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f61f8000c00 tx=0x7f61f8005d90 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.361+0000 7f6217765740 1 -- 172.21.15.45:0/1990486902 shutdown_connections 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.361+0000 7f6217765740 1 --2- 172.21.15.45:0/1990486902 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5ceb90 0x5562ff5d0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.361+0000 7f6217765740 1 --2- 172.21.15.45:0/1990486902 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5ce650 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.361+0000 7f6217765740 1 --2- 172.21.15.45:0/1990486902 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c39b0 0x5562ff5c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.361+0000 7f6217765740 1 -- 172.21.15.45:0/1990486902 >> 172.21.15.45:0/1990486902 conn(0x5562ff5bfec0 msgr2=0x5562ff5c02a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.361+0000 7f6217765740 1 -- 172.21.15.45:0/1990486902 shutdown_connections 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.361+0000 7f6217765740 1 -- 172.21.15.45:0/1990486902 wait complete. 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.362+0000 7f6217765740 1 -- start start 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.362+0000 7f6217765740 1 Processor -- start 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.362+0000 7f6217765740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.362+0000 7f6217765740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.362+0000 7f6217765740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5ceb90 0x5562ff5cc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.365 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f6217765740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5562ff5d17c0 con 0x5562ff5ceb90 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f6217765740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x5562ff4e3310 con 0x5562ff5c39b0 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f6217765740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x5562ff5d1940 con 0x5562ff5c42d0 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f6217765740 10 client.0 _ll_register_callbacks cb 0x5562ff4b1cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36916/0 (socket says 172.21.15.45:36916) 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620de00640 1 -- 172.21.15.45:0/713456387 learned_addr learned my addr 172.21.15.45:0/713456387 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620e800640 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5ceb90 0x5562ff5cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620d400640 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620de00640 1 -- 172.21.15.45:0/713456387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 msgr2=0x5562ff5cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620de00640 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5cc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620de00640 1 -- 172.21.15.45:0/713456387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5ceb90 msgr2=0x5562ff5cc600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620de00640 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5ceb90 0x5562ff5cc600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.366 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620de00640 1 -- 172.21.15.45:0/713456387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ff308620 con 0x5562ff5c39b0 2024-04-25T19:08:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.363+0000 7f620d400640 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:08:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.364+0000 7f620de00640 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cbb80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f620001c040 tx=0x7f6200019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f532d96de613d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:01.367 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.364+0000 7f61f3400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:08:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.364+0000 7f61f3400640 1 -- 172.21.15.45:0/713456387 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200027060 con 0x5562ff5c39b0 2024-04-25T19:08:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.364+0000 7f61f3400640 1 -- 172.21.15.45:0/713456387 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f620000f640 con 0x5562ff5c39b0 2024-04-25T19:08:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.364+0000 7f6217765740 1 -- 172.21.15.45:0/713456387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f620000faf0 con 0x5562ff5c39b0 2024-04-25T19:08:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.364+0000 7f61f3400640 1 -- 172.21.15.45:0/713456387 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f620001e040 con 0x5562ff5c39b0 2024-04-25T19:08:01.369 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.364+0000 7f6217765740 10 client.26015 fetch_fsmap learned FSMap version 32 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.364+0000 7f6217765740 1 -- 172.21.15.45:0/713456387 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562ff5dd9f0 con 0x5562ff5c39b0 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.365+0000 7f61f3400640 1 -- 172.21.15.45:0/713456387 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f620000faf0 con 0x5562ff5c39b0 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.365+0000 7f61f3400640 1 -- 172.21.15.45:0/713456387 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f62000035f0 con 0x5562ff5c39b0 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.365+0000 7f6217765740 10 client.26015 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.365+0000 7f6217765740 10 client.26015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.365+0000 7f6217765740 20 client.26015 populate_metadata read hostname 'smithi045' 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.365+0000 7f6217765740 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5562ff5deda0 0x5562ff5e1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.365+0000 7f6217765740 4 client.26015 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.365+0000 7f6217765740 1 -- 172.21.15.45:0/713456387 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562ff53d8b0 con 0x5562ff5deda0 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.365+0000 7f620d400640 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5562ff5deda0 0x5562ff5e1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.366+0000 7f620d400640 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5562ff5deda0 0x5562ff5e1190 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b625a36f698614c2 server_cookie=97af09766a72c6e3 in_seq=0 out_seq=0 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.366+0000 7f61f3400640 10 client.26015 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.366+0000 7f61f3400640 1 -- 172.21.15.45:0/713456387 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5562ff53d8b0 con 0x5562ff5deda0 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f61f3400640 10 client.26015 handle_command_reply: tid=0 2024-04-25T19:08:01.370 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 client.26015 shutdown 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 -- 172.21.15.45:0/713456387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 msgr2=0x5562ff5cbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cbb80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f620001c040 tx=0x7f6200019640 comp rx=0 tx=0).stop 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 -- 172.21.15.45:0/713456387 shutdown_connections 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5562ff5deda0 0x5562ff5e1190 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5ceb90 0x5562ff5cc600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5cc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 --2- 172.21.15.45:0/713456387 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cbb80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f61f3400640 0 client.26015 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 -- 172.21.15.45:0/713456387 >> 172.21.15.45:0/713456387 conn(0x5562ff5bfec0 msgr2=0x5562ff5c5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 -- 172.21.15.45:0/713456387 shutdown_connections 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 1 -- 172.21.15.45:0/713456387 wait complete. 2024-04-25T19:08:01.371 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.367+0000 7f6217765740 20 client.26015 trim_cache size 0 max 16384 2024-04-25T19:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.396+0000 7f6217765740 1 Processor -- start 2024-04-25T19:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.396+0000 7f6217765740 1 -- start start 2024-04-25T19:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.396+0000 7f6217765740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5bfec0 0x5562ff5da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.396+0000 7f6217765740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.396+0000 7f6217765740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c42d0 0x5562ff5e6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.396+0000 7f6217765740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62101b2620 con 0x5562ff5c42d0 2024-04-25T19:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.396+0000 7f6217765740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f62101afef0 con 0x5562ff5c39b0 2024-04-25T19:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.396+0000 7f6217765740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62101add60 con 0x5562ff5bfec0 2024-04-25T19:08:01.398 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5bfec0 0x5562ff5da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c42d0 0x5562ff5e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5bfec0 0x5562ff5da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:59496/0 (socket says 172.21.15.45:59496) 2024-04-25T19:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36926/0 (socket says 172.21.15.45:36926) 2024-04-25T19:08:01.399 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620de00640 1 -- 172.21.15.45:0/2991901511 learned_addr learned my addr 172.21.15.45:0/2991901511 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620de00640 1 -- 172.21.15.45:0/2991901511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 msgr2=0x5562ff5cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620de00640 1 --2- 172.21.15.45:0/2991901511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.400 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620de00640 1 -- 172.21.15.45:0/2991901511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c42d0 msgr2=0x5562ff5e6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620de00640 1 --2- 172.21.15.45:0/2991901511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c42d0 0x5562ff5e6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620de00640 1 -- 172.21.15.45:0/2991901511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562ff5d16a0 con 0x5562ff5bfec0 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.397+0000 7f620de00640 1 --2- 172.21.15.45:0/2991901511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5bfec0 0x5562ff5da680 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6200009ea0 tx=0x7f62000059b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72e0db2d452961bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.398+0000 7f61fe600640 1 -- 172.21.15.45:0/2991901511 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200016070 con 0x5562ff5bfec0 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.398+0000 7f61fe600640 1 -- 172.21.15.45:0/2991901511 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6200003e00 con 0x5562ff5bfec0 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.398+0000 7f61fe600640 1 -- 172.21.15.45:0/2991901511 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200027040 con 0x5562ff5bfec0 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.398+0000 7f6217765740 1 -- 172.21.15.45:0/2991901511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5bfec0 msgr2=0x5562ff5da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.398+0000 7f6217765740 1 --2- 172.21.15.45:0/2991901511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5bfec0 0x5562ff5da680 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6200009ea0 tx=0x7f62000059b0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.399+0000 7f6217765740 1 -- 172.21.15.45:0/2991901511 shutdown_connections 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.399+0000 7f6217765740 1 --2- 172.21.15.45:0/2991901511 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c42d0 0x5562ff5e6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.399+0000 7f6217765740 1 --2- 172.21.15.45:0/2991901511 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5c39b0 0x5562ff5cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.399+0000 7f6217765740 1 --2- 172.21.15.45:0/2991901511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5bfec0 0x5562ff5da680 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.399+0000 7f6217765740 1 -- 172.21.15.45:0/2991901511 >> 172.21.15.45:0/2991901511 conn(0x5562ff5e1500 msgr2=0x5562ff5cd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:01.401 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.399+0000 7f6217765740 1 -- 172.21.15.45:0/2991901511 shutdown_connections 2024-04-25T19:08:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.399+0000 7f6217765740 1 -- 172.21.15.45:0/2991901511 wait complete. 2024-04-25T19:08:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.399+0000 7f6217765740 1 -- start start 2024-04-25T19:08:01.402 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.399+0000 7f6217765740 1 Processor -- start 2024-04-25T19:08:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f6217765740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5bfec0 0x5562ff5d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f6217765740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c39b0 0x5562ff5d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f6217765740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5d5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f6217765740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62101ae3b0 con 0x5562ff5c39b0 2024-04-25T19:08:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f6217765740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f62101af820 con 0x5562ff5bfec0 2024-04-25T19:08:01.404 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f6217765740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62101aea80 con 0x5562ff5c42d0 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f6217765740 10 client.0 _ll_register_callbacks cb 0x7f6200005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f620de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5bfec0 0x5562ff5d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f620d400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c39b0 0x5562ff5d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f620de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5bfec0 0x5562ff5d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:36930/0 (socket says 172.21.15.45:36930) 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f620e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5d5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f620de00640 1 -- 172.21.15.45:0/4044362847 learned_addr learned my addr 172.21.15.45:0/4044362847 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f620de00640 1 -- 172.21.15.45:0/4044362847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 msgr2=0x5562ff5d5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f620de00640 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5d5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f620de00640 1 -- 172.21.15.45:0/4044362847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c39b0 msgr2=0x5562ff5d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f620de00640 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c39b0 0x5562ff5d4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.400+0000 7f620de00640 1 -- 172.21.15.45:0/4044362847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562ff644410 con 0x5562ff5bfec0 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.401+0000 7f620de00640 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5bfec0 0x5562ff5d47f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6200006010 tx=0x7f620000f440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36997455cee45e2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.401+0000 7f61f3400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.401+0000 7f61f3400640 1 -- 172.21.15.45:0/4044362847 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200016070 con 0x5562ff5bfec0 2024-04-25T19:08:01.405 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.401+0000 7f6217765740 1 -- 172.21.15.45:0/4044362847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6200003e00 con 0x5562ff5bfec0 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.401+0000 7f61f3400640 1 -- 172.21.15.45:0/4044362847 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8571+0+0 (secure 0 0 0) 0x7f6200011240 con 0x5562ff5bfec0 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.401+0000 7f61f3400640 1 -- 172.21.15.45:0/4044362847 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6200011440 con 0x5562ff5bfec0 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.401+0000 7f6217765740 10 client.26021 fetch_fsmap learned FSMap version 32 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.401+0000 7f6217765740 1 -- 172.21.15.45:0/4044362847 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5562ff6456e0 con 0x5562ff5bfec0 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.402+0000 7f61f3400640 1 -- 172.21.15.45:0/4044362847 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6200003e00 con 0x5562ff5bfec0 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.402+0000 7f61f3400640 1 -- 172.21.15.45:0/4044362847 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 32) ==== 8338+0+0 (secure 0 0 0) 0x7f6200019c80 con 0x5562ff5bfec0 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.402+0000 7f6217765740 10 client.26021 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.402+0000 7f6217765740 10 client.26021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.402+0000 7f6217765740 20 client.26021 populate_metadata read hostname 'smithi045' 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.402+0000 7f6217765740 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5562ff5ceb90 0x5562ff647c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.402+0000 7f6217765740 4 client.26021 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.402+0000 7f6217765740 1 -- 172.21.15.45:0/4044362847 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562ff537320 con 0x5562ff5ceb90 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.402+0000 7f620d400640 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5562ff5ceb90 0x5562ff647c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.403+0000 7f620d400640 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5562ff5ceb90 0x5562ff647c50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d649f3f5c95e36f server_cookie=668ff0974dba8218 in_seq=0 out_seq=0 2024-04-25T19:08:01.406 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.403+0000 7f61f3400640 10 client.26021 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.423+0000 7f61f3400640 1 -- 172.21.15.45:0/4044362847 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5562ff537320 con 0x5562ff5ceb90 2024-04-25T19:08:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.423+0000 7f61f3400640 10 client.26021 handle_command_reply: tid=0 2024-04-25T19:08:01.424 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.423+0000 7f6217765740 1 client.26021 shutdown 2024-04-25T19:08:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f6217765740 1 -- 172.21.15.45:0/4044362847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5bfec0 msgr2=0x5562ff5d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f6217765740 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5bfec0 0x5562ff5d47f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6200006010 tx=0x7f620000f440 comp rx=0 tx=0).stop 2024-04-25T19:08:01.425 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f6217765740 1 -- 172.21.15.45:0/4044362847 shutdown_connections 2024-04-25T19:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f6217765740 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x5562ff5ceb90 0x5562ff647c50 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f6217765740 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x5562ff5c42d0 0x5562ff5d5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f6217765740 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5562ff5c39b0 0x5562ff5d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f6217765740 1 --2- 172.21.15.45:0/4044362847 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x5562ff5bfec0 0x5562ff5d47f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f6217765740 1 -- 172.21.15.45:0/4044362847 >> 172.21.15.45:0/4044362847 conn(0x5562ff5e1500 msgr2=0x5562ff5c5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f61f3400640 0 client.26021 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f6217765740 1 -- 172.21.15.45:0/4044362847 shutdown_connections 2024-04-25T19:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.424+0000 7f6217765740 1 -- 172.21.15.45:0/4044362847 wait complete. 2024-04-25T19:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.425+0000 7f6217765740 20 client.26021 trim_cache size 0 max 16384 2024-04-25T19:08:01.426 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.425+0000 7f620f200640 1 -- 172.21.15.45:0/3575987784 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f61f40a3d80 msgr2=0x7f61f40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.425+0000 7f620f200640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f61f40a3d80 0x7f61f40a6240 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f61f8000c00 tx=0x7f61f8005d90 comp rx=0 tx=0).stop 2024-04-25T19:08:01.427 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.425+0000 7f620f200640 1 -- 172.21.15.45:0/3575987784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621019a5e0 msgr2=0x7f621019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.425+0000 7f620f200640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621019a5e0 0x7f621019aa90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6208009fd0 tx=0x7f620800ef10 comp rx=0 tx=0).stop 2024-04-25T19:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.426+0000 7f620f200640 1 -- 172.21.15.45:0/3575987784 shutdown_connections 2024-04-25T19:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.426+0000 7f620f200640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f61f40a3d80 0x7f61f40a6240 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.426+0000 7f620f200640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f621019a5e0 0x7f621019aa90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.426+0000 7f620f200640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f62101070b0 0x7f62101a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.426+0000 7f620f200640 1 --2- 172.21.15.45:0/3575987784 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6210101fe0 0x7f62101a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.426+0000 7f620f200640 1 -- 172.21.15.45:0/3575987784 >> 172.21.15.45:0/3575987784 conn(0x7f62100fd860 msgr2=0x7f62100ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.426+0000 7f620f200640 1 -- 172.21.15.45:0/3575987784 shutdown_connections 2024-04-25T19:08:01.428 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:01.426+0000 7f620f200640 1 -- 172.21.15.45:0/3575987784 wait complete. 2024-04-25T19:08:02.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:02 smithi079 ceph-mon[33002]: pgmap v269: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 434 op/s 2024-04-25T19:08:02.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:02 smithi045 ceph-mon[27776]: pgmap v269: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 434 op/s 2024-04-25T19:08:02.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:02 smithi062 ceph-mon[32844]: pgmap v269: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 434 op/s 2024-04-25T19:08:04.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:04 smithi045 ceph-mon[27776]: pgmap v270: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 857 KiB/s wr, 285 op/s 2024-04-25T19:08:04.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:04 smithi062 ceph-mon[32844]: pgmap v270: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 857 KiB/s wr, 285 op/s 2024-04-25T19:08:04.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:04 smithi079 ceph-mon[33002]: pgmap v270: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 857 KiB/s wr, 285 op/s 2024-04-25T19:08:06.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:06 smithi045 ceph-mon[27776]: pgmap v271: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.3 MiB/s wr, 458 op/s 2024-04-25T19:08:06.496 INFO:tasks.workunit.client.0.smithi045.stdout:All 50000 operations completed A-OK! 2024-04-25T19:08:06.550 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T19:08:06.550 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T19:08:06.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:06 smithi062 ceph-mon[32844]: pgmap v271: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.3 MiB/s wr, 458 op/s 2024-04-25T19:08:06.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:06 smithi079 ceph-mon[33002]: pgmap v271: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 1.3 MiB/s wr, 458 op/s 2024-04-25T19:08:07.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:07 smithi062 ceph-mon[32844]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T19:08:07.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:07 smithi079 ceph-mon[33002]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T19:08:07.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:07 smithi045 ceph-mon[27776]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T19:08:08.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:08 smithi062 ceph-mon[32844]: pgmap v273: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-04-25T19:08:08.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:08 smithi079 ceph-mon[33002]: pgmap v273: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-04-25T19:08:08.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:08 smithi045 ceph-mon[27776]: pgmap v273: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-04-25T19:08:09.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:09 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:09.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:09 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:09.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:09 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:10.585 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:10 smithi079 ceph-mon[33002]: pgmap v274: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-04-25T19:08:10.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:10 smithi062 ceph-mon[32844]: pgmap v274: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-04-25T19:08:10.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:10 smithi045 ceph-mon[27776]: pgmap v274: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-04-25T19:08:12.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:12 smithi062 ceph-mon[32844]: pgmap v275: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.4 MiB/s wr, 339 op/s 2024-04-25T19:08:12.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:12 smithi062 ceph-mon[32844]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T19:08:12.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:12 smithi079 ceph-mon[33002]: pgmap v275: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.4 MiB/s wr, 339 op/s 2024-04-25T19:08:12.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:12 smithi079 ceph-mon[33002]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T19:08:12.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:12 smithi045 ceph-mon[27776]: pgmap v275: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.4 MiB/s wr, 339 op/s 2024-04-25T19:08:12.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:12 smithi045 ceph-mon[27776]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T19:08:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.574+0000 7fdb34c00640 1 -- 172.21.15.45:0/74410052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb301063d0 msgr2=0x7fdb30106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.574+0000 7fdb34c00640 1 --2- 172.21.15.45:0/74410052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb301063d0 0x7fdb30106880 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009900 tx=0x7fdb2802a480 comp rx=0 tx=0).stop 2024-04-25T19:08:14.575 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.574+0000 7fdb34c00640 1 -- 172.21.15.45:0/74410052 shutdown_connections 2024-04-25T19:08:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.574+0000 7fdb34c00640 1 --2- 172.21.15.45:0/74410052 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb301063d0 0x7fdb30106880 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.574+0000 7fdb34c00640 1 --2- 172.21.15.45:0/74410052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301018f0 0x7fdb30105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.574+0000 7fdb34c00640 1 --2- 172.21.15.45:0/74410052 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb30100fd0 0x7fdb301013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.574+0000 7fdb34c00640 1 -- 172.21.15.45:0/74410052 >> 172.21.15.45:0/74410052 conn(0x7fdb30077fd0 msgr2=0x7fdb300783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.574+0000 7fdb34c00640 1 -- 172.21.15.45:0/74410052 shutdown_connections 2024-04-25T19:08:14.576 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.574+0000 7fdb34c00640 1 -- 172.21.15.45:0/74410052 wait complete. 2024-04-25T19:08:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.575+0000 7fdb34c00640 1 Processor -- start 2024-04-25T19:08:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.575+0000 7fdb34c00640 1 -- start start 2024-04-25T19:08:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.575+0000 7fdb34c00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb30100fd0 0x7fdb3019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.575+0000 7fdb34c00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb301018f0 0x7fdb3019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.575+0000 7fdb34c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301063d0 0x7fdb301a3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb34c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30118800 con 0x7fdb301063d0 2024-04-25T19:08:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb34c00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30118680 con 0x7fdb30100fd0 2024-04-25T19:08:14.578 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb34c00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30118980 con 0x7fdb301018f0 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb30100fd0 0x7fdb3019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301063d0 0x7fdb301a3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb30100fd0 0x7fdb3019eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:46264/0 (socket says 172.21.15.45:46264) 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2de00640 1 -- 172.21.15.45:0/3206493717 learned_addr learned my addr 172.21.15.45:0/3206493717 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2d400640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb301018f0 0x7fdb3019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2e800640 1 -- 172.21.15.45:0/3206493717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb301018f0 msgr2=0x7fdb3019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2e800640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb301018f0 0x7fdb3019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2e800640 1 -- 172.21.15.45:0/3206493717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb30100fd0 msgr2=0x7fdb3019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2e800640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb30100fd0 0x7fdb3019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2e800640 1 -- 172.21.15.45:0/3206493717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb301a3d40 con 0x7fdb301063d0 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.576+0000 7fdb2de00640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb30100fd0 0x7fdb3019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.577+0000 7fdb2e800640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301063d0 0x7fdb301a3610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdb2802a9a0 tx=0x7fdb28003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.577+0000 7fdb26a00640 1 -- 172.21.15.45:0/3206493717 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2803f070 con 0x7fdb301063d0 2024-04-25T19:08:14.579 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.577+0000 7fdb34c00640 1 -- 172.21.15.45:0/3206493717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb301a3fd0 con 0x7fdb301063d0 2024-04-25T19:08:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.577+0000 7fdb26a00640 1 -- 172.21.15.45:0/3206493717 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2803bb10 con 0x7fdb301063d0 2024-04-25T19:08:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.577+0000 7fdb26a00640 1 -- 172.21.15.45:0/3206493717 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28035030 con 0x7fdb301063d0 2024-04-25T19:08:14.580 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.577+0000 7fdb34c00640 1 -- 172.21.15.45:0/3206493717 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb301ab8c0 con 0x7fdb301063d0 2024-04-25T19:08:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.580+0000 7fdb37bbf740 1 Processor -- start 2024-04-25T19:08:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.580+0000 7fdb26a00640 1 -- 172.21.15.45:0/3206493717 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdb28042070 con 0x7fdb301063d0 2024-04-25T19:08:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb37bbf740 1 -- start start 2024-04-25T19:08:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8376f820 0x561a8376fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a8377a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 0x561a8377cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561a836d5900 con 0x561a8377aa00 2024-04-25T19:08:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561a8367bef0 con 0x561a83770140 2024-04-25T19:08:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561a836d18c0 con 0x561a8376f820 2024-04-25T19:08:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb2de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a8377a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb2de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a8377a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:46270/0 (socket says 172.21.15.45:46270) 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb2de00640 1 -- 172.21.15.45:0/736720519 learned_addr learned my addr 172.21.15.45:0/736720519 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb2d400640 1 --2- 172.21.15.45:0/736720519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8376f820 0x561a8376fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.581+0000 7fdb2d400640 1 --2- 172.21.15.45:0/736720519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 0x561a8377cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb2de00640 1 -- 172.21.15.45:0/736720519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8376f820 msgr2=0x561a8376fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb2de00640 1 --2- 172.21.15.45:0/736720519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8376f820 0x561a8376fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb2de00640 1 -- 172.21.15.45:0/736720519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 msgr2=0x561a8377cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb2de00640 1 --2- 172.21.15.45:0/736720519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 0x561a8377cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb2de00640 1 -- 172.21.15.45:0/736720519 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a833e8210 con 0x561a83770140 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb2d400640 1 --2- 172.21.15.45:0/736720519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 0x561a8377cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb2de00640 1 --2- 172.21.15.45:0/736720519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a8377a4c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000f8d0 tx=0x7fdb2000fd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd8a9467541ca2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb1f000640 1 -- 172.21.15.45:0/736720519 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20011c20 con 0x561a83770140 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb1f000640 1 -- 172.21.15.45:0/736720519 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb20011dc0 con 0x561a83770140 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb1f000640 1 -- 172.21.15.45:0/736720519 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20018700 con 0x561a83770140 2024-04-25T19:08:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb26a00640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdb140a3d10 0x7fdb140a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.582+0000 7fdb26a00640 1 -- 172.21.15.45:0/3206493717 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fdb280f0d80 con 0x7fdb301063d0 2024-04-25T19:08:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb2d400640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdb140a3d10 0x7fdb140a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb37bbf740 1 -- 172.21.15.45:0/736720519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 msgr2=0x561a8377a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/736720519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a8377a4c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000f8d0 tx=0x7fdb2000fd90 comp rx=0 tx=0).stop 2024-04-25T19:08:14.588 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb37bbf740 1 -- 172.21.15.45:0/736720519 shutdown_connections 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/736720519 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 0x561a8377cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/736720519 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a8377a4c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/736720519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8376f820 0x561a8376fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb37bbf740 1 -- 172.21.15.45:0/736720519 >> 172.21.15.45:0/736720519 conn(0x561a8376bd30 msgr2=0x561a8376c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb37bbf740 1 -- 172.21.15.45:0/736720519 shutdown_connections 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb37bbf740 1 -- 172.21.15.45:0/736720519 wait complete. 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.583+0000 7fdb2d400640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdb140a3d10 0x7fdb140a61d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x561a8376d150 tx=0x7fdb18004730 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.584+0000 7fdb37bbf740 1 -- start start 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.584+0000 7fdb37bbf740 1 Processor -- start 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.584+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8376f820 0x561a837778f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.584+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a83777e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.584+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 0x561a83778370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.584+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561a8377d630 con 0x561a8377aa00 2024-04-25T19:08:14.589 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x561a8368f310 con 0x561a83770140 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x561a8377d7b0 con 0x561a8376f820 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb37bbf740 10 client.0 _ll_register_callbacks cb 0x561a8365dcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb2d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a83777e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb2d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a83777e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:46278/0 (socket says 172.21.15.45:46278) 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb2d400640 1 -- 172.21.15.45:0/652444006 learned_addr learned my addr 172.21.15.45:0/652444006 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb2e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 0x561a83778370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb2de00640 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8376f820 0x561a837778f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb2d400640 1 -- 172.21.15.45:0/652444006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8376f820 msgr2=0x561a837778f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb2d400640 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8376f820 0x561a837778f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb2d400640 1 -- 172.21.15.45:0/652444006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 msgr2=0x561a83778370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb2d400640 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 0x561a83778370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.585+0000 7fdb2d400640 1 -- 172.21.15.45:0/652444006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a834b4620 con 0x561a83770140 2024-04-25T19:08:14.590 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.586+0000 7fdb2d400640 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a83777e30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdb1801fc10 tx=0x7fdb18005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52477133d795b007 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.586+0000 7fdb13400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.586+0000 7fdb13400640 1 -- 172.21.15.45:0/652444006 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1802d060 con 0x561a83770140 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.586+0000 7fdb13400640 1 -- 172.21.15.45:0/652444006 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fdb1801f0d0 con 0x561a83770140 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.586+0000 7fdb37bbf740 1 -- 172.21.15.45:0/652444006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb20011dc0 con 0x561a83770140 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.586+0000 7fdb13400640 1 -- 172.21.15.45:0/652444006 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb18009a80 con 0x561a83770140 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.586+0000 7fdb37bbf740 10 client.26033 fetch_fsmap learned FSMap version 34 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.586+0000 7fdb37bbf740 1 -- 172.21.15.45:0/652444006 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a83789800 con 0x561a83770140 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.586+0000 7fdb13400640 1 -- 172.21.15.45:0/652444006 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb20011dc0 con 0x561a83770140 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.587+0000 7fdb13400640 1 -- 172.21.15.45:0/652444006 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fdb18028890 con 0x561a83770140 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.587+0000 7fdb37bbf740 10 client.26033 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.587+0000 7fdb37bbf740 10 client.26033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.587+0000 7fdb37bbf740 20 client.26033 populate_metadata read hostname 'smithi045' 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.587+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561a8378ad10 0x561a8378d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.591 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.587+0000 7fdb37bbf740 4 client.26033 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.587+0000 7fdb37bbf740 1 -- 172.21.15.45:0/652444006 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a836e98b0 con 0x561a8378ad10 2024-04-25T19:08:14.592 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.587+0000 7fdb2de00640 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561a8378ad10 0x561a8378d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.588+0000 7fdb2de00640 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561a8378ad10 0x561a8378d100 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58787f993c21d038 server_cookie=c0e3b4d4475e17a0 in_seq=0 out_seq=0 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.588+0000 7fdb13400640 10 client.26033 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb13400640 1 -- 172.21.15.45:0/652444006 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a836e98b0 con 0x561a8378ad10 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb13400640 10 client.26033 handle_command_reply: tid=0 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 client.26033 shutdown 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 -- 172.21.15.45:0/652444006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 msgr2=0x561a83777e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a83777e30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdb1801fc10 tx=0x7fdb18005f90 comp rx=0 tx=0).stop 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 -- 172.21.15.45:0/652444006 shutdown_connections 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561a8378ad10 0x561a8378d100 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8377aa00 0x561a83778370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.593 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a83770140 0x561a83777e30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/652444006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8376f820 0x561a837778f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 -- 172.21.15.45:0/652444006 >> 172.21.15.45:0/652444006 conn(0x561a8376bd30 msgr2=0x561a83771a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb13400640 0 client.26033 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 -- 172.21.15.45:0/652444006 shutdown_connections 2024-04-25T19:08:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 1 -- 172.21.15.45:0/652444006 wait complete. 2024-04-25T19:08:14.594 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.589+0000 7fdb37bbf740 20 client.26033 trim_cache size 0 max 16384 2024-04-25T19:08:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.606+0000 7fdb37bbf740 1 Processor -- start 2024-04-25T19:08:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.606+0000 7fdb37bbf740 1 -- start start 2024-04-25T19:08:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.606+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8376f820 0x561a837864f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.606+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a83770140 0x561a8376b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.607 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.606+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8377aa00 0x561a83792a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.606+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301b6c40 con 0x561a8376f820 2024-04-25T19:08:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.606+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301b44b0 con 0x561a8377aa00 2024-04-25T19:08:14.608 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.606+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301b2320 con 0x561a83770140 2024-04-25T19:08:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.606+0000 7fdb2de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8376f820 0x561a837864f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.606+0000 7fdb2e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8377aa00 0x561a83792a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2d400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a83770140 0x561a8376b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.609 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8376f820 0x561a837864f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51662/0 (socket says 172.21.15.45:51662) 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2de00640 1 -- 172.21.15.45:0/3317817149 learned_addr learned my addr 172.21.15.45:0/3317817149 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2d400640 1 -- 172.21.15.45:0/3317817149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8377aa00 msgr2=0x561a83792a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2d400640 1 --2- 172.21.15.45:0/3317817149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8377aa00 0x561a83792a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2d400640 1 -- 172.21.15.45:0/3317817149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8376f820 msgr2=0x561a837864f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2d400640 1 --2- 172.21.15.45:0/3317817149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8376f820 0x561a837864f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2d400640 1 -- 172.21.15.45:0/3317817149 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a8376ff70 con 0x561a83770140 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2e800640 1 --2- 172.21.15.45:0/3317817149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8377aa00 0x561a83792a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2de00640 1 --2- 172.21.15.45:0/3317817149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8376f820 0x561a837864f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb2d400640 1 --2- 172.21.15.45:0/3317817149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a83770140 0x561a8376b610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdb1801ffd0 tx=0x7fdb1801f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8076ed97feef3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.607+0000 7fdb1e600640 1 -- 172.21.15.45:0/3317817149 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1801f190 con 0x561a83770140 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb1e600640 1 -- 172.21.15.45:0/3317817149 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb180054d0 con 0x561a83770140 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb1e600640 1 -- 172.21.15.45:0/3317817149 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb180283e0 con 0x561a83770140 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3317817149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a83770140 msgr2=0x561a8376b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:14.610 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/3317817149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a83770140 0x561a8376b610 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fdb1801ffd0 tx=0x7fdb1801f850 comp rx=0 tx=0).stop 2024-04-25T19:08:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3317817149 shutdown_connections 2024-04-25T19:08:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/3317817149 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8377aa00 0x561a83792a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/3317817149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a83770140 0x561a8376b610 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/3317817149 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a8376f820 0x561a837864f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3317817149 >> 172.21.15.45:0/3317817149 conn(0x561a8376bd30 msgr2=0x561a83772c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3317817149 shutdown_connections 2024-04-25T19:08:14.611 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.608+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3317817149 wait complete. 2024-04-25T19:08:14.613 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.609+0000 7fdb37bbf740 1 -- start start 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.609+0000 7fdb37bbf740 1 Processor -- start 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.609+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8376f820 0x561a8377f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.609+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a83770140 0x561a8377ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.609+0000 7fdb37bbf740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8377aa00 0x561a83780450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.609+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301b2970 con 0x561a83770140 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.609+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301b3de0 con 0x561a8376f820 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.609+0000 7fdb37bbf740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdb301b3040 con 0x561a8377aa00 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb37bbf740 10 client.0 _ll_register_callbacks cb 0x7fdb1801f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8376f820 0x561a8377f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2de00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8376f820 0x561a8377f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:46298/0 (socket says 172.21.15.45:46298) 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2de00640 1 -- 172.21.15.45:0/3934040320 learned_addr learned my addr 172.21.15.45:0/3934040320 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2d400640 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a83770140 0x561a8377ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2e800640 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8377aa00 0x561a83780450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.614 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2de00640 1 -- 172.21.15.45:0/3934040320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8377aa00 msgr2=0x561a83780450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2de00640 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8377aa00 0x561a83780450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2de00640 1 -- 172.21.15.45:0/3934040320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a83770140 msgr2=0x561a8377ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2de00640 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a83770140 0x561a8377ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2de00640 1 -- 172.21.15.45:0/3934040320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a837ef300 con 0x561a8376f820 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb2de00640 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8376f820 0x561a8377f9d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x561a83770dc0 tx=0x7fdb20019f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=701ad758a3f64451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.610+0000 7fdb13400640 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.611+0000 7fdb13400640 1 -- 172.21.15.45:0/3934040320 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2002bd60 con 0x561a8376f820 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.611+0000 7fdb13400640 1 -- 172.21.15.45:0/3934040320 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fdb20003970 con 0x561a8376f820 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.611+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3934040320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb180054d0 con 0x561a8376f820 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.611+0000 7fdb13400640 1 -- 172.21.15.45:0/3934040320 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb20003b70 con 0x561a8376f820 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.611+0000 7fdb37bbf740 10 client.26039 fetch_fsmap learned FSMap version 34 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.611+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3934040320 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x561a837f0610 con 0x561a8376f820 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.611+0000 7fdb13400640 1 -- 172.21.15.45:0/3934040320 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb180054d0 con 0x561a8376f820 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.612+0000 7fdb13400640 1 -- 172.21.15.45:0/3934040320 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 34) ==== 8346+0+0 (secure 0 0 0) 0x7fdb20005140 con 0x561a8376f820 2024-04-25T19:08:14.615 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.612+0000 7fdb37bbf740 10 client.26039 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T19:08:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.612+0000 7fdb37bbf740 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.612+0000 7fdb37bbf740 20 client.26039 populate_metadata read hostname 'smithi045' 2024-04-25T19:08:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.612+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561a837f0bf0 0x561a837f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.612+0000 7fdb37bbf740 4 client.26039 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:08:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.612+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3934040320 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a836e3320 con 0x561a837f0bf0 2024-04-25T19:08:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.612+0000 7fdb2d400640 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561a837f0bf0 0x561a837f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.612+0000 7fdb2d400640 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561a837f0bf0 0x561a837f2fe0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e8b58d3a2bc8a44 server_cookie=3764a99905e30d58 in_seq=0 out_seq=0 2024-04-25T19:08:14.616 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:14.612+0000 7fdb13400640 10 client.26039 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:14.616 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:14 smithi062 ceph-mon[32844]: pgmap v277: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-04-25T19:08:14.616 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:14 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:14.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:14 smithi079 ceph-mon[33002]: pgmap v277: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-04-25T19:08:14.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:14 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:14.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:14 smithi045 ceph-mon[27776]: pgmap v277: 97 pgs: 97 active+clean; 108 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-04-25T19:08:14.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:14 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:16.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:16 smithi062 ceph-mon[32844]: pgmap v278: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 306 op/s 2024-04-25T19:08:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.665+0000 7fdb13400640 1 -- 172.21.15.45:0/3934040320 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a836e3320 con 0x561a837f0bf0 2024-04-25T19:08:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.665+0000 7fdb13400640 10 client.26039 handle_command_reply: tid=0 2024-04-25T19:08:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.665+0000 7fdb37bbf740 1 client.26039 shutdown 2024-04-25T19:08:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.666+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3934040320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8376f820 msgr2=0x561a8377f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.666+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8376f820 0x561a8377f9d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x561a83770dc0 tx=0x7fdb20019f90 comp rx=0 tx=0).stop 2024-04-25T19:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.667+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3934040320 shutdown_connections 2024-04-25T19:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.667+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x561a837f0bf0 0x561a837f2fe0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.667+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x561a8377aa00 0x561a83780450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.667+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561a83770140 0x561a8377ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.667+0000 7fdb37bbf740 1 --2- 172.21.15.45:0/3934040320 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x561a8376f820 0x561a8377f9d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.667+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3934040320 >> 172.21.15.45:0/3934040320 conn(0x561a8376bd30 msgr2=0x561a837760f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.668+0000 7fdb13400640 0 client.26039 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.668+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3934040320 shutdown_connections 2024-04-25T19:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.668+0000 7fdb37bbf740 1 -- 172.21.15.45:0/3934040320 wait complete. 2024-04-25T19:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.669+0000 7fdb37bbf740 20 client.26039 trim_cache size 0 max 16384 2024-04-25T19:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:08:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 -- 172.21.15.45:0/3206493717 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdb140a3d10 msgr2=0x7fdb140a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdb140a3d10 0x7fdb140a61d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x561a8376d150 tx=0x7fdb18004730 comp rx=0 tx=0).stop 2024-04-25T19:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 -- 172.21.15.45:0/3206493717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301063d0 msgr2=0x7fdb301a3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301063d0 0x7fdb301a3610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdb2802a9a0 tx=0x7fdb28003b90 comp rx=0 tx=0).stop 2024-04-25T19:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 -- 172.21.15.45:0/3206493717 shutdown_connections 2024-04-25T19:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fdb140a3d10 0x7fdb140a61d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdb301063d0 0x7fdb301a3610 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdb301018f0 0x7fdb3019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 --2- 172.21.15.45:0/3206493717 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdb30100fd0 0x7fdb3019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 -- 172.21.15.45:0/3206493717 >> 172.21.15.45:0/3206493717 conn(0x7fdb30077fd0 msgr2=0x7fdb30103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.670+0000 7fdb34c00640 1 -- 172.21.15.45:0/3206493717 shutdown_connections 2024-04-25T19:08:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:16.671+0000 7fdb34c00640 1 -- 172.21.15.45:0/3206493717 wait complete. 2024-04-25T19:08:16.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:16 smithi079 ceph-mon[33002]: pgmap v278: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 306 op/s 2024-04-25T19:08:16.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:16 smithi045 ceph-mon[27776]: pgmap v278: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.8 MiB/s wr, 306 op/s 2024-04-25T19:08:18.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:18 smithi062 ceph-mon[32844]: pgmap v279: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 269 op/s 2024-04-25T19:08:18.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:18 smithi079 ceph-mon[33002]: pgmap v279: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 269 op/s 2024-04-25T19:08:18.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:18 smithi045 ceph-mon[27776]: pgmap v279: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 269 op/s 2024-04-25T19:08:20.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:20 smithi062 ceph-mon[32844]: pgmap v280: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 269 op/s 2024-04-25T19:08:20.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:20 smithi079 ceph-mon[33002]: pgmap v280: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 269 op/s 2024-04-25T19:08:20.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:20 smithi045 ceph-mon[27776]: pgmap v280: 97 pgs: 97 active+clean; 124 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 269 op/s 2024-04-25T19:08:21.920 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-25T19:08:21.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T19:08:22.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:22 smithi062 ceph-mon[32844]: pgmap v281: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 467 op/s 2024-04-25T19:08:22.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:22 smithi079 ceph-mon[33002]: pgmap v281: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 467 op/s 2024-04-25T19:08:22.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:22 smithi045 ceph-mon[27776]: pgmap v281: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 467 op/s 2024-04-25T19:08:23.149 DEBUG:teuthology.parallel:result is None 2024-04-25T19:08:23.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T19:08:23.244 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T19:08:23.244 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T19:08:23.255 INFO:tasks.quiescer:joining Quiescers 2024-04-25T19:08:23.255 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-25T19:08:23.256 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:08:23.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:23 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:23.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:23 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:23.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:23 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:23.716 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:08:24.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:24 smithi045 ceph-mon[27776]: pgmap v282: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 404 op/s 2024-04-25T19:08:24.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:24 smithi062 ceph-mon[32844]: pgmap v282: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 404 op/s 2024-04-25T19:08:24.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:24 smithi079 ceph-mon[33002]: pgmap v282: 97 pgs: 97 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 404 op/s 2024-04-25T19:08:26.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.046+0000 7fa3a5400700 1 -- 172.21.15.45:0/546895691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a0109c80 msgr2=0x7fa3a010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.046+0000 7fa3a5400700 1 --2- 172.21.15.45:0/546895691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a0109c80 0x7fa3a010dcd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa3880099c0 tx=0x7fa388009cd0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.046+0000 7fa3a5400700 1 -- 172.21.15.45:0/546895691 shutdown_connections 2024-04-25T19:08:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.046+0000 7fa3a5400700 1 --2- 172.21.15.45:0/546895691 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a01092d0 0x7fa3a01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.046+0000 7fa3a5400700 1 --2- 172.21.15.45:0/546895691 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a010e3b0 0x7fa3a0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.046+0000 7fa3a5400700 1 --2- 172.21.15.45:0/546895691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a0109c80 0x7fa3a010dcd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.046+0000 7fa3a5400700 1 -- 172.21.15.45:0/546895691 >> 172.21.15.45:0/546895691 conn(0x7fa3a0076210 msgr2=0x7fa3a0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.046+0000 7fa3a5400700 1 -- 172.21.15.45:0/546895691 shutdown_connections 2024-04-25T19:08:26.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.046+0000 7fa3a5400700 1 -- 172.21.15.45:0/546895691 wait complete. 2024-04-25T19:08:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.048+0000 7fa3a5400700 1 Processor -- start 2024-04-25T19:08:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.048+0000 7fa3a5400700 1 -- start start 2024-04-25T19:08:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.048+0000 7fa3a5400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a01092d0 0x7fa3a01a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.048+0000 7fa3a5400700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a0109c80 0x7fa3a01a5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.048+0000 7fa3a5400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a010e3b0 0x7fa3a019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.048+0000 7fa3a5400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0119270 con 0x7fa3a01092d0 2024-04-25T19:08:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.048+0000 7fa3a5400700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01190f0 con 0x7fa3a010e3b0 2024-04-25T19:08:26.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.048+0000 7fa3a5400700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01193f0 con 0x7fa3a0109c80 2024-04-25T19:08:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa39e800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a01092d0 0x7fa3a01a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa39f200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a010e3b0 0x7fa3a019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa39e800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a01092d0 0x7fa3a01a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36356/0 (socket says 172.21.15.45:36356) 2024-04-25T19:08:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa39f200700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a010e3b0 0x7fa3a019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41078/0 (socket says 172.21.15.45:41078) 2024-04-25T19:08:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa397e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a0109c80 0x7fa3a01a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa39e800700 1 -- 172.21.15.45:0/631187732 learned_addr learned my addr 172.21.15.45:0/631187732 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa397e00700 1 -- 172.21.15.45:0/631187732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a010e3b0 msgr2=0x7fa3a019f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.052 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa397e00700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a010e3b0 0x7fa3a019f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa397e00700 1 -- 172.21.15.45:0/631187732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a01092d0 msgr2=0x7fa3a01a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa397e00700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a01092d0 0x7fa3a01a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa397e00700 1 -- 172.21.15.45:0/631187732 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a019fd50 con 0x7fa3a0109c80 2024-04-25T19:08:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.049+0000 7fa39f200700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a010e3b0 0x7fa3a019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.050+0000 7fa397e00700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a0109c80 0x7fa3a01a5970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa3880098e0 tx=0x7fa3880055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.050+0000 7fa397400700 1 -- 172.21.15.45:0/631187732 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388026070 con 0x7fa3a0109c80 2024-04-25T19:08:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.050+0000 7fa397400700 1 -- 172.21.15.45:0/631187732 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38800f460 con 0x7fa3a0109c80 2024-04-25T19:08:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.050+0000 7fa3a5400700 1 -- 172.21.15.45:0/631187732 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a019ffe0 con 0x7fa3a0109c80 2024-04-25T19:08:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.050+0000 7fa397400700 1 -- 172.21.15.45:0/631187732 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388009e60 con 0x7fa3a0109c80 2024-04-25T19:08:26.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.051+0000 7fa3a5400700 1 -- 172.21.15.45:0/631187732 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a01ac300 con 0x7fa3a0109c80 2024-04-25T19:08:26.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.052+0000 7fa3a5400700 1 -- 172.21.15.45:0/631187732 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a0069730 con 0x7fa3a0109c80 2024-04-25T19:08:26.056 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.054+0000 7fa397400700 1 -- 172.21.15.45:0/631187732 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fa388029020 con 0x7fa3a0109c80 2024-04-25T19:08:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.056+0000 7fa397400700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fa3840a3e30 0x7fa3840a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.056+0000 7fa397400700 1 -- 172.21.15.45:0/631187732 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7fa3880d1450 con 0x7fa3a0109c80 2024-04-25T19:08:26.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.056+0000 7fa39e800700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fa3840a3e30 0x7fa3840a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:26.059 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.057+0000 7fa39e800700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fa3840a3e30 0x7fa3840a62f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa390007c90 tx=0x7fa390007480 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:26.060 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.058+0000 7fa397400700 1 -- 172.21.15.45:0/631187732 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa388013070 con 0x7fa3a0109c80 2024-04-25T19:08:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.341+0000 7fa3a5400700 1 -- 172.21.15.45:0/631187732 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3a002df80 con 0x7fa3a0109c80 2024-04-25T19:08:26.344 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.342+0000 7fa397400700 1 -- 172.21.15.45:0/631187732 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11853 (secure 0 0 0) 0x7fa38800f610 con 0x7fa3a0109c80 2024-04-25T19:08:26.345 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:08:26.345 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T19:08:22.141951+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25789":{"gid":25789,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/620692737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":620692737},{"type":"v1","addr":"172.21.15.79:6837","nonce":620692737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24512]},"id":1}]} 2024-04-25T19:08:26.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.345+0000 7fa3a5400700 1 -- 172.21.15.45:0/631187732 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fa3840a3e30 msgr2=0x7fa3840a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.345+0000 7fa3a5400700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fa3840a3e30 0x7fa3840a62f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa390007c90 tx=0x7fa390007480 comp rx=0 tx=0).stop 2024-04-25T19:08:26.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.345+0000 7fa3a5400700 1 -- 172.21.15.45:0/631187732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a0109c80 msgr2=0x7fa3a01a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:26.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.345+0000 7fa3a5400700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a0109c80 0x7fa3a01a5970 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa3880098e0 tx=0x7fa3880055f0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.348 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.346+0000 7fa3a5400700 1 -- 172.21.15.45:0/631187732 shutdown_connections 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.346+0000 7fa3a5400700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fa3840a3e30 0x7fa3840a62f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.346+0000 7fa3a5400700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fa3a010e3b0 0x7fa3a019f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.346+0000 7fa3a5400700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3a01092d0 0x7fa3a01a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.346+0000 7fa3a5400700 1 --2- 172.21.15.45:0/631187732 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3a0109c80 0x7fa3a01a5970 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.346+0000 7fa3a5400700 1 -- 172.21.15.45:0/631187732 >> 172.21.15.45:0/631187732 conn(0x7fa3a0076210 msgr2=0x7fa3a0077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.346+0000 7fa3a5400700 1 -- 172.21.15.45:0/631187732 shutdown_connections 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:26.346+0000 7fa3a5400700 1 -- 172.21.15.45:0/631187732 wait complete. 2024-04-25T19:08:26.349 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 35 2024-04-25T19:08:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:26 smithi062 ceph-mon[32844]: pgmap v283: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 500 op/s 2024-04-25T19:08:26.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:26 smithi079 ceph-mon[33002]: pgmap v283: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 500 op/s 2024-04-25T19:08:26.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:26 smithi045 ceph-mon[27776]: pgmap v283: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.6 MiB/s wr, 500 op/s 2024-04-25T19:08:27.150 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24524 quiesce db --cancel --all' 2024-04-25T19:08:27.150 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24524 quiesce db --cancel --all 2024-04-25T19:08:27.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:27 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/631187732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:08:27.603 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:08:27.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:27 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/631187732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:08:27.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:27 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/631187732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:08:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:28 smithi062 ceph-mon[32844]: pgmap v284: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 384 op/s 2024-04-25T19:08:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:28 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:28 smithi079 ceph-mon[33002]: pgmap v284: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 384 op/s 2024-04-25T19:08:28.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:28 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:28.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:28 smithi045 ceph-mon[27776]: pgmap v284: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 384 op/s 2024-04-25T19:08:28.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:28 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:08:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.652+0000 7f4135e00700 1 -- 172.21.15.45:0/2360550624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130109fa0 msgr2=0x7f4130111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.652+0000 7f4135e00700 1 --2- 172.21.15.45:0/2360550624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130109fa0 0x7f4130111b70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f412800b0d0 tx=0x7f412800b3e0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.653+0000 7f4135e00700 1 -- 172.21.15.45:0/2360550624 shutdown_connections 2024-04-25T19:08:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.653+0000 7f4135e00700 1 --2- 172.21.15.45:0/2360550624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4130104ec0 0x7f41301052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.653+0000 7f4135e00700 1 --2- 172.21.15.45:0/2360550624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130109fa0 0x7f4130111b70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.653+0000 7f4135e00700 1 --2- 172.21.15.45:0/2360550624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4130105870 0x7f41301098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.653+0000 7f4135e00700 1 -- 172.21.15.45:0/2360550624 >> 172.21.15.45:0/2360550624 conn(0x7f4130100750 msgr2=0x7f4130102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.653+0000 7f4135e00700 1 -- 172.21.15.45:0/2360550624 shutdown_connections 2024-04-25T19:08:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.653+0000 7f4135e00700 1 -- 172.21.15.45:0/2360550624 wait complete. 2024-04-25T19:08:29.656 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.654+0000 7f4135e00700 1 Processor -- start 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.654+0000 7f4135e00700 1 -- start start 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.654+0000 7f4135e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4130104ec0 0x7f4130076f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.654+0000 7f4135e00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4130105870 0x7f4130077490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.654+0000 7f4135e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130109fa0 0x7f41300779d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.654+0000 7f4135e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4130114eb0 con 0x7f4130109fa0 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.654+0000 7f4135e00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f4130114d30 con 0x7f4130104ec0 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.654+0000 7f4135e00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4130115030 con 0x7f4130105870 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.655+0000 7f412ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4130104ec0 0x7f4130076f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.655+0000 7f412f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130109fa0 0x7f41300779d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.655+0000 7f412ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4130104ec0 0x7f4130076f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41092/0 (socket says 172.21.15.45:41092) 2024-04-25T19:08:29.658 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.655+0000 7f412ea00700 1 -- 172.21.15.45:0/583202445 learned_addr learned my addr 172.21.15.45:0/583202445 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.655+0000 7f4127e00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4130105870 0x7f4130077490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.655+0000 7f412ea00700 1 -- 172.21.15.45:0/583202445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4130105870 msgr2=0x7f4130077490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.655+0000 7f412ea00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4130105870 0x7f4130077490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.655+0000 7f412ea00700 1 -- 172.21.15.45:0/583202445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130109fa0 msgr2=0x7f41300779d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.655+0000 7f412ea00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130109fa0 0x7f41300779d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.655+0000 7f412ea00700 1 -- 172.21.15.45:0/583202445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41301ab9d0 con 0x7f4130104ec0 2024-04-25T19:08:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.656+0000 7f412ea00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4130104ec0 0x7f4130076f50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f412000eb30 tx=0x7f412000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.659 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.656+0000 7f4134a00700 1 -- 172.21.15.45:0/583202445 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412000cbd0 con 0x7f4130104ec0 2024-04-25T19:08:29.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.656+0000 7f4135e00700 1 -- 172.21.15.45:0/583202445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41301abcc0 con 0x7f4130104ec0 2024-04-25T19:08:29.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.656+0000 7f4134a00700 1 -- 172.21.15.45:0/583202445 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41200044d0 con 0x7f4130104ec0 2024-04-25T19:08:29.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.656+0000 7f4134a00700 1 -- 172.21.15.45:0/583202445 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120010430 con 0x7f4130104ec0 2024-04-25T19:08:29.660 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.656+0000 7f4135e00700 1 -- 172.21.15.45:0/583202445 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41301ac1f0 con 0x7f4130104ec0 2024-04-25T19:08:29.662 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.660+0000 7f4134a00700 1 -- 172.21.15.45:0/583202445 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f4120003e90 con 0x7f4130104ec0 2024-04-25T19:08:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.661+0000 7f4134a00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f41140a3de0 0x7f41140a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.661+0000 7f41406d1b80 1 Processor -- start 2024-04-25T19:08:29.663 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.661+0000 7f4134a00700 1 -- 172.21.15.45:0/583202445 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f41200d0ae0 con 0x7f4130104ec0 2024-04-25T19:08:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.661+0000 7f41406d1b80 1 -- start start 2024-04-25T19:08:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.661+0000 7f4127e00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f41140a3de0 0x7f41140a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d79890 0x55e2f5d79c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d84b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 0x55e2f5d87450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f41406d1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5cccf00 con 0x55e2f5d85060 2024-04-25T19:08:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f41406d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5c737a0 con 0x55e2f5d79890 2024-04-25T19:08:29.664 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f41406d1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5cc8ec0 con 0x55e2f5d7a1b0 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f412f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 0x55e2f5d87450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f412ea00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d79890 0x55e2f5d79c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f412f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 0x55e2f5d87450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36386/0 (socket says 172.21.15.45:36386) 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f412f400700 1 -- 172.21.15.45:0/1437716348 learned_addr learned my addr 172.21.15.45:0/1437716348 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f4127e00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f41140a3de0 0x7f41140a62a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41300786c0 tx=0x7f4118009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.662+0000 7f4127e00700 1 --2- 172.21.15.45:0/1437716348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d84b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.663+0000 7f412ea00700 1 -- 172.21.15.45:0/1437716348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 msgr2=0x55e2f5d84b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.663+0000 7f412ea00700 1 --2- 172.21.15.45:0/1437716348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d84b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.663+0000 7f412ea00700 1 -- 172.21.15.45:0/1437716348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 msgr2=0x55e2f5d87450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.663+0000 7f412ea00700 1 --2- 172.21.15.45:0/1437716348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 0x55e2f5d87450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.663+0000 7f412ea00700 1 -- 172.21.15.45:0/1437716348 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f5c696f0 con 0x55e2f5d79890 2024-04-25T19:08:29.666 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.663+0000 7f412f400700 1 --2- 172.21.15.45:0/1437716348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 0x55e2f5d87450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:29.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.663+0000 7f412ea00700 1 --2- 172.21.15.45:0/1437716348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d79890 0x55e2f5d79c70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f41200ce1a0 tx=0x7f412009c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=454165d3fc0438eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.663+0000 7f4124c00700 1 -- 172.21.15.45:0/1437716348 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412000ce90 con 0x55e2f5d79890 2024-04-25T19:08:29.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.663+0000 7f4124c00700 1 -- 172.21.15.45:0/1437716348 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4120009500 con 0x55e2f5d79890 2024-04-25T19:08:29.667 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.663+0000 7f4124c00700 1 -- 172.21.15.45:0/1437716348 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120099060 con 0x55e2f5d79890 2024-04-25T19:08:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.664+0000 7f41406d1b80 1 -- 172.21.15.45:0/1437716348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d79890 msgr2=0x55e2f5d79c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.668 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.664+0000 7f41406d1b80 1 --2- 172.21.15.45:0/1437716348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d79890 0x55e2f5d79c70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f41200ce1a0 tx=0x7f412009c350 comp rx=0 tx=0).stop 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.664+0000 7f41406d1b80 1 -- 172.21.15.45:0/1437716348 shutdown_connections 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.664+0000 7f41406d1b80 1 --2- 172.21.15.45:0/1437716348 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d79890 0x55e2f5d79c70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.664+0000 7f41406d1b80 1 --2- 172.21.15.45:0/1437716348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 0x55e2f5d87450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.664+0000 7f41406d1b80 1 --2- 172.21.15.45:0/1437716348 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d84b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.664+0000 7f41406d1b80 1 -- 172.21.15.45:0/1437716348 >> 172.21.15.45:0/1437716348 conn(0x55e2f5ca8610 msgr2=0x55e2f5d75d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.665+0000 7f41406d1b80 1 -- 172.21.15.45:0/1437716348 shutdown_connections 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.665+0000 7f41406d1b80 1 -- 172.21.15.45:0/1437716348 wait complete. 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.665+0000 7f41406d1b80 1 -- start start 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.665+0000 7f41406d1b80 1 Processor -- start 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d847e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d81ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d82020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.669 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f41406d1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5d87b80 con 0x55e2f5d79890 2024-04-25T19:08:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f41406d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5c868d0 con 0x55e2f5d85060 2024-04-25T19:08:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f41406d1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e2f5d87d00 con 0x55e2f5d7a1b0 2024-04-25T19:08:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f41406d1b80 10 client.0 _ll_register_callbacks cb 0x55e2f5c65790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f412f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d82020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f412f400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d82020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41118/0 (socket says 172.21.15.45:41118) 2024-04-25T19:08:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f412ea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f412f400700 1 -- 172.21.15.45:0/2290010562 learned_addr learned my addr 172.21.15.45:0/2290010562 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:29.670 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.666+0000 7f4127e00700 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d81ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.667+0000 7f412f400700 1 -- 172.21.15.45:0/2290010562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 msgr2=0x55e2f5d81ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.667+0000 7f412f400700 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d81ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.667+0000 7f412f400700 1 -- 172.21.15.45:0/2290010562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 msgr2=0x55e2f5d847e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.667+0000 7f412f400700 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d847e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.667+0000 7f412f400700 1 -- 172.21.15.45:0/2290010562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f5c1a940 con 0x55e2f5d85060 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.667+0000 7f412ea00700 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.667+0000 7f412f400700 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d82020 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4128009850 tx=0x7f412800bd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f42655ca3119e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.667+0000 7f411d600700 10 client.0 ms_handle_connect on v2:172.21.15.62:3300/0 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.668+0000 7f411d600700 1 -- 172.21.15.45:0/2290010562 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128003e10 con 0x55e2f5d85060 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.668+0000 7f41406d1b80 1 -- 172.21.15.45:0/2290010562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4120009500 con 0x55e2f5d85060 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.668+0000 7f411d600700 1 -- 172.21.15.45:0/2290010562 <== mon.2 v2:172.21.15.62:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f4128005070 con 0x55e2f5d85060 2024-04-25T19:08:29.672 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.668+0000 7f411d600700 1 -- 172.21.15.45:0/2290010562 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f41280052b0 con 0x55e2f5d85060 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.668+0000 7f41406d1b80 10 client.26057 fetch_fsmap learned FSMap version 36 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.668+0000 7f41406d1b80 1 -- 172.21.15.45:0/2290010562 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e2f5c63b60 con 0x55e2f5d85060 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.668+0000 7f411d600700 1 -- 172.21.15.45:0/2290010562 <== mon.2 v2:172.21.15.62:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4120009500 con 0x55e2f5d85060 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.668+0000 7f411d600700 1 -- 172.21.15.45:0/2290010562 <== mon.2 v2:172.21.15.62:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f4128011070 con 0x55e2f5d85060 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.669+0000 7f41406d1b80 10 client.26057 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.669+0000 7f41406d1b80 10 client.26057 resolve_mds: validated gid 24524 aka daemon mds.b 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.669+0000 7f41406d1b80 20 client.26057 populate_metadata read hostname 'smithi045' 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.669+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e2f5d94890 0x55e2f5d96cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.669+0000 7f41406d1b80 4 client.26057 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.669+0000 7f41406d1b80 1 -- 172.21.15.45:0/2290010562 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2f5cda6f0 con 0x55e2f5d94890 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.669+0000 7f412ea00700 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e2f5d94890 0x55e2f5d96cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.670+0000 7f412ea00700 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e2f5d94890 0x55e2f5d96cb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5fdcee25c3e2ac8 server_cookie=bd38376ff6a125a8 in_seq=0 out_seq=0 2024-04-25T19:08:29.673 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.670+0000 7f411d600700 10 client.26057 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.670+0000 7f411d600700 1 -- 172.21.15.45:0/2290010562 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2f5cda6f0 con 0x55e2f5d94890 2024-04-25T19:08:29.674 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.670+0000 7f411d600700 10 client.26057 handle_command_reply: tid=0 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 client.26057 shutdown 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 -- 172.21.15.45:0/2290010562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 msgr2=0x55e2f5d82020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d82020 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4128009850 tx=0x7f412800bd40 comp rx=0 tx=0).stop 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 -- 172.21.15.45:0/2290010562 shutdown_connections 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e2f5d94890 0x55e2f5d96cb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d82020 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f411d600700 0 client.26057 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2290010562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d81ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 -- 172.21.15.45:0/2290010562 >> 172.21.15.45:0/2290010562 conn(0x55e2f5ca8610 msgr2=0x55e2f5d75930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 -- 172.21.15.45:0/2290010562 shutdown_connections 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.671+0000 7f41406d1b80 1 -- 172.21.15.45:0/2290010562 wait complete. 2024-04-25T19:08:29.675 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.672+0000 7f41406d1b80 20 client.26057 trim_cache size 0 max 16384 2024-04-25T19:08:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.693+0000 7f41406d1b80 1 Processor -- start 2024-04-25T19:08:29.695 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.693+0000 7f41406d1b80 1 -- start start 2024-04-25T19:08:29.696 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.693+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d80ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.693+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d83a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.693+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d8cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.693+0000 7f41406d1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b6380 con 0x55e2f5d79890 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.693+0000 7f41406d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b3bb0 con 0x55e2f5d85060 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.693+0000 7f41406d1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b1a70 con 0x55e2f5d7a1b0 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.694+0000 7f412ea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d80ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.694+0000 7f412ea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d80ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36406/0 (socket says 172.21.15.45:36406) 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.694+0000 7f412ea00700 1 -- 172.21.15.45:0/420819366 learned_addr learned my addr 172.21.15.45:0/420819366 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.694+0000 7f4127e00700 1 --2- 172.21.15.45:0/420819366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d83a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.694+0000 7f412f400700 1 --2- 172.21.15.45:0/420819366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d8cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.694+0000 7f412ea00700 1 -- 172.21.15.45:0/420819366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 msgr2=0x55e2f5d83a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.694+0000 7f412ea00700 1 --2- 172.21.15.45:0/420819366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d83a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.694+0000 7f412ea00700 1 -- 172.21.15.45:0/420819366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 msgr2=0x55e2f5d8cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.694+0000 7f412ea00700 1 --2- 172.21.15.45:0/420819366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d8cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.697 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.694+0000 7f412ea00700 1 -- 172.21.15.45:0/420819366 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2f5d82170 con 0x55e2f5d79890 2024-04-25T19:08:29.698 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.695+0000 7f412f400700 1 --2- 172.21.15.45:0/420819366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d8cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:08:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.695+0000 7f412ea00700 1 --2- 172.21.15.45:0/420819366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d80ca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f412009c6a0 tx=0x7f4120003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3635e88e4196df99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.695+0000 7f411fe00700 1 -- 172.21.15.45:0/420819366 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41200b7070 con 0x55e2f5d79890 2024-04-25T19:08:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.695+0000 7f411fe00700 1 -- 172.21.15.45:0/420819366 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4120009500 con 0x55e2f5d79890 2024-04-25T19:08:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.695+0000 7f411fe00700 1 -- 172.21.15.45:0/420819366 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41200a5660 con 0x55e2f5d79890 2024-04-25T19:08:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.696+0000 7f41406d1b80 1 -- 172.21.15.45:0/420819366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 msgr2=0x55e2f5d80ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.696+0000 7f41406d1b80 1 --2- 172.21.15.45:0/420819366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d80ca0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f412009c6a0 tx=0x7f4120003930 comp rx=0 tx=0).stop 2024-04-25T19:08:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.696+0000 7f41406d1b80 1 -- 172.21.15.45:0/420819366 shutdown_connections 2024-04-25T19:08:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.696+0000 7f41406d1b80 1 --2- 172.21.15.45:0/420819366 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d85060 0x55e2f5d8cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.699 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.696+0000 7f41406d1b80 1 --2- 172.21.15.45:0/420819366 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d79890 0x55e2f5d80ca0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.696+0000 7f41406d1b80 1 --2- 172.21.15.45:0/420819366 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d83a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.696+0000 7f41406d1b80 1 -- 172.21.15.45:0/420819366 >> 172.21.15.45:0/420819366 conn(0x55e2f5ca8610 msgr2=0x55e2f5d83310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.696+0000 7f41406d1b80 1 -- 172.21.15.45:0/420819366 shutdown_connections 2024-04-25T19:08:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.696+0000 7f41406d1b80 1 -- 172.21.15.45:0/420819366 wait complete. 2024-04-25T19:08:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.697+0000 7f41406d1b80 1 -- start start 2024-04-25T19:08:29.700 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.697+0000 7f41406d1b80 1 Processor -- start 2024-04-25T19:08:29.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.697+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d79890 0x55e2f5d8d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.697+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d8b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.697+0000 7f41406d1b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 0x55e2f5d8bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.697+0000 7f41406d1b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b2070 con 0x55e2f5d85060 2024-04-25T19:08:29.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.697+0000 7f41406d1b80 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b34e0 con 0x55e2f5d7a1b0 2024-04-25T19:08:29.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.697+0000 7f41406d1b80 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f41301b2740 con 0x55e2f5d79890 2024-04-25T19:08:29.702 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.697+0000 7f41406d1b80 10 client.0 _ll_register_callbacks cb 0x55e2f5c65790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f4127e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d8b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f4127e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d8b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41148/0 (socket says 172.21.15.45:41148) 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f4127e00700 1 -- 172.21.15.45:0/2263158083 learned_addr learned my addr 172.21.15.45:0/2263158083 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f412ea00700 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d79890 0x55e2f5d8d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f412f400700 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 0x55e2f5d8bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f412ea00700 1 -- 172.21.15.45:0/2263158083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d7a1b0 msgr2=0x55e2f5d8b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f412ea00700 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d8b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f412ea00700 1 -- 172.21.15.45:0/2263158083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 msgr2=0x55e2f5d8bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f412ea00700 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 0x55e2f5d8bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f412ea00700 1 -- 172.21.15.45:0/2263158083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2f5d8c4c0 con 0x55e2f5d79890 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.698+0000 7f4127e00700 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d8b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.699+0000 7f412ea00700 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d79890 0x55e2f5d8d340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4120009230 tx=0x7f4120005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73639527501e1286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.699+0000 7f411d600700 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.699+0000 7f411d600700 1 -- 172.21.15.45:0/2263158083 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41200b7040 con 0x55e2f5d79890 2024-04-25T19:08:29.703 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.699+0000 7f41406d1b80 1 -- 172.21.15.45:0/2263158083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4120009500 con 0x55e2f5d79890 2024-04-25T19:08:29.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.699+0000 7f411d600700 1 -- 172.21.15.45:0/2263158083 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f412009cc50 con 0x55e2f5d79890 2024-04-25T19:08:29.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.699+0000 7f411d600700 1 -- 172.21.15.45:0/2263158083 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f412009d0f0 con 0x55e2f5d79890 2024-04-25T19:08:29.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.699+0000 7f41406d1b80 10 client.26074 fetch_fsmap learned FSMap version 36 2024-04-25T19:08:29.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.699+0000 7f41406d1b80 1 -- 172.21.15.45:0/2263158083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e2f5dfbe00 con 0x55e2f5d79890 2024-04-25T19:08:29.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.700+0000 7f411d600700 1 -- 172.21.15.45:0/2263158083 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4120009500 con 0x55e2f5d79890 2024-04-25T19:08:29.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.700+0000 7f411d600700 1 -- 172.21.15.45:0/2263158083 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f41200bbbb0 con 0x55e2f5d79890 2024-04-25T19:08:29.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.700+0000 7f41406d1b80 10 client.26074 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T19:08:29.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.700+0000 7f41406d1b80 10 client.26074 resolve_mds: validated gid 24524 aka daemon mds.b 2024-04-25T19:08:29.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.700+0000 7f41406d1b80 20 client.26074 populate_metadata read hostname 'smithi045' 2024-04-25T19:08:29.704 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.700+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e2f5dfc020 0x55e2f5dfe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.700+0000 7f41406d1b80 4 client.26074 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-25T19:08:29.705 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.700+0000 7f41406d1b80 1 -- 172.21.15.45:0/2263158083 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55e2f5d74a10 con 0x55e2f5dfc020 2024-04-25T19:08:29.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.700+0000 7f4127e00700 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e2f5dfc020 0x55e2f5dfe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.701+0000 7f4127e00700 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e2f5dfc020 0x55e2f5dfe410 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1c54281541cd24 server_cookie=5ccea0cc9004c80e in_seq=0 out_seq=0 2024-04-25T19:08:29.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.701+0000 7f411d600700 10 client.26074 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:29.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.701+0000 7f411d600700 1 -- 172.21.15.45:0/2263158083 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x55e2f5d74a10 con 0x55e2f5dfc020 2024-04-25T19:08:29.706 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.701+0000 7f411d600700 10 client.26074 handle_command_reply: tid=0 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.701+0000 7f41406d1b80 1 client.26074 shutdown 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.701+0000 7f41406d1b80 1 -- 172.21.15.45:0/2263158083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d79890 msgr2=0x55e2f5d8d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d79890 0x55e2f5d8d340 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4120009230 tx=0x7f4120005b40 comp rx=0 tx=0).stop 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f41406d1b80 1 -- 172.21.15.45:0/2263158083 shutdown_connections 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x55e2f5dfc020 0x55e2f5dfe410 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e2f5d85060 0x55e2f5d8bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e2f5d79890 0x55e2f5d8d340 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f411d600700 0 client.26074 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f41406d1b80 1 --2- 172.21.15.45:0/2263158083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x55e2f5d7a1b0 0x55e2f5d8b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f41406d1b80 1 -- 172.21.15.45:0/2263158083 >> 172.21.15.45:0/2263158083 conn(0x55e2f5ca8610 msgr2=0x55e2f5d83ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f41406d1b80 1 -- 172.21.15.45:0/2263158083 shutdown_connections 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f41406d1b80 1 -- 172.21.15.45:0/2263158083 wait complete. 2024-04-25T19:08:29.707 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.702+0000 7f41406d1b80 20 client.26074 trim_cache size 0 max 16384 2024-04-25T19:08:29.708 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:08:29.708 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 36, 2024-04-25T19:08:29.708 INFO:teuthology.orchestra.run.smithi045.stdout: "leader": 24524, 2024-04-25T19:08:29.708 INFO:teuthology.orchestra.run.smithi045.stdout: "set_version": 6, 2024-04-25T19:08:29.708 INFO:teuthology.orchestra.run.smithi045.stdout: "sets": {} 2024-04-25T19:08:29.708 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 -- 172.21.15.45:0/583202445 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f41140a3de0 msgr2=0x7f41140a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f41140a3de0 0x7f41140a62a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f41300786c0 tx=0x7f4118009260 comp rx=0 tx=0).stop 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 -- 172.21.15.45:0/583202445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4130104ec0 msgr2=0x7f4130076f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4130104ec0 0x7f4130076f50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f412000eb30 tx=0x7f412000ee40 comp rx=0 tx=0).stop 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 -- 172.21.15.45:0/583202445 shutdown_connections 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f41140a3de0 0x7f41140a62a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f4130104ec0 0x7f4130076f50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4130109fa0 0x7f41300779d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 --2- 172.21.15.45:0/583202445 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4130105870 0x7f4130077490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 -- 172.21.15.45:0/583202445 >> 172.21.15.45:0/583202445 conn(0x7f4130100750 msgr2=0x7f4130102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 -- 172.21.15.45:0/583202445 shutdown_connections 2024-04-25T19:08:29.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:08:29.703+0000 7f4135e00700 1 -- 172.21.15.45:0/583202445 wait complete. 2024-04-25T19:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.821+0000 7f7497e00640 1 -- 172.21.15.45:0/1453537211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490102b80 msgr2=0x7f7490106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.821+0000 7f7497e00640 1 --2- 172.21.15.45:0/1453537211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490102b80 0x7f7490106bc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f747c009900 tx=0x7f747c02a3f0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.822+0000 7f7497e00640 1 -- 172.21.15.45:0/1453537211 shutdown_connections 2024-04-25T19:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.822+0000 7f7497e00640 1 --2- 172.21.15.45:0/1453537211 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f74901072a0 0x7f749010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.822+0000 7f7497e00640 1 --2- 172.21.15.45:0/1453537211 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f7490102b80 0x7f7490106bc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.824 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.822+0000 7f7497e00640 1 --2- 172.21.15.45:0/1453537211 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74901021d0 0x7f74901025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.822+0000 7f7497e00640 1 -- 172.21.15.45:0/1453537211 >> 172.21.15.45:0/1453537211 conn(0x7f74900fda10 msgr2=0x7f74900ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.822+0000 7f7497e00640 1 -- 172.21.15.45:0/1453537211 shutdown_connections 2024-04-25T19:08:29.825 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.822+0000 7f7497e00640 1 -- 172.21.15.45:0/1453537211 wait complete. 2024-04-25T19:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.823+0000 7f7497e00640 1 Processor -- start 2024-04-25T19:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.823+0000 7f7497e00640 1 -- start start 2024-04-25T19:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.823+0000 7f7497e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74901021d0 0x7f74901a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.823+0000 7f7497e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74901072a0 0x7f74901a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.823+0000 7f7497e00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f749019a790 0x7f749019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.823+0000 7f7497e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7490112230 con 0x7f74901072a0 2024-04-25T19:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.823+0000 7f7497e00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74901120b0 con 0x7f74901021d0 2024-04-25T19:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.823+0000 7f7497e00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f74901123b0 con 0x7f749019a790 2024-04-25T19:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.824+0000 7f7495400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74901021d0 0x7f74901a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.826 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.824+0000 7f7494a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74901072a0 0x7f74901a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.824+0000 7f7495400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74901021d0 0x7f74901a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41178/0 (socket says 172.21.15.45:41178) 2024-04-25T19:08:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.824+0000 7f7495400640 1 -- 172.21.15.45:0/132393408 learned_addr learned my addr 172.21.15.45:0/132393408 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:29.827 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.824+0000 7f7495e00640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f749019a790 0x7f749019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.824+0000 7f7495400640 1 -- 172.21.15.45:0/132393408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f749019a790 msgr2=0x7f749019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.824+0000 7f7495400640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f749019a790 0x7f749019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.824+0000 7f7495400640 1 -- 172.21.15.45:0/132393408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74901072a0 msgr2=0x7f74901a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.824+0000 7f7495400640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74901072a0 0x7f74901a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.824+0000 7f7495400640 1 -- 172.21.15.45:0/132393408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f749019b320 con 0x7f74901021d0 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.825+0000 7f7495400640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74901021d0 0x7f74901a0620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f748400ebc0 tx=0x7f748400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.825+0000 7f748e000640 1 -- 172.21.15.45:0/132393408 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7484010630 con 0x7f74901021d0 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.825+0000 7f748e000640 1 -- 172.21.15.45:0/132393408 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7484010c80 con 0x7f74901021d0 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.825+0000 7f748e000640 1 -- 172.21.15.45:0/132393408 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7484018b60 con 0x7f74901021d0 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.825+0000 7f7497e00640 1 -- 172.21.15.45:0/132393408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f749019b610 con 0x7f74901021d0 2024-04-25T19:08:29.828 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.825+0000 7f7497e00640 1 -- 172.21.15.45:0/132393408 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901a7770 con 0x7f74901021d0 2024-04-25T19:08:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.828+0000 7f749abf3740 1 Processor -- start 2024-04-25T19:08:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.828+0000 7f749abf3740 1 -- start start 2024-04-25T19:08:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f748e000640 1 -- 172.21.15.45:0/132393408 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7484018d00 con 0x7f74901021d0 2024-04-25T19:08:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f749abf3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abd820 0x559e92abdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f749abf3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abe140 0x559e92ac84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f749abf3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92acadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f749abf3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559e92a23900 con 0x559e92abd820 2024-04-25T19:08:29.831 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f749abf3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559e929c9ef0 con 0x559e92ac8a00 2024-04-25T19:08:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f749abf3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559e92a1f8c0 con 0x559e92abe140 2024-04-25T19:08:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f7495e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abd820 0x559e92abdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f7495400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abe140 0x559e92ac84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f7494a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92acadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f7495e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abd820 0x559e92abdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36434/0 (socket says 172.21.15.45:36434) 2024-04-25T19:08:29.832 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f7495e00640 1 -- 172.21.15.45:0/1175544662 learned_addr learned my addr 172.21.15.45:0/1175544662 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f7494a00640 1 -- 172.21.15.45:0/1175544662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abe140 msgr2=0x559e92ac84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f7494a00640 1 --2- 172.21.15.45:0/1175544662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abe140 0x559e92ac84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f7494a00640 1 -- 172.21.15.45:0/1175544662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abd820 msgr2=0x559e92abdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.829+0000 7f7494a00640 1 --2- 172.21.15.45:0/1175544662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abd820 0x559e92abdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.830+0000 7f7494a00640 1 -- 172.21.15.45:0/1175544662 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e92736210 con 0x559e92ac8a00 2024-04-25T19:08:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.830+0000 7f7495e00640 1 --2- 172.21.15.45:0/1175544662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abd820 0x559e92abdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.830+0000 7f7494a00640 1 --2- 172.21.15.45:0/1175544662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92acadf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f747c02a910 tx=0x7f747c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415af63df04defe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.830+0000 7f7482600640 1 -- 172.21.15.45:0/1175544662 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c035040 con 0x559e92ac8a00 2024-04-25T19:08:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.830+0000 7f7482600640 1 -- 172.21.15.45:0/1175544662 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f747c0054f0 con 0x559e92ac8a00 2024-04-25T19:08:29.834 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.830+0000 7f7482600640 1 -- 172.21.15.45:0/1175544662 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747c0035f0 con 0x559e92ac8a00 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.830+0000 7f748e000640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f74780a3d90 0x7f74780a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f748e000640 1 -- 172.21.15.45:0/132393408 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f74840d0de0 con 0x7f74901021d0 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f7495e00640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f74780a3d90 0x7f74780a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f749abf3740 1 -- 172.21.15.45:0/1175544662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 msgr2=0x559e92acadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f749abf3740 1 --2- 172.21.15.45:0/1175544662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92acadf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f747c02a910 tx=0x7f747c0040c0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f749abf3740 1 -- 172.21.15.45:0/1175544662 shutdown_connections 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f749abf3740 1 --2- 172.21.15.45:0/1175544662 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92acadf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f749abf3740 1 --2- 172.21.15.45:0/1175544662 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abe140 0x559e92ac84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f749abf3740 1 --2- 172.21.15.45:0/1175544662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abd820 0x559e92abdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f749abf3740 1 -- 172.21.15.45:0/1175544662 >> 172.21.15.45:0/1175544662 conn(0x559e92ab9d30 msgr2=0x559e92aba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f749abf3740 1 -- 172.21.15.45:0/1175544662 shutdown_connections 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.831+0000 7f749abf3740 1 -- 172.21.15.45:0/1175544662 wait complete. 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.832+0000 7f7495e00640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f74780a3d90 0x7f74780a6250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7488009e70 tx=0x7f7488005f90 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.835 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.832+0000 7f749abf3740 1 -- start start 2024-04-25T19:08:29.836 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.832+0000 7f749abf3740 1 Processor -- start 2024-04-25T19:08:29.839 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.832+0000 7f749abf3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 0x559e92ac59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.832+0000 7f749abf3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92abe140 0x559e92ac5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f749abf3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92ac8a00 0x559e92ac6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f749abf3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559e92acb5d0 con 0x559e92ac8a00 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f749abf3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x559e929dd310 con 0x559e92abe140 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f749abf3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x559e92acb750 con 0x559e92abd820 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f749abf3740 10 client.0 _ll_register_callbacks cb 0x559e929abcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7495e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92ac8a00 0x559e92ac6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7495e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92ac8a00 0x559e92ac6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36450/0 (socket says 172.21.15.45:36450) 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7495e00640 1 -- 172.21.15.45:0/3235585538 learned_addr learned my addr 172.21.15.45:0/3235585538 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7495400640 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 0x559e92ac59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7494a00640 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92abe140 0x559e92ac5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7495e00640 1 -- 172.21.15.45:0/3235585538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 msgr2=0x559e92ac59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7495e00640 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 0x559e92ac59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.840 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7495e00640 1 -- 172.21.15.45:0/3235585538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92abe140 msgr2=0x559e92ac5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7495e00640 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92abe140 0x559e92ac5f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7495e00640 1 -- 172.21.15.45:0/3235585538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e92802620 con 0x559e92ac8a00 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.833+0000 7f7495400640 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 0x559e92ac59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.834+0000 7f7495e00640 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92ac8a00 0x559e92ac6460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7488004630 tx=0x7f7488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f63ce88e5a4797 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.834+0000 7f7476a00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.834+0000 7f7476a00640 1 -- 172.21.15.45:0/3235585538 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488012e60 con 0x559e92ac8a00 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.834+0000 7f7476a00640 1 -- 172.21.15.45:0/3235585538 <== mon.0 v2:172.21.15.45:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f748801f630 con 0x559e92ac8a00 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.834+0000 7f749abf3740 1 -- 172.21.15.45:0/3235585538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f747c0054f0 con 0x559e92ac8a00 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.834+0000 7f7476a00640 1 -- 172.21.15.45:0/3235585538 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f748801f830 con 0x559e92ac8a00 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.834+0000 7f749abf3740 10 client.15924 fetch_fsmap learned FSMap version 36 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.834+0000 7f749abf3740 1 -- 172.21.15.45:0/3235585538 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559e92ad78e0 con 0x559e92ac8a00 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.835+0000 7f7476a00640 1 -- 172.21.15.45:0/3235585538 <== mon.0 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f747c0054f0 con 0x559e92ac8a00 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.835+0000 7f7476a00640 1 -- 172.21.15.45:0/3235585538 <== mon.0 v2:172.21.15.45:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f7488027070 con 0x559e92ac8a00 2024-04-25T19:08:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.835+0000 7f749abf3740 10 client.15924 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.835+0000 7f749abf3740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.835+0000 7f749abf3740 20 client.15924 populate_metadata read hostname 'smithi045' 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.835+0000 7f749abf3740 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559e92ad8df0 0x559e92adb200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.835+0000 7f749abf3740 4 client.15924 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.835+0000 7f749abf3740 1 -- 172.21.15.45:0/3235585538 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e92a378b0 con 0x559e92ad8df0 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.836+0000 7f7495400640 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559e92ad8df0 0x559e92adb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.836+0000 7f7495400640 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559e92ad8df0 0x559e92adb200 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be820cd39e2b01e3 server_cookie=594a88ccf38dd472 in_seq=0 out_seq=0 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.836+0000 7f7476a00640 10 client.15924 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f7476a00640 1 -- 172.21.15.45:0/3235585538 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559e92a378b0 con 0x559e92ad8df0 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f7476a00640 10 client.15924 handle_command_reply: tid=0 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 client.15924 shutdown 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 -- 172.21.15.45:0/3235585538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92ac8a00 msgr2=0x559e92ac6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92ac8a00 0x559e92ac6460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7488004630 tx=0x7f7488004730 comp rx=0 tx=0).stop 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 -- 172.21.15.45:0/3235585538 shutdown_connections 2024-04-25T19:08:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559e92ad8df0 0x559e92adb200 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92ac8a00 0x559e92ac6460 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92abe140 0x559e92ac5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 --2- 172.21.15.45:0/3235585538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 0x559e92ac59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f7476a00640 0 client.15924 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 -- 172.21.15.45:0/3235585538 >> 172.21.15.45:0/3235585538 conn(0x559e92ab9d30 msgr2=0x559e92abfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 -- 172.21.15.45:0/3235585538 shutdown_connections 2024-04-25T19:08:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.837+0000 7f749abf3740 1 -- 172.21.15.45:0/3235585538 wait complete. 2024-04-25T19:08:29.843 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.838+0000 7f749abf3740 20 client.15924 trim_cache size 0 max 16384 2024-04-25T19:08:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.856+0000 7f749abf3740 1 Processor -- start 2024-04-25T19:08:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f749abf3740 1 -- start start 2024-04-25T19:08:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f749abf3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 0x559e92ad4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f749abf3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92ac0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f749abf3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92ae0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f749abf3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74901b2860 con 0x559e92abe140 2024-04-25T19:08:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f749abf3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74901b00d0 con 0x559e92ac8a00 2024-04-25T19:08:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f749abf3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f74901adf40 con 0x559e92abd820 2024-04-25T19:08:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f7495e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92ae0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f7495400640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 0x559e92ad4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f7494a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92ac0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f7495e00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92ae0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:41206/0 (socket says 172.21.15.45:41206) 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.857+0000 7f7495e00640 1 -- 172.21.15.45:0/3475533464 learned_addr learned my addr 172.21.15.45:0/3475533464 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7495400640 1 -- 172.21.15.45:0/3475533464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 msgr2=0x559e92ae0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7495400640 1 --2- 172.21.15.45:0/3475533464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92ae0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7495400640 1 -- 172.21.15.45:0/3475533464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 msgr2=0x559e92ac0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7495400640 1 --2- 172.21.15.45:0/3475533464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92ac0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7495400640 1 -- 172.21.15.45:0/3475533464 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e92ac0770 con 0x559e92abd820 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7495e00640 1 --2- 172.21.15.45:0/3475533464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92ae0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7494a00640 1 --2- 172.21.15.45:0/3475533464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92ac0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7495400640 1 --2- 172.21.15.45:0/3475533464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 0x559e92ad4490 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f74840ce4a0 tx=0x7f74840d32a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb509f4046cf8c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7481c00640 1 -- 172.21.15.45:0/3475533464 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748409c700 con 0x559e92abd820 2024-04-25T19:08:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7481c00640 1 -- 172.21.15.45:0/3475533464 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7484018420 con 0x559e92abd820 2024-04-25T19:08:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.858+0000 7f7481c00640 1 -- 172.21.15.45:0/3475533464 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748409b0a0 con 0x559e92abd820 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.859+0000 7f749abf3740 1 -- 172.21.15.45:0/3475533464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 msgr2=0x559e92ad4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.859+0000 7f749abf3740 1 --2- 172.21.15.45:0/3475533464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 0x559e92ad4490 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f74840ce4a0 tx=0x7f74840d32a0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.859+0000 7f749abf3740 1 -- 172.21.15.45:0/3475533464 shutdown_connections 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.859+0000 7f749abf3740 1 --2- 172.21.15.45:0/3475533464 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92ac8a00 0x559e92ae0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.859+0000 7f749abf3740 1 --2- 172.21.15.45:0/3475533464 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92ac0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.859+0000 7f749abf3740 1 --2- 172.21.15.45:0/3475533464 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92abd820 0x559e92ad4490 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.859+0000 7f749abf3740 1 -- 172.21.15.45:0/3475533464 >> 172.21.15.45:0/3475533464 conn(0x559e92ab9d30 msgr2=0x559e92abe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.859+0000 7f749abf3740 1 -- 172.21.15.45:0/3475533464 shutdown_connections 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.859+0000 7f749abf3740 1 -- 172.21.15.45:0/3475533464 wait complete. 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.860+0000 7f749abf3740 1 -- start start 2024-04-25T19:08:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.860+0000 7f749abf3740 1 Processor -- start 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.860+0000 7f749abf3740 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92abd820 0x559e92b36680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.860+0000 7f749abf3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92b36bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.860+0000 7f749abf3740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92ac8a00 0x559e92acdd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.860+0000 7f749abf3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f74901ae590 con 0x559e92abe140 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.860+0000 7f749abf3740 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f74901afa00 con 0x559e92abd820 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.860+0000 7f749abf3740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f74901aec60 con 0x559e92ac8a00 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.860+0000 7f749abf3740 10 client.0 _ll_register_callbacks cb 0x7f74880044c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7494a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92b36bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7494a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92b36bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36468/0 (socket says 172.21.15.45:36468) 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7494a00640 1 -- 172.21.15.45:0/2763522424 learned_addr learned my addr 172.21.15.45:0/2763522424 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7495e00640 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92ac8a00 0x559e92acdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7495400640 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92abd820 0x559e92b36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7495e00640 1 -- 172.21.15.45:0/2763522424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92abd820 msgr2=0x559e92b36680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7495e00640 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92abd820 0x559e92b36680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7495e00640 1 -- 172.21.15.45:0/2763522424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 msgr2=0x559e92b36bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7495e00640 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92b36bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7495e00640 1 -- 172.21.15.45:0/2763522424 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e92ace500 con 0x559e92ac8a00 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7494a00640 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92b36bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7495e00640 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92ac8a00 0x559e92acdd20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7488004490 tx=0x7f748801fc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36510b482c341779 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.861+0000 7f7476a00640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.862+0000 7f7476a00640 1 -- 172.21.15.45:0/2763522424 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7488012e60 con 0x559e92ac8a00 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.862+0000 7f7476a00640 1 -- 172.21.15.45:0/2763522424 <== mon.1 v2:172.21.15.79:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8571+0+0 (secure 0 0 0) 0x7f7488029650 con 0x559e92ac8a00 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.862+0000 7f749abf3740 1 -- 172.21.15.45:0/2763522424 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e92a31320 con 0x559e92ac8a00 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.862+0000 7f7476a00640 1 -- 172.21.15.45:0/2763522424 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f748801fe50 con 0x559e92ac8a00 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.862+0000 7f749abf3740 10 client.26092 fetch_fsmap learned FSMap version 36 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.862+0000 7f749abf3740 1 -- 172.21.15.45:0/2763522424 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559e92b3ee90 con 0x559e92ac8a00 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.862+0000 7f7476a00640 1 -- 172.21.15.45:0/2763522424 <== mon.1 v2:172.21.15.79:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559e92a31320 con 0x559e92ac8a00 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.863+0000 7f7476a00640 1 -- 172.21.15.45:0/2763522424 <== mon.1 v2:172.21.15.79:3300/0 5 ==== fsmap(e 36) ==== 8354+0+0 (secure 0 0 0) 0x7f7488040070 con 0x559e92ac8a00 2024-04-25T19:08:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.863+0000 7f749abf3740 10 client.26092 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T19:08:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.863+0000 7f749abf3740 10 client.26092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T19:08:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.863+0000 7f749abf3740 20 client.26092 populate_metadata read hostname 'smithi045' 2024-04-25T19:08:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.863+0000 7f749abf3740 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559e92b3f550 0x559e92b41940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:08:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.863+0000 7f749abf3740 4 client.26092 mds_command: new command op to 24524 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T19:08:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.863+0000 7f749abf3740 1 -- 172.21.15.45:0/2763522424 --> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e92a2b190 con 0x559e92b3f550 2024-04-25T19:08:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.863+0000 7f7495400640 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559e92b3f550 0x559e92b41940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:08:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.864+0000 7f7495400640 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559e92b3f550 0x559e92b41940 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47c4da1d7a85753e server_cookie=338a13348dbe22d7 in_seq=0 out_seq=0 2024-04-25T19:08:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.864+0000 7f7476a00640 10 client.26092 ms_handle_connect on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.982+0000 7f7476a00640 1 -- 172.21.15.45:0/2763522424 <== mds.0 v2:172.21.15.62:6838/3288398937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559e92a2b190 con 0x559e92b3f550 2024-04-25T19:08:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.982+0000 7f7476a00640 10 client.26092 handle_command_reply: tid=0 2024-04-25T19:08:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.982+0000 7f749abf3740 1 client.26092 shutdown 2024-04-25T19:08:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.982+0000 7f749abf3740 1 -- 172.21.15.45:0/2763522424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92ac8a00 msgr2=0x559e92acdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.982+0000 7f749abf3740 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92ac8a00 0x559e92acdd20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7488004490 tx=0x7f748801fc60 comp rx=0 tx=0).stop 2024-04-25T19:08:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.983+0000 7f749abf3740 1 -- 172.21.15.45:0/2763522424 shutdown_connections 2024-04-25T19:08:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.983+0000 7f749abf3740 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.62:6838/3288398937,v1:172.21.15.62:6839/3288398937] conn(0x559e92b3f550 0x559e92b41940 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.983+0000 7f749abf3740 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x559e92ac8a00 0x559e92acdd20 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.983+0000 7f749abf3740 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559e92abe140 0x559e92b36bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.985 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.983+0000 7f749abf3740 1 --2- 172.21.15.45:0/2763522424 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x559e92abd820 0x559e92b36680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.983+0000 7f749abf3740 1 -- 172.21.15.45:0/2763522424 >> 172.21.15.45:0/2763522424 conn(0x559e92ab9d30 msgr2=0x559e92ab8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.983+0000 7f7476a00640 0 client.26092 ms_handle_reset on v2:172.21.15.62:6838/3288398937 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.983+0000 7f749abf3740 1 -- 172.21.15.45:0/2763522424 shutdown_connections 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.983+0000 7f749abf3740 1 -- 172.21.15.45:0/2763522424 wait complete. 2024-04-25T19:08:29.986 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.983+0000 7f749abf3740 20 client.26092 trim_cache size 0 max 16384 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:{ 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "message": "", 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout: "return_code": 0 2024-04-25T19:08:29.987 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stdout:} 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.984+0000 7f7497e00640 1 -- 172.21.15.45:0/132393408 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f74780a3d90 msgr2=0x7f74780a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.984+0000 7f7497e00640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f74780a3d90 0x7f74780a6250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7488009e70 tx=0x7f7488005f90 comp rx=0 tx=0).stop 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.985+0000 7f7497e00640 1 -- 172.21.15.45:0/132393408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74901021d0 msgr2=0x7f74901a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.985+0000 7f7497e00640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74901021d0 0x7f74901a0620 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f748400ebc0 tx=0x7f748400c6a0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.985+0000 7f7497e00640 1 -- 172.21.15.45:0/132393408 shutdown_connections 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.985+0000 7f7497e00640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f74780a3d90 0x7f74780a6250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.985+0000 7f7497e00640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f749019a790 0x7f749019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.985+0000 7f7497e00640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f74901072a0 0x7f74901a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.985+0000 7f7497e00640 1 --2- 172.21.15.45:0/132393408 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f74901021d0 0x7f74901a0620 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.985+0000 7f7497e00640 1 -- 172.21.15.45:0/132393408 >> 172.21.15.45:0/132393408 conn(0x7f74900fda10 msgr2=0x7f74900fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:08:29.988 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.985+0000 7f7497e00640 1 -- 172.21.15.45:0/132393408 shutdown_connections 2024-04-25T19:08:29.989 INFO:teuthology.task.background_exec.ubuntu@smithi045.front.sepia.ceph.com.smithi045.stderr:2024-04-25T19:08:29.985+0000 7f7497e00640 1 -- 172.21.15.45:0/132393408 wait complete. 2024-04-25T19:08:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:30 smithi062 ceph-mon[32844]: pgmap v285: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 384 op/s 2024-04-25T19:08:30.652 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:30 smithi045 ceph-mon[27776]: pgmap v285: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 384 op/s 2024-04-25T19:08:30.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:30 smithi079 ceph-mon[33002]: pgmap v285: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 384 op/s 2024-04-25T19:08:30.730 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks/quiescer.py", line 411, in task stop_all_quiescers(ctx.ceph[cluster_name].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph_7797745e6d8b3ac036504dc61491d368615024da/qa/tasks/quiescer.py", line 370, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error quiescing set '695e6b8e': 110 (ETIMEDOUT) 2024-04-25T19:08:30.731 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T19:08:30.741 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T19:08:30.767 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T19:08:30.776 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T19:08:30.776 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T19:08:30.776 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T19:08:30.777 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T19:08:30.909 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T19:08:32.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:32 smithi062 ceph-mon[32844]: pgmap v286: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-04-25T19:08:32.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:32 smithi079 ceph-mon[33002]: pgmap v286: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-04-25T19:08:32.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:32 smithi045 ceph-mon[27776]: pgmap v286: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 389 op/s 2024-04-25T19:08:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:34 smithi062 ceph-mon[32844]: pgmap v287: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 358 KiB/s wr, 115 op/s 2024-04-25T19:08:34.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:34 smithi079 ceph-mon[33002]: pgmap v287: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 358 KiB/s wr, 115 op/s 2024-04-25T19:08:34.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:34 smithi045 ceph-mon[27776]: pgmap v287: 97 pgs: 97 active+clean; 101 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 674 KiB/s rd, 358 KiB/s wr, 115 op/s 2024-04-25T19:08:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:36 smithi062 ceph-mon[32844]: pgmap v288: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 444 KiB/s wr, 154 op/s 2024-04-25T19:08:36.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:36 smithi079 ceph-mon[33002]: pgmap v288: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 444 KiB/s wr, 154 op/s 2024-04-25T19:08:36.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:36 smithi045 ceph-mon[27776]: pgmap v288: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 682 KiB/s rd, 444 KiB/s wr, 154 op/s 2024-04-25T19:08:38.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:38 smithi062 ceph-mon[32844]: pgmap v289: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 99 KiB/s wr, 43 op/s 2024-04-25T19:08:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:38 smithi079 ceph-mon[33002]: pgmap v289: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 99 KiB/s wr, 43 op/s 2024-04-25T19:08:38.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:38 smithi045 ceph-mon[27776]: pgmap v289: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 99 KiB/s wr, 43 op/s 2024-04-25T19:08:40.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:40 smithi062 ceph-mon[32844]: pgmap v290: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 99 KiB/s wr, 43 op/s 2024-04-25T19:08:40.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:40 smithi079 ceph-mon[33002]: pgmap v290: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 99 KiB/s wr, 43 op/s 2024-04-25T19:08:40.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:40 smithi045 ceph-mon[27776]: pgmap v290: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 99 KiB/s wr, 43 op/s 2024-04-25T19:08:42.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:42 smithi062 ceph-mon[32844]: pgmap v291: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 46 op/s 2024-04-25T19:08:42.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:42 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:08:42.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:42 smithi079 ceph-mon[33002]: pgmap v291: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 46 op/s 2024-04-25T19:08:42.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:42 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:08:42.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:42 smithi045 ceph-mon[27776]: pgmap v291: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 102 KiB/s wr, 46 op/s 2024-04-25T19:08:42.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:42 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:08:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:43 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:08:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:43 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:08:43.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:43 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:08:43.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:43 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:08:43.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:43 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:08:43.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:43 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:08:43.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:43 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:08:43.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:43 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:08:43.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:43 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:08:44.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:44 smithi079 ceph-mon[33002]: pgmap v292: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 88 KiB/s wr, 41 op/s 2024-04-25T19:08:44.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:44 smithi045 ceph-mon[27776]: pgmap v292: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 88 KiB/s wr, 41 op/s 2024-04-25T19:08:44.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:44 smithi062 ceph-mon[32844]: pgmap v292: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 88 KiB/s wr, 41 op/s 2024-04-25T19:08:46.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:46 smithi079 ceph-mon[33002]: pgmap v293: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 88 KiB/s wr, 44 op/s 2024-04-25T19:08:46.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:46 smithi045 ceph-mon[27776]: pgmap v293: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 88 KiB/s wr, 44 op/s 2024-04-25T19:08:46.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:46 smithi062 ceph-mon[32844]: pgmap v293: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 88 KiB/s wr, 44 op/s 2024-04-25T19:08:48.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:48 smithi079 ceph-mon[33002]: pgmap v294: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-25T19:08:48.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:48 smithi045 ceph-mon[27776]: pgmap v294: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-25T19:08:48.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:48 smithi062 ceph-mon[32844]: pgmap v294: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-25T19:08:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:50 smithi079 ceph-mon[33002]: pgmap v295: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-25T19:08:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:50 smithi045 ceph-mon[27776]: pgmap v295: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-25T19:08:50.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:50 smithi062 ceph-mon[32844]: pgmap v295: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.1 KiB/s wr, 5 op/s 2024-04-25T19:08:52.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:52 smithi079 ceph-mon[33002]: pgmap v296: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-25T19:08:52.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:52 smithi045 ceph-mon[27776]: pgmap v296: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-25T19:08:52.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:52 smithi062 ceph-mon[32844]: pgmap v296: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-25T19:08:54.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:54 smithi079 ceph-mon[33002]: pgmap v297: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:08:54.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:54 smithi045 ceph-mon[27776]: pgmap v297: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:08:54.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:54 smithi062 ceph-mon[32844]: pgmap v297: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:08:56.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:56 smithi079 ceph-mon[33002]: pgmap v298: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:08:56.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:56 smithi045 ceph-mon[27776]: pgmap v298: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:08:56.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:56 smithi062 ceph-mon[32844]: pgmap v298: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:08:58.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:08:58 smithi079 ceph-mon[33002]: pgmap v299: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:08:58.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:08:58 smithi045 ceph-mon[27776]: pgmap v299: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:08:58.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:08:58 smithi062 ceph-mon[32844]: pgmap v299: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:00.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:00 smithi079 ceph-mon[33002]: pgmap v300: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:00.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:00 smithi045 ceph-mon[27776]: pgmap v300: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:00.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:00 smithi062 ceph-mon[32844]: pgmap v300: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:01.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:01 smithi079 ceph-mon[33002]: pgmap v301: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:01.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:01 smithi045 ceph-mon[27776]: pgmap v301: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:01.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:01 smithi062 ceph-mon[32844]: pgmap v301: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:04.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:04 smithi062 ceph-mon[32844]: pgmap v302: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:04.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:04 smithi079 ceph-mon[33002]: pgmap v302: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:04.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:04 smithi045 ceph-mon[27776]: pgmap v302: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:06.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:06 smithi062 ceph-mon[32844]: pgmap v303: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T19:09:06.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:06 smithi079 ceph-mon[33002]: pgmap v303: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T19:09:06.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:06 smithi045 ceph-mon[27776]: pgmap v303: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T19:09:08.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:08 smithi062 ceph-mon[32844]: pgmap v304: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:09:08.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:08 smithi079 ceph-mon[33002]: pgmap v304: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:09:08.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:08 smithi045 ceph-mon[27776]: pgmap v304: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:09:10.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:10 smithi062 ceph-mon[32844]: pgmap v305: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:09:10.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:10 smithi079 ceph-mon[33002]: pgmap v305: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:09:10.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:10 smithi045 ceph-mon[27776]: pgmap v305: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:09:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:12 smithi062 ceph-mon[32844]: pgmap v306: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T19:09:12.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:12 smithi062 ceph-mon[32844]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T19:09:12.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:12 smithi079 ceph-mon[33002]: pgmap v306: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T19:09:12.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:12 smithi079 ceph-mon[33002]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T19:09:12.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:12 smithi045 ceph-mon[27776]: pgmap v306: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T19:09:12.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:12 smithi045 ceph-mon[27776]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T19:09:14.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:14 smithi079 ceph-mon[33002]: pgmap v308: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T19:09:14.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:14 smithi045 ceph-mon[27776]: pgmap v308: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T19:09:14.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:14 smithi062 ceph-mon[32844]: pgmap v308: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T19:09:16.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:16 smithi079 ceph-mon[33002]: pgmap v309: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T19:09:16.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:16 smithi045 ceph-mon[27776]: pgmap v309: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T19:09:16.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:16 smithi062 ceph-mon[32844]: pgmap v309: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T19:09:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:17 smithi062 ceph-mon[32844]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T19:09:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:17 smithi062 ceph-mon[32844]: pgmap v311: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T19:09:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:17 smithi079 ceph-mon[33002]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T19:09:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:17 smithi079 ceph-mon[33002]: pgmap v311: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T19:09:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:17 smithi045 ceph-mon[27776]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T19:09:17.964 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:17 smithi045 ceph-mon[27776]: pgmap v311: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T19:09:20.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:20 smithi062 ceph-mon[32844]: pgmap v312: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T19:09:20.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:20 smithi079 ceph-mon[33002]: pgmap v312: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T19:09:20.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:20 smithi045 ceph-mon[27776]: pgmap v312: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2024-04-25T19:09:22.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:22 smithi062 ceph-mon[32844]: pgmap v313: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 309 B/s wr, 6 op/s 2024-04-25T19:09:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:22 smithi079 ceph-mon[33002]: pgmap v313: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 309 B/s wr, 6 op/s 2024-04-25T19:09:22.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:22 smithi045 ceph-mon[27776]: pgmap v313: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 309 B/s wr, 6 op/s 2024-04-25T19:09:23.014 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T19:09:23.236 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.234+0000 7fea3c200640 1 -- 172.21.15.45:0/3202108711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea3410e660 msgr2=0x7fea3410ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.234+0000 7fea3c200640 1 --2- 172.21.15.45:0/3202108711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea3410e660 0x7fea3410ea40 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fea28009890 tx=0x7fea2802a360 comp rx=0 tx=0).stop 2024-04-25T19:09:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.234+0000 7fea3c200640 1 -- 172.21.15.45:0/3202108711 shutdown_connections 2024-04-25T19:09:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.234+0000 7fea3c200640 1 --2- 172.21.15.45:0/3202108711 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea34108f60 0x7fea34115420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.234+0000 7fea3c200640 1 --2- 172.21.15.45:0/3202108711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea341085a0 0x7fea34108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.234+0000 7fea3c200640 1 --2- 172.21.15.45:0/3202108711 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea3410e660 0x7fea3410ea40 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.234+0000 7fea3c200640 1 -- 172.21.15.45:0/3202108711 >> 172.21.15.45:0/3202108711 conn(0x7fea34076460 msgr2=0x7fea34078880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.234+0000 7fea3c200640 1 -- 172.21.15.45:0/3202108711 shutdown_connections 2024-04-25T19:09:23.237 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.234+0000 7fea3c200640 1 -- 172.21.15.45:0/3202108711 wait complete. 2024-04-25T19:09:23.239 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.235+0000 7fea3c200640 1 Processor -- start 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.235+0000 7fea3c200640 1 -- start start 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea3c200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea341085a0 0x7fea341a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea3c200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34108f60 0x7fea341a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea3c200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3410e660 0x7fea3419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea3c200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fea34118850 con 0x7fea34108f60 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea3c200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fea341186d0 con 0x7fea341085a0 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea3c200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fea341189d0 con 0x7fea3410e660 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea38e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34108f60 0x7fea341a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea38e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34108f60 0x7fea341a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36280/0 (socket says 172.21.15.45:36280) 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea38e00640 1 -- 172.21.15.45:0/2172956191 learned_addr learned my addr 172.21.15.45:0/2172956191 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea39800640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea341085a0 0x7fea341a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea3a200640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3410e660 0x7fea3419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea38e00640 1 -- 172.21.15.45:0/2172956191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3410e660 msgr2=0x7fea3419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:23.240 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea38e00640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3410e660 0x7fea3419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea38e00640 1 -- 172.21.15.45:0/2172956191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea341085a0 msgr2=0x7fea341a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea38e00640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea341085a0 0x7fea341a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea38e00640 1 -- 172.21.15.45:0/2172956191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3419f3f0 con 0x7fea34108f60 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea3a200640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3410e660 0x7fea3419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.236+0000 7fea39800640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea341085a0 0x7fea341a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.237+0000 7fea38e00640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34108f60 0x7fea341a4fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea2400c420 tx=0x7fea24009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.237+0000 7fea32000640 1 -- 172.21.15.45:0/2172956191 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24003970 con 0x7fea34108f60 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.237+0000 7fea32000640 1 -- 172.21.15.45:0/2172956191 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea24004020 con 0x7fea34108f60 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.237+0000 7fea3c200640 1 -- 172.21.15.45:0/2172956191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3419f6e0 con 0x7fea34108f60 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.237+0000 7fea32000640 1 -- 172.21.15.45:0/2172956191 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2400b980 con 0x7fea34108f60 2024-04-25T19:09:23.241 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.237+0000 7fea3c200640 1 -- 172.21.15.45:0/2172956191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea341ab9f0 con 0x7fea34108f60 2024-04-25T19:09:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.238+0000 7fea3c200640 1 -- 172.21.15.45:0/2172956191 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea34109cc0 con 0x7fea34108f60 2024-04-25T19:09:23.242 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.241+0000 7fea32000640 1 -- 172.21.15.45:0/2172956191 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fea2401d020 con 0x7fea34108f60 2024-04-25T19:09:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.242+0000 7fea32000640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fea1c0a3d10 0x7fea1c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.242+0000 7fea32000640 1 -- 172.21.15.45:0/2172956191 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7fea240d0780 con 0x7fea34108f60 2024-04-25T19:09:23.244 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.242+0000 7fea39800640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fea1c0a3d10 0x7fea1c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:23.245 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.243+0000 7fea39800640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fea1c0a3d10 0x7fea1c0a61d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea28000c00 tx=0x7fea28005890 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:23.249 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.247+0000 7fea32000640 1 -- 172.21.15.45:0/2172956191 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea2409cd90 con 0x7fea34108f60 2024-04-25T19:09:23.493 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.491+0000 7fea3c200640 1 -- 172.21.15.45:0/2172956191 --> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fea34062390 con 0x7fea1c0a3d10 2024-04-25T19:09:23.507 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.505+0000 7fea32000640 1 -- 172.21.15.45:0/2172956191 <== mgr.25049 v2:172.21.15.45:6800/1345298339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fea34062390 con 0x7fea1c0a3d10 2024-04-25T19:09:23.508 INFO:teuthology.orchestra.run.smithi045.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T18:59:20", "first": "2024-04-25T19:00:00", "last": "2024-04-25T19:09:00", "last_pruned": "2024-04-25T19:09:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-04-25T19:09:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.508+0000 7fea3c200640 1 -- 172.21.15.45:0/2172956191 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fea1c0a3d10 msgr2=0x7fea1c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.508+0000 7fea3c200640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fea1c0a3d10 0x7fea1c0a61d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fea28000c00 tx=0x7fea28005890 comp rx=0 tx=0).stop 2024-04-25T19:09:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.508+0000 7fea3c200640 1 -- 172.21.15.45:0/2172956191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34108f60 msgr2=0x7fea341a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.508+0000 7fea3c200640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34108f60 0x7fea341a4fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fea2400c420 tx=0x7fea24009f90 comp rx=0 tx=0).stop 2024-04-25T19:09:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.509+0000 7fea3c200640 1 -- 172.21.15.45:0/2172956191 shutdown_connections 2024-04-25T19:09:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.509+0000 7fea3c200640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fea1c0a3d10 0x7fea1c0a61d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.509+0000 7fea3c200640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fea3410e660 0x7fea3419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.509+0000 7fea3c200640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fea34108f60 0x7fea341a4fb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.509+0000 7fea3c200640 1 --2- 172.21.15.45:0/2172956191 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fea341085a0 0x7fea341a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.509+0000 7fea3c200640 1 -- 172.21.15.45:0/2172956191 >> 172.21.15.45:0/2172956191 conn(0x7fea34076460 msgr2=0x7fea34077cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.509+0000 7fea3c200640 1 -- 172.21.15.45:0/2172956191 shutdown_connections 2024-04-25T19:09:23.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.509+0000 7fea3c200640 1 -- 172.21.15.45:0/2172956191 wait complete. 2024-04-25T19:09:23.531 DEBUG:teuthology.orchestra.run.smithi045:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T19:09:23.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.737+0000 7f65e2a00640 1 -- 172.21.15.45:0/2549778985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc104d70 msgr2=0x7f65dc108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:23.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.737+0000 7f65e2a00640 1 --2- 172.21.15.45:0/2549778985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc104d70 0x7f65dc108db0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009ca0 tx=0x7f65c402a8c0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.737+0000 7f65e2a00640 1 -- 172.21.15.45:0/2549778985 shutdown_connections 2024-04-25T19:09:23.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.737+0000 7f65e2a00640 1 --2- 172.21.15.45:0/2549778985 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc109490 0x7f65dc111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.737+0000 7f65e2a00640 1 --2- 172.21.15.45:0/2549778985 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc104d70 0x7f65dc108db0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.737+0000 7f65e2a00640 1 --2- 172.21.15.45:0/2549778985 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65dc1043c0 0x7f65dc1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.737+0000 7f65e2a00640 1 -- 172.21.15.45:0/2549778985 >> 172.21.15.45:0/2549778985 conn(0x7f65dc0ffc40 msgr2=0x7f65dc102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:23.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.737+0000 7f65e2a00640 1 -- 172.21.15.45:0/2549778985 shutdown_connections 2024-04-25T19:09:23.740 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.738+0000 7f65e2a00640 1 -- 172.21.15.45:0/2549778985 wait complete. 2024-04-25T19:09:23.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.738+0000 7f65e2a00640 1 Processor -- start 2024-04-25T19:09:23.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65e2a00640 1 -- start start 2024-04-25T19:09:23.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65e2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc1043c0 0x7f65dc1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:23.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65e2a00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc109490 0x7f65dc1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:23.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65e2a00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65dc19a7c0 0x7f65dc19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:23.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65e2a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc1143b0 con 0x7f65dc1043c0 2024-04-25T19:09:23.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65e2a00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc114230 con 0x7f65dc109490 2024-04-25T19:09:23.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65e2a00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc114530 con 0x7f65dc19a7c0 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65db400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc109490 0x7f65dc1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65db400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc109490 0x7f65dc1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:55002/0 (socket says 172.21.15.45:55002) 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65dbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc1043c0 0x7f65dc1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65db400640 1 -- 172.21.15.45:0/2095598627 learned_addr learned my addr 172.21.15.45:0/2095598627 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.739+0000 7f65e0a00640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65dc19a7c0 0x7f65dc19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.740+0000 7f65db400640 1 -- 172.21.15.45:0/2095598627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65dc19a7c0 msgr2=0x7f65dc19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.740+0000 7f65db400640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65dc19a7c0 0x7f65dc19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.740+0000 7f65db400640 1 -- 172.21.15.45:0/2095598627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc1043c0 msgr2=0x7f65dc1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.740+0000 7f65db400640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc1043c0 0x7f65dc1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.740+0000 7f65db400640 1 -- 172.21.15.45:0/2095598627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc19b350 con 0x7f65dc109490 2024-04-25T19:09:23.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.740+0000 7f65db400640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc109490 0x7f65dc1a0b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f65c4033040 tx=0x7f65c40316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:23.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.740+0000 7f65d8c00640 1 -- 172.21.15.45:0/2095598627 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c4003b70 con 0x7f65dc109490 2024-04-25T19:09:23.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.741+0000 7f65e2a00640 1 -- 172.21.15.45:0/2095598627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc19b5e0 con 0x7f65dc109490 2024-04-25T19:09:23.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.741+0000 7f65d8c00640 1 -- 172.21.15.45:0/2095598627 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65c40318e0 con 0x7f65dc109490 2024-04-25T19:09:23.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.741+0000 7f65d8c00640 1 -- 172.21.15.45:0/2095598627 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65c402f950 con 0x7f65dc109490 2024-04-25T19:09:23.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.741+0000 7f65e2a00640 1 -- 172.21.15.45:0/2095598627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc1a7810 con 0x7f65dc109490 2024-04-25T19:09:23.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.742+0000 7f65e2a00640 1 -- 172.21.15.45:0/2095598627 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc105890 con 0x7f65dc109490 2024-04-25T19:09:23.746 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.744+0000 7f65d8c00640 1 -- 172.21.15.45:0/2095598627 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f65c402faf0 con 0x7f65dc109490 2024-04-25T19:09:23.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.745+0000 7f65d8c00640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65c00a3d10 0x7f65c00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:23.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.746+0000 7f65d8c00640 1 -- 172.21.15.45:0/2095598627 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f65c40f1c00 con 0x7f65dc109490 2024-04-25T19:09:23.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.746+0000 7f65dbe00640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65c00a3d10 0x7f65c00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:23.748 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.746+0000 7f65dbe00640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65c00a3d10 0x7f65c00a61d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009bd0 tx=0x7f65d4005f90 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:23.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.748+0000 7f65d8c00640 1 -- 172.21.15.45:0/2095598627 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f65c40be3b0 con 0x7f65dc109490 2024-04-25T19:09:23.985 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.983+0000 7f65e2a00640 1 -- 172.21.15.45:0/2095598627 --> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f65dc104d70 con 0x7f65c00a3d10 2024-04-25T19:09:24.001 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:23.999+0000 7f65d8c00640 1 -- 172.21.15.45:0/2095598627 <== mgr.25049 v2:172.21.15.45:6800/1345298339 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f65dc104d70 con 0x7f65c00a3d10 2024-04-25T19:09:24.002 INFO:teuthology.orchestra.run.smithi045.stdout:/ 1m 6m3h 2024-04-25T19:09:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.002+0000 7f65e2a00640 1 -- 172.21.15.45:0/2095598627 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65c00a3d10 msgr2=0x7f65c00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.002+0000 7f65e2a00640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65c00a3d10 0x7f65c00a61d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f65d4009bd0 tx=0x7f65d4005f90 comp rx=0 tx=0).stop 2024-04-25T19:09:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.002+0000 7f65e2a00640 1 -- 172.21.15.45:0/2095598627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc109490 msgr2=0x7f65dc1a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.004 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.002+0000 7f65e2a00640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc109490 0x7f65dc1a0b90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f65c4033040 tx=0x7f65c40316f0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.002+0000 7f65e2a00640 1 -- 172.21.15.45:0/2095598627 shutdown_connections 2024-04-25T19:09:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.002+0000 7f65e2a00640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f65c00a3d10 0x7f65c00a61d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.002+0000 7f65e2a00640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f65dc19a7c0 0x7f65dc19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.002+0000 7f65e2a00640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f65dc109490 0x7f65dc1a0b90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.002+0000 7f65e2a00640 1 --2- 172.21.15.45:0/2095598627 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65dc1043c0 0x7f65dc1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.002+0000 7f65e2a00640 1 -- 172.21.15.45:0/2095598627 >> 172.21.15.45:0/2095598627 conn(0x7f65dc0ffc40 msgr2=0x7f65dc1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.003+0000 7f65e2a00640 1 -- 172.21.15.45:0/2095598627 shutdown_connections 2024-04-25T19:09:24.005 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.003+0000 7f65e2a00640 1 -- 172.21.15.45:0/2095598627 wait complete. 2024-04-25T19:09:24.026 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T19:09:24.037 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T19:09:24.251 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.249+0000 7f7262200640 1 -- 172.21.15.45:0/636047369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f725c109360 msgr2=0x7f725c110f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.249+0000 7f7262200640 1 --2- 172.21.15.45:0/636047369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f725c109360 0x7f725c110f30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7254009860 tx=0x7f725402a340 comp rx=0 tx=0).stop 2024-04-25T19:09:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.249+0000 7f7262200640 1 -- 172.21.15.45:0/636047369 shutdown_connections 2024-04-25T19:09:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.249+0000 7f7262200640 1 --2- 172.21.15.45:0/636047369 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f725c109360 0x7f725c110f30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.249+0000 7f7262200640 1 --2- 172.21.15.45:0/636047369 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c104c40 0x7f725c108c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.249+0000 7f7262200640 1 --2- 172.21.15.45:0/636047369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725c104290 0x7f725c104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.249+0000 7f7262200640 1 -- 172.21.15.45:0/636047369 >> 172.21.15.45:0/636047369 conn(0x7f725c0ffad0 msgr2=0x7f725c101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.249+0000 7f7262200640 1 -- 172.21.15.45:0/636047369 shutdown_connections 2024-04-25T19:09:24.252 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.249+0000 7f7262200640 1 -- 172.21.15.45:0/636047369 wait complete. 2024-04-25T19:09:24.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.250+0000 7f7262200640 1 Processor -- start 2024-04-25T19:09:24.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.250+0000 7f7262200640 1 -- start start 2024-04-25T19:09:24.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f7262200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c104290 0x7f725c1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:24.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f7262200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f725c109360 0x7f725c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:24.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f7262200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725c19a610 0x7f725c19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:24.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f7262200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f725c1142b0 con 0x7f725c19a610 2024-04-25T19:09:24.255 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f7262200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f725c114130 con 0x7f725c104290 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f7262200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f725c114430 con 0x7f725c109360 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f725be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f725c109360 0x7f725c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f7261800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725c19a610 0x7f725c19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f725be00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f725c109360 0x7f725c1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:35322/0 (socket says 172.21.15.45:35322) 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f725be00640 1 -- 172.21.15.45:0/101161280 learned_addr learned my addr 172.21.15.45:0/101161280 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f7260e00640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c104290 0x7f725c1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f725be00640 1 -- 172.21.15.45:0/101161280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c104290 msgr2=0x7f725c1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f725be00640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c104290 0x7f725c1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f725be00640 1 -- 172.21.15.45:0/101161280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725c19a610 msgr2=0x7f725c19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f725be00640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725c19a610 0x7f725c19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f725be00640 1 -- 172.21.15.45:0/101161280 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725c19b130 con 0x7f725c109360 2024-04-25T19:09:24.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.251+0000 7f7261800640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725c19a610 0x7f725c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:24.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.252+0000 7f725be00640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f725c109360 0x7f725c1a09a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f724400eb90 tx=0x7f724400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:24.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.252+0000 7f7259600640 1 -- 172.21.15.45:0/101161280 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244010630 con 0x7f725c109360 2024-04-25T19:09:24.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.252+0000 7f7259600640 1 -- 172.21.15.45:0/101161280 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f724400ee90 con 0x7f725c109360 2024-04-25T19:09:24.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.252+0000 7f7262200640 1 -- 172.21.15.45:0/101161280 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725c19b420 con 0x7f725c109360 2024-04-25T19:09:24.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.252+0000 7f7259600640 1 -- 172.21.15.45:0/101161280 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7244018ac0 con 0x7f725c109360 2024-04-25T19:09:24.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.252+0000 7f7262200640 1 -- 172.21.15.45:0/101161280 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725c1a76c0 con 0x7f725c109360 2024-04-25T19:09:24.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.253+0000 7f7262200640 1 -- 172.21.15.45:0/101161280 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f725c04fd50 con 0x7f725c109360 2024-04-25T19:09:24.258 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.256+0000 7f7259600640 1 -- 172.21.15.45:0/101161280 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f7244018c60 con 0x7f725c109360 2024-04-25T19:09:24.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.257+0000 7f7259600640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72400a3d90 0x7f72400a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:24.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.257+0000 7f7259600640 1 -- 172.21.15.45:0/101161280 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f72440d1190 con 0x7f725c109360 2024-04-25T19:09:24.259 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.257+0000 7f7260e00640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72400a3d90 0x7f72400a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:24.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.258+0000 7f7260e00640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72400a3d90 0x7f72400a6250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f724c009530 tx=0x7f724c009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:24.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.259+0000 7f7259600640 1 -- 172.21.15.45:0/101161280 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f724409d150 con 0x7f725c109360 2024-04-25T19:09:24.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:24 smithi062 ceph-mon[32844]: pgmap v314: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T19:09:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:24 smithi079 ceph-mon[33002]: pgmap v314: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T19:09:24.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:24 smithi045 ceph-mon[27776]: pgmap v314: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-04-25T19:09:24.471 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.469+0000 7f7262200640 1 -- 172.21.15.45:0/101161280 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f725c10c5f0 con 0x7f725c109360 2024-04-25T19:09:24.473 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.472+0000 7f7259600640 1 -- 172.21.15.45:0/101161280 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+23123 (secure 0 0 0) 0x7f72440a1450 con 0x7f725c109360 2024-04-25T19:09:24.474 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:09:24.475 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":104,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T19:09:16.467912+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:55:17.591864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:55:22.070218+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/943541413":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/2599503580":"2024-04-26T18:59:18.746957+0000","172.21.15.45:6800/943541413":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/3590060181":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/4010951391":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3367087840":"2024-04-26T18:59:18.746957+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:09:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.474+0000 7f7262200640 1 -- 172.21.15.45:0/101161280 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72400a3d90 msgr2=0x7f72400a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72400a3d90 0x7f72400a6250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f724c009530 tx=0x7f724c009260 comp rx=0 tx=0).stop 2024-04-25T19:09:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 -- 172.21.15.45:0/101161280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f725c109360 msgr2=0x7f725c1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f725c109360 0x7f725c1a09a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f724400eb90 tx=0x7f724400c6a0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.478 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 -- 172.21.15.45:0/101161280 shutdown_connections 2024-04-25T19:09:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72400a3d90 0x7f72400a6250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f725c19a610 0x7f725c19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f725c109360 0x7f725c1a09a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 --2- 172.21.15.45:0/101161280 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f725c104290 0x7f725c1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 -- 172.21.15.45:0/101161280 >> 172.21.15.45:0/101161280 conn(0x7f725c0ffad0 msgr2=0x7f725c101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:24.479 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 -- 172.21.15.45:0/101161280 shutdown_connections 2024-04-25T19:09:24.480 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.475+0000 7f7262200640 1 -- 172.21.15.45:0/101161280 wait complete. 2024-04-25T19:09:24.498 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T18:50:00.259702+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T18:55:17.591864+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T18:55:22.070218+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 103, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T19:09:24.498 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T19:09:24.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.715+0000 7f72efe00640 1 -- 172.21.15.45:0/2064098559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72f0103820 msgr2=0x7f72f010bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.715+0000 7f72efe00640 1 --2- 172.21.15.45:0/2064098559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72f0103820 0x7f72f010bd00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009900 tx=0x7f72d802a410 comp rx=0 tx=0).stop 2024-04-25T19:09:24.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.716+0000 7f72efe00640 1 -- 172.21.15.45:0/2064098559 shutdown_connections 2024-04-25T19:09:24.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.716+0000 7f72efe00640 1 --2- 172.21.15.45:0/2064098559 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72f010c370 0x7f72f010e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.716+0000 7f72efe00640 1 --2- 172.21.15.45:0/2064098559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72f0103820 0x7f72f010bd00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.716+0000 7f72efe00640 1 --2- 172.21.15.45:0/2064098559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f0102f00 0x7f72f01032e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.716+0000 7f72efe00640 1 -- 172.21.15.45:0/2064098559 >> 172.21.15.45:0/2064098559 conn(0x7f72f00fc760 msgr2=0x7f72f00feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:24.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.716+0000 7f72efe00640 1 -- 172.21.15.45:0/2064098559 shutdown_connections 2024-04-25T19:09:24.719 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.716+0000 7f72efe00640 1 -- 172.21.15.45:0/2064098559 wait complete. 2024-04-25T19:09:24.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.717+0000 7f72efe00640 1 Processor -- start 2024-04-25T19:09:24.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.717+0000 7f72efe00640 1 -- start start 2024-04-25T19:09:24.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.717+0000 7f72efe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72f0102f00 0x7f72f01a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:24.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.717+0000 7f72efe00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f0103820 0x7f72f01a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:24.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.717+0000 7f72efe00640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72f010c370 0x7f72f019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:24.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.717+0000 7f72efe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0111fc0 con 0x7f72f0102f00 2024-04-25T19:09:24.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.717+0000 7f72efe00640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0111e40 con 0x7f72f010c370 2024-04-25T19:09:24.720 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.717+0000 7f72efe00640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f72f0112140 con 0x7f72f0103820 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.717+0000 7f72eea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72f0102f00 0x7f72f01a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.717+0000 7f72eea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72f0102f00 0x7f72f01a0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45190/0 (socket says 172.21.15.45:45190) 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.718+0000 7f72eea00640 1 -- 172.21.15.45:0/2538039779 learned_addr learned my addr 172.21.15.45:0/2538039779 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.718+0000 7f72ee000640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f0103820 0x7f72f01a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.718+0000 7f72ef400640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72f010c370 0x7f72f019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.718+0000 7f72eea00640 1 -- 172.21.15.45:0/2538039779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f0103820 msgr2=0x7f72f01a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.718+0000 7f72eea00640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f0103820 0x7f72f01a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.718+0000 7f72eea00640 1 -- 172.21.15.45:0/2538039779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72f010c370 msgr2=0x7f72f019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.718+0000 7f72eea00640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72f010c370 0x7f72f019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.718+0000 7f72eea00640 1 -- 172.21.15.45:0/2538039779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72f019ad60 con 0x7f72f0102f00 2024-04-25T19:09:24.721 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.718+0000 7f72eea00640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72f0102f00 0x7f72f01a0430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f72e000e9f0 tx=0x7f72e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:24.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.718+0000 7f72e7400640 1 -- 172.21.15.45:0/2538039779 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e000ccc0 con 0x7f72f0102f00 2024-04-25T19:09:24.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.719+0000 7f72e7400640 1 -- 172.21.15.45:0/2538039779 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e000ce60 con 0x7f72f0102f00 2024-04-25T19:09:24.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.719+0000 7f72efe00640 1 -- 172.21.15.45:0/2538039779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72f019b050 con 0x7f72f0102f00 2024-04-25T19:09:24.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.719+0000 7f72e7400640 1 -- 172.21.15.45:0/2538039779 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0010430 con 0x7f72f0102f00 2024-04-25T19:09:24.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.719+0000 7f72efe00640 1 -- 172.21.15.45:0/2538039779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72f01a72f0 con 0x7f72f0102f00 2024-04-25T19:09:24.722 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.720+0000 7f72efe00640 1 -- 172.21.15.45:0/2538039779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72f004fce0 con 0x7f72f0102f00 2024-04-25T19:09:24.724 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.722+0000 7f72e7400640 1 -- 172.21.15.45:0/2538039779 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f72e00046c0 con 0x7f72f0102f00 2024-04-25T19:09:24.725 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.723+0000 7f72e7400640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72d40a3d20 0x7f72d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:24.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.724+0000 7f72e7400640 1 -- 172.21.15.45:0/2538039779 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f72e00d0e50 con 0x7f72f0102f00 2024-04-25T19:09:24.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.724+0000 7f72ee000640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72d40a3d20 0x7f72d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:24.726 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.724+0000 7f72ee000640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72d40a3d20 0x7f72d40a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000c00 tx=0x7f72d8031400 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:24.729 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.726+0000 7f72e7400640 1 -- 172.21.15.45:0/2538039779 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f72e0053ff0 con 0x7f72f0102f00 2024-04-25T19:09:24.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.932+0000 7f72efe00640 1 -- 172.21.15.45:0/2538039779 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f72f01a7550 con 0x7f72f0102f00 2024-04-25T19:09:24.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.933+0000 7f72e7400640 1 -- 172.21.15.45:0/2538039779 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) ==== 93+0+10 (secure 0 0 0) 0x7f72e009d460 con 0x7f72f0102f00 2024-04-25T19:09:24.938 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 1 2024-04-25T19:09:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 -- 172.21.15.45:0/2538039779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72d40a3d20 msgr2=0x7f72d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72d40a3d20 0x7f72d40a61e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000c00 tx=0x7f72d8031400 comp rx=0 tx=0).stop 2024-04-25T19:09:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 -- 172.21.15.45:0/2538039779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72f0102f00 msgr2=0x7f72f01a0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72f0102f00 0x7f72f01a0430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f72e000e9f0 tx=0x7f72e000edf0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 -- 172.21.15.45:0/2538039779 shutdown_connections 2024-04-25T19:09:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f72d40a3d20 0x7f72d40a61e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.939 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f72f010c370 0x7f72f019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72f0103820 0x7f72f01a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 --2- 172.21.15.45:0/2538039779 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f72f0102f00 0x7f72f01a0430 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 -- 172.21.15.45:0/2538039779 >> 172.21.15.45:0/2538039779 conn(0x7f72f00fc760 msgr2=0x7f72f0101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.936+0000 7f72efe00640 1 -- 172.21.15.45:0/2538039779 shutdown_connections 2024-04-25T19:09:24.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:24.937+0000 7f72efe00640 1 -- 172.21.15.45:0/2538039779 wait complete. 2024-04-25T19:09:24.959 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T19:09:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.171+0000 7fb37f200640 1 -- 172.21.15.45:0/267627828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb38010e660 msgr2=0x7fb380110a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.171+0000 7fb37f200640 1 --2- 172.21.15.45:0/267627828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb38010e660 0x7fb380110a50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb378009860 tx=0x7fb37802a340 comp rx=0 tx=0).stop 2024-04-25T19:09:25.174 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.171+0000 7fb37f200640 1 -- 172.21.15.45:0/267627828 shutdown_connections 2024-04-25T19:09:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.172+0000 7fb37f200640 1 --2- 172.21.15.45:0/267627828 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb38010e660 0x7fb380110a50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.172+0000 7fb37f200640 1 --2- 172.21.15.45:0/267627828 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb380101460 0x7fb38010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.172+0000 7fb37f200640 1 --2- 172.21.15.45:0/267627828 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb380100ab0 0x7fb380100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.172+0000 7fb37f200640 1 -- 172.21.15.45:0/267627828 >> 172.21.15.45:0/267627828 conn(0x7fb380077a00 msgr2=0x7fb3800fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.172+0000 7fb37f200640 1 -- 172.21.15.45:0/267627828 shutdown_connections 2024-04-25T19:09:25.175 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.172+0000 7fb37f200640 1 -- 172.21.15.45:0/267627828 wait complete. 2024-04-25T19:09:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.173+0000 7fb37f200640 1 Processor -- start 2024-04-25T19:09:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.173+0000 7fb37f200640 1 -- start start 2024-04-25T19:09:25.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.173+0000 7fb37f200640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb380100ab0 0x7fb38019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.173+0000 7fb37f200640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb380101460 0x7fb38019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.173+0000 7fb37f200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb38010e660 0x7fb38019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.173+0000 7fb37f200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb3801141c0 con 0x7fb38010e660 2024-04-25T19:09:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.173+0000 7fb37f200640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fb380114040 con 0x7fb380101460 2024-04-25T19:09:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.173+0000 7fb37f200640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb380114340 con 0x7fb380100ab0 2024-04-25T19:09:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb38010e660 0x7fb38019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb380101460 0x7fb38019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37de00640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb380100ab0 0x7fb38019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:25.179 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37d400640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb380101460 0x7fb38019ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:57484/0 (socket says 172.21.15.45:57484) 2024-04-25T19:09:25.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37d400640 1 -- 172.21.15.45:0/3491427083 learned_addr learned my addr 172.21.15.45:0/3491427083 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:09:25.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37de00640 1 -- 172.21.15.45:0/3491427083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb380101460 msgr2=0x7fb38019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:25.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37de00640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb380101460 0x7fb38019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37de00640 1 -- 172.21.15.45:0/3491427083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb38010e660 msgr2=0x7fb38019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:25.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37de00640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb38010e660 0x7fb38019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37de00640 1 -- 172.21.15.45:0/3491427083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38019f720 con 0x7fb380100ab0 2024-04-25T19:09:25.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.174+0000 7fb37e800640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb38010e660 0x7fb38019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:25.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.175+0000 7fb37de00640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb380100ab0 0x7fb38019a6e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e9f0 tx=0x7fb37000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:25.180 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.175+0000 7fb376a00640 1 -- 172.21.15.45:0/3491427083 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37000cd80 con 0x7fb380100ab0 2024-04-25T19:09:25.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.175+0000 7fb376a00640 1 -- 172.21.15.45:0/3491427083 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3700035f0 con 0x7fb380100ab0 2024-04-25T19:09:25.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.175+0000 7fb37f200640 1 -- 172.21.15.45:0/3491427083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38019fa10 con 0x7fb380100ab0 2024-04-25T19:09:25.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.175+0000 7fb376a00640 1 -- 172.21.15.45:0/3491427083 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370010690 con 0x7fb380100ab0 2024-04-25T19:09:25.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.175+0000 7fb37f200640 1 -- 172.21.15.45:0/3491427083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3801a7250 con 0x7fb380100ab0 2024-04-25T19:09:25.181 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.176+0000 7fb37f200640 1 -- 172.21.15.45:0/3491427083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38004fce0 con 0x7fb380100ab0 2024-04-25T19:09:25.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.178+0000 7fb376a00640 1 -- 172.21.15.45:0/3491427083 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fb3700037a0 con 0x7fb380100ab0 2024-04-25T19:09:25.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.179+0000 7fb376a00640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb3600a3d20 0x7fb3600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:25.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.180+0000 7fb376a00640 1 -- 172.21.15.45:0/3491427083 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7fb3700d0b50 con 0x7fb380100ab0 2024-04-25T19:09:25.182 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.180+0000 7fb37d400640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb3600a3d20 0x7fb3600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:25.183 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.180+0000 7fb37d400640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb3600a3d20 0x7fb3600a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb38010fa10 tx=0x7fb368010040 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:25.186 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.184+0000 7fb376a00640 1 -- 172.21.15.45:0/3491427083 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb37009d160 con 0x7fb380100ab0 2024-04-25T19:09:25.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:25 smithi045 ceph-mon[27776]: from='client.15936 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:09:25.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:25 smithi045 ceph-mon[27776]: from='client.26093 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:09:25.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:25 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/101161280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:09:25.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:25 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2538039779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T19:09:25.396 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.394+0000 7fb37f200640 1 -- 172.21.15.45:0/3491427083 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb38002d5c0 con 0x7fb380100ab0 2024-04-25T19:09:25.397 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.395+0000 7fb376a00640 1 -- 172.21.15.45:0/3491427083 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) ==== 104+0+11 (secure 0 0 0) 0x7fb37009ca60 con 0x7fb380100ab0 2024-04-25T19:09:25.397 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 32 2024-04-25T19:09:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 -- 172.21.15.45:0/3491427083 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb3600a3d20 msgr2=0x7fb3600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb3600a3d20 0x7fb3600a61e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb38010fa10 tx=0x7fb368010040 comp rx=0 tx=0).stop 2024-04-25T19:09:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 -- 172.21.15.45:0/3491427083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb380100ab0 msgr2=0x7fb38019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb380100ab0 0x7fb38019a6e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb37000e9f0 tx=0x7fb37000eeb0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 -- 172.21.15.45:0/3491427083 shutdown_connections 2024-04-25T19:09:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fb3600a3d20 0x7fb3600a61e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb38010e660 0x7fb38019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fb380101460 0x7fb38019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 --2- 172.21.15.45:0/3491427083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb380100ab0 0x7fb38019a6e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.400 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 -- 172.21.15.45:0/3491427083 >> 172.21.15.45:0/3491427083 conn(0x7fb380077a00 msgr2=0x7fb3800ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 -- 172.21.15.45:0/3491427083 shutdown_connections 2024-04-25T19:09:25.401 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.398+0000 7fb37f200640 1 -- 172.21.15.45:0/3491427083 wait complete. 2024-04-25T19:09:25.420 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T19:09:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:25 smithi079 ceph-mon[33002]: from='client.15936 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:09:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:25 smithi079 ceph-mon[33002]: from='client.26093 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:09:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:25 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/101161280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:09:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:25 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2538039779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T19:09:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:25 smithi062 ceph-mon[32844]: from='client.15936 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:09:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:25 smithi062 ceph-mon[32844]: from='client.26093 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T19:09:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:25 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/101161280' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:09:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:25 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2538039779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T19:09:25.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.635+0000 7fad29000640 1 -- 172.21.15.45:0/608131624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad24108ff0 msgr2=0x7fad24115510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:25.637 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.635+0000 7fad29000640 1 --2- 172.21.15.45:0/608131624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad24108ff0 0x7fad24115510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009c10 tx=0x7fad1c02a6f0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.635+0000 7fad29000640 1 -- 172.21.15.45:0/608131624 shutdown_connections 2024-04-25T19:09:25.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.635+0000 7fad29000640 1 --2- 172.21.15.45:0/608131624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad24108ff0 0x7fad24115510 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.635+0000 7fad29000640 1 --2- 172.21.15.45:0/608131624 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad24108670 0x7fad24108ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.635+0000 7fad29000640 1 --2- 172.21.15.45:0/608131624 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad2410a3f0 0x7fad2410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.635+0000 7fad29000640 1 -- 172.21.15.45:0/608131624 >> 172.21.15.45:0/608131624 conn(0x7fad24077dc0 msgr2=0x7fad24106d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:25.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.635+0000 7fad29000640 1 -- 172.21.15.45:0/608131624 shutdown_connections 2024-04-25T19:09:25.638 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.635+0000 7fad29000640 1 -- 172.21.15.45:0/608131624 wait complete. 2024-04-25T19:09:25.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.636+0000 7fad29000640 1 Processor -- start 2024-04-25T19:09:25.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.636+0000 7fad29000640 1 -- start start 2024-04-25T19:09:25.640 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.636+0000 7fad29000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad24108670 0x7fad2419ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.636+0000 7fad29000640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad24108ff0 0x7fad2419f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad29000640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad2410a3f0 0x7fad241a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad29000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fad24118870 con 0x7fad24108670 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad29000640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fad241186f0 con 0x7fad2410a3f0 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad29000640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fad241189f0 con 0x7fad24108ff0 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad21e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad24108670 0x7fad2419ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad21e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad24108670 0x7fad2419ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45220/0 (socket says 172.21.15.45:45220) 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad21e00640 1 -- 172.21.15.45:0/907725964 learned_addr learned my addr 172.21.15.45:0/907725964 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad22800640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad2410a3f0 0x7fad241a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad21400640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad24108ff0 0x7fad2419f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad21e00640 1 -- 172.21.15.45:0/907725964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad24108ff0 msgr2=0x7fad2419f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad21e00640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad24108ff0 0x7fad2419f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad21e00640 1 -- 172.21.15.45:0/907725964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad2410a3f0 msgr2=0x7fad241a3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:25.641 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad21e00640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad2410a3f0 0x7fad241a3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.637+0000 7fad21e00640 1 -- 172.21.15.45:0/907725964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad241a3d30 con 0x7fad24108670 2024-04-25T19:09:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.638+0000 7fad21e00640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad24108670 0x7fad2419ed90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fad1400e9c0 tx=0x7fad1400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.638+0000 7fad1aa00640 1 -- 172.21.15.45:0/907725964 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1400ccc0 con 0x7fad24108670 2024-04-25T19:09:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.638+0000 7fad1aa00640 1 -- 172.21.15.45:0/907725964 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad140054d0 con 0x7fad24108670 2024-04-25T19:09:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.638+0000 7fad29000640 1 -- 172.21.15.45:0/907725964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad241a4020 con 0x7fad24108670 2024-04-25T19:09:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.638+0000 7fad1aa00640 1 -- 172.21.15.45:0/907725964 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14010430 con 0x7fad24108670 2024-04-25T19:09:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.638+0000 7fad29000640 1 -- 172.21.15.45:0/907725964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad241ab8b0 con 0x7fad24108670 2024-04-25T19:09:25.642 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.639+0000 7fad29000640 1 -- 172.21.15.45:0/907725964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad24069250 con 0x7fad24108670 2024-04-25T19:09:25.643 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.642+0000 7fad1aa00640 1 -- 172.21.15.45:0/907725964 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fad14004f80 con 0x7fad24108670 2024-04-25T19:09:25.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.643+0000 7fad1aa00640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fad080a3d20 0x7fad080a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:25.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.643+0000 7fad1aa00640 1 -- 172.21.15.45:0/907725964 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7fad140d0fa0 con 0x7fad24108670 2024-04-25T19:09:25.645 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.643+0000 7fad21400640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fad080a3d20 0x7fad080a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:25.646 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.644+0000 7fad21400640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fad080a3d20 0x7fad080a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fad2410eee0 tx=0x7fad10005f90 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:25.647 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.645+0000 7fad1aa00640 1 -- 172.21.15.45:0/907725964 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fad1409d5b0 con 0x7fad24108670 2024-04-25T19:09:25.853 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.851+0000 7fad29000640 1 -- 172.21.15.45:0/907725964 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad24110bd0 con 0x7fad24108670 2024-04-25T19:09:25.854 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.852+0000 7fad1aa00640 1 -- 172.21.15.45:0/907725964 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) ==== 100+0+11 (secure 0 0 0) 0x7fad1409ceb0 con 0x7fad24108670 2024-04-25T19:09:25.854 INFO:teuthology.orchestra.run.smithi045.stdout:pg_num: 64 2024-04-25T19:09:25.857 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 -- 172.21.15.45:0/907725964 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fad080a3d20 msgr2=0x7fad080a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fad080a3d20 0x7fad080a61e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fad2410eee0 tx=0x7fad10005f90 comp rx=0 tx=0).stop 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 -- 172.21.15.45:0/907725964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad24108670 msgr2=0x7fad2419ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad24108670 0x7fad2419ed90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fad1400e9c0 tx=0x7fad1400ee80 comp rx=0 tx=0).stop 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 -- 172.21.15.45:0/907725964 shutdown_connections 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fad080a3d20 0x7fad080a61e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fad2410a3f0 0x7fad241a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad24108ff0 0x7fad2419f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 --2- 172.21.15.45:0/907725964 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fad24108670 0x7fad2419ed90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 -- 172.21.15.45:0/907725964 >> 172.21.15.45:0/907725964 conn(0x7fad24077dc0 msgr2=0x7fad24078fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 -- 172.21.15.45:0/907725964 shutdown_connections 2024-04-25T19:09:25.858 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:25.855+0000 7fad29000640 1 -- 172.21.15.45:0/907725964 wait complete. 2024-04-25T19:09:25.878 DEBUG:teuthology.orchestra.run.smithi045:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T19:09:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.073+0000 7f2b7e800640 1 -- 172.21.15.45:0/1003424176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b7810c400 msgr2=0x7f2b7810e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:26.076 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.073+0000 7f2b7e800640 1 --2- 172.21.15.45:0/1003424176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b7810c400 0x7f2b7810e8a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009ca0 tx=0x7f2b7002a950 comp rx=0 tx=0).stop 2024-04-25T19:09:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.074+0000 7f2b7e800640 1 -- 172.21.15.45:0/1003424176 shutdown_connections 2024-04-25T19:09:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.074+0000 7f2b7e800640 1 --2- 172.21.15.45:0/1003424176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b7810c400 0x7f2b7810e8a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.074+0000 7f2b7e800640 1 --2- 172.21.15.45:0/1003424176 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78103900 0x7f2b7810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.074+0000 7f2b7e800640 1 --2- 172.21.15.45:0/1003424176 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b78102fe0 0x7f2b781033c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.074+0000 7f2b7e800640 1 -- 172.21.15.45:0/1003424176 >> 172.21.15.45:0/1003424176 conn(0x7f2b780fc7e0 msgr2=0x7f2b780fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.074+0000 7f2b7e800640 1 -- 172.21.15.45:0/1003424176 shutdown_connections 2024-04-25T19:09:26.077 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.074+0000 7f2b7e800640 1 -- 172.21.15.45:0/1003424176 wait complete. 2024-04-25T19:09:26.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.075+0000 7f2b7e800640 1 Processor -- start 2024-04-25T19:09:26.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.075+0000 7f2b7e800640 1 -- start start 2024-04-25T19:09:26.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.075+0000 7f2b7e800640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78102fe0 0x7f2b78198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:26.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.075+0000 7f2b7e800640 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78103900 0x7f2b78198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:26.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.075+0000 7f2b7e800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7810c400 0x7f2b7819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:26.079 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.075+0000 7f2b7e800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781120e0 con 0x7f2b7810c400 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.075+0000 7f2b7e800640 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f2b78111f60 con 0x7f2b78103900 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.075+0000 7f2b7e800640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2b78112260 con 0x7f2b78102fe0 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.075+0000 7f2b7de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7810c400 0x7f2b7819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7de00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7810c400 0x7f2b7819ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45236/0 (socket says 172.21.15.45:45236) 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7de00640 1 -- 172.21.15.45:0/3944617017 learned_addr learned my addr 172.21.15.45:0/3944617017 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7d400640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78102fe0 0x7f2b78198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7ca00640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78103900 0x7f2b78198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7ca00640 1 -- 172.21.15.45:0/3944617017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78102fe0 msgr2=0x7f2b78198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7ca00640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78102fe0 0x7f2b78198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7ca00640 1 -- 172.21.15.45:0/3944617017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7810c400 msgr2=0x7f2b7819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7ca00640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7810c400 0x7f2b7819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7ca00640 1 -- 172.21.15.45:0/3944617017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7819d5e0 con 0x7f2b78103900 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7de00640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7810c400 0x7f2b7819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:09:26.080 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.076+0000 7f2b7ca00640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78103900 0x7f2b78198b80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400e7c0 tx=0x7f2b6400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.077+0000 7f2b76000640 1 -- 172.21.15.45:0/3944617017 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64009530 con 0x7f2b78103900 2024-04-25T19:09:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.077+0000 7f2b7e800640 1 -- 172.21.15.45:0/3944617017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7819d8d0 con 0x7f2b78103900 2024-04-25T19:09:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.077+0000 7f2b76000640 1 -- 172.21.15.45:0/3944617017 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b6400ccc0 con 0x7f2b78103900 2024-04-25T19:09:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.077+0000 7f2b76000640 1 -- 172.21.15.45:0/3944617017 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64004a00 con 0x7f2b78103900 2024-04-25T19:09:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.077+0000 7f2b7e800640 1 -- 172.21.15.45:0/3944617017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b781a5160 con 0x7f2b78103900 2024-04-25T19:09:26.081 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.078+0000 7f2b7e800640 1 -- 172.21.15.45:0/3944617017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7804fd50 con 0x7f2b78103900 2024-04-25T19:09:26.083 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.081+0000 7f2b76000640 1 -- 172.21.15.45:0/3944617017 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f2b64004ba0 con 0x7f2b78103900 2024-04-25T19:09:26.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.083+0000 7f2b76000640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b600a3d10 0x7f2b600a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:09:26.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.083+0000 7f2b76000640 1 -- 172.21.15.45:0/3944617017 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f2b640d0720 con 0x7f2b78103900 2024-04-25T19:09:26.085 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.083+0000 7f2b7d400640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b600a3d10 0x7f2b600a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:09:26.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.084+0000 7f2b7d400640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b600a3d10 0x7f2b600a61d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009bd0 tx=0x7f2b6c005f90 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:09:26.086 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.084+0000 7f2b76000640 1 -- 172.21.15.45:0/3944617017 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b6409cdc0 con 0x7f2b78103900 2024-04-25T19:09:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:26 smithi045 ceph-mon[27776]: pgmap v315: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T19:09:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:26 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3491427083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T19:09:26.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:26 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/907725964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T19:09:26.345 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.343+0000 7f2b7e800640 1 -- 172.21.15.45:0/3944617017 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f2b78199940 con 0x7f2b78103900 2024-04-25T19:09:26.358 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.356+0000 7f2b76000640 1 -- 172.21.15.45:0/3944617017 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v27) ==== 81+0+746062 (secure 0 0 0) 0x7f2b78199940 con 0x7f2b78103900 2024-04-25T19:09:26.371 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:09:26.371 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "epoch": 27, 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "active_gid": 25049, 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "active_name": "x", 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addrs": { 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6800", 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1345298339 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v1", 2024-04-25T19:09:26.372 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:6801", 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 1345298339 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "active_addr": "172.21.15.45:6801/1345298339", 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "active_change": "2024-04-25T18:59:18.747215+0000", 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "available": true, 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "standbys": [ 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 25045, 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "z", 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701547738038271, 2024-04-25T19:09:26.373 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.383 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.384 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.387 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-04-25T19:09:26.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.392 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-04-25T19:09:26.393 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T19:09:26.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.396 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.398 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T19:09:26.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-04-25T19:09:26.402 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.405 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.407 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-04-25T19:09:26.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.411 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-04-25T19:09:26.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-04-25T19:09:26.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.416 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:26.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.421 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.422 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.423 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-04-25T19:09:26.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-04-25T19:09:26.425 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-04-25T19:09:26.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.429 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T19:09:26.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.432 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.434 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.437 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-04-25T19:09:26.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.441 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.442 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.443 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.445 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.446 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.447 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T19:09:26.449 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.452 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.454 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-04-25T19:09:26.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.457 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.458 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-04-25T19:09:26.460 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.461 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.462 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T19:09:26.463 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.466 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.467 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.470 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.471 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-04-25T19:09:26.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.473 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T19:09:26.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.485 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T19:09:26.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.488 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.491 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.492 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.493 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "[]", 2024-04-25T19:09:26.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "{}", 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.496 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.498 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.499 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.500 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T19:09:26.501 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.503 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.505 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.507 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.508 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T19:09:26.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T19:09:26.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:26.513 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.514 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.516 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.517 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.519 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.520 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.521 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.522 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.525 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.526 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T19:09:26.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T19:09:26.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T19:09:26.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T19:09:26.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.533 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.534 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.535 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.536 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.537 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.538 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.541 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.542 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.543 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:26.545 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.546 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.550 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.551 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.553 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.554 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-04-25T19:09:26.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-04-25T19:09:26.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-04-25T19:09:26.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:26.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.561 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.562 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-04-25T19:09:26.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.569 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.570 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.572 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.573 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.574 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.575 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.577 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.578 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.580 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.582 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.584 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.585 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.586 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.587 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.588 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-04-25T19:09:26.589 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.590 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.591 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.593 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.594 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-04-25T19:09:26.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-04-25T19:09:26.596 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-04-25T19:09:26.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-04-25T19:09:26.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-04-25T19:09:26.600 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.601 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.602 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.603 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.604 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.606 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.607 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.608 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.609 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.610 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.611 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.612 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.613 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.614 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.615 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.616 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.617 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.618 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.619 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.620 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.621 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.622 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.623 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.624 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.625 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.626 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.627 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.628 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.629 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.630 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.631 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.632 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.633 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.634 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.635 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.636 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.637 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.638 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.639 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.640 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.641 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.642 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.643 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.644 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.645 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.646 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-04-25T19:09:26.647 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.648 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-04-25T19:09:26.649 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.650 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.651 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.652 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.653 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.654 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.655 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.656 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-04-25T19:09:26.657 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-04-25T19:09:26.658 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.659 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.660 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.661 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-04-25T19:09:26.662 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T19:09:26.663 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.664 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.665 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.666 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.667 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.668 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.669 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.670 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-04-25T19:09:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-04-25T19:09:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.671 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-04-25T19:09:26.672 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.673 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.674 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.675 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.676 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.677 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.678 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T19:09:26.679 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.680 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-04-25T19:09:26.681 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.682 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.683 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.684 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.685 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.686 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.687 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.688 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.689 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.690 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.691 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.693 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.694 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.695 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.696 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.697 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.698 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.699 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.700 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.701 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.702 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.703 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.704 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.705 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-04-25T19:09:26.706 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.707 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.708 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.709 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.710 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.711 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.712 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.713 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.714 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.715 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.716 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.717 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.718 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.719 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.720 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.721 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.722 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.723 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.724 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.725 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.726 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.727 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.728 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.729 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.730 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.731 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.732 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.733 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.734 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.735 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T19:09:26.736 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.737 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.738 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.739 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.740 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.741 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.742 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.743 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-04-25T19:09:26.744 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.745 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.746 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.747 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.748 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.749 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.750 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.751 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.752 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.753 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.754 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.755 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.756 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.757 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.758 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.759 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.760 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.761 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.762 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.763 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.764 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.765 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T19:09:26.766 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T19:09:26.767 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:26.768 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.769 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.770 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.771 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.772 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.773 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.774 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.775 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.776 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.777 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.778 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "gid": 25046, 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "y", 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "mgr_features": 4540701547738038271, 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-04-25T19:09:26.779 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.780 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.781 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.782 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.783 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.784 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.785 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.786 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.787 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.788 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-04-25T19:09:26.789 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.790 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.791 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.792 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-04-25T19:09:26.793 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-04-25T19:09:26.794 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-04-25T19:09:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-04-25T19:09:26.807 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.808 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T19:09:26.809 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T19:09:26.810 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.811 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T19:09:26.812 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-04-25T19:09:26.813 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-04-25T19:09:26.814 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.815 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:26.816 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.817 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.818 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.819 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.820 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.821 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T19:09:26.822 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.823 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.824 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.825 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.826 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.827 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.828 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-04-25T19:09:26.829 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.830 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.831 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.832 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-04-25T19:09:26.833 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-04-25T19:09:26.834 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-04-25T19:09:26.835 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-04-25T19:09:26.836 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-04-25T19:09:26.837 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.838 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.839 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.840 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.841 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.842 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.843 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.844 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.845 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T19:09:26.846 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T19:09:26.847 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.848 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.849 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.850 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.851 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.852 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.853 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.854 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-04-25T19:09:26.855 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-04-25T19:09:26.856 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.858 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-04-25T19:09:26.859 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-04-25T19:09:26.860 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-04-25T19:09:26.861 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-04-25T19:09:26.862 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.863 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.864 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.865 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-04-25T19:09:26.866 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.867 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.868 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.869 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.870 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.871 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.872 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.873 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.874 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.875 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.876 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.877 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.878 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-04-25T19:09:26.879 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.880 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T19:09:26.881 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T19:09:26.882 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.883 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T19:09:26.884 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.885 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.886 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.887 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.888 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.889 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.890 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.892 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.893 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.894 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.895 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.896 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.897 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.898 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.899 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.900 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.901 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "[]", 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.902 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "{}", 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.903 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.904 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.905 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.906 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.907 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.908 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.909 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.910 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.911 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T19:09:26.912 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T19:09:26.913 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T19:09:26.914 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T19:09:26.915 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.916 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.917 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.918 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.919 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.920 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.921 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:26.922 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.923 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.924 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.925 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.926 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.927 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.928 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.929 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.930 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.931 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.932 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.933 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.934 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.935 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.936 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.937 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.938 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.939 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.940 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.941 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T19:09:26.942 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.943 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-04-25T19:09:26.944 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.945 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.946 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.947 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.948 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.949 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.950 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.951 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.952 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.953 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T19:09:26.954 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.955 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.956 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.957 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.958 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.959 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.960 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.961 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.962 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.963 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.964 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.965 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.966 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.967 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-04-25T19:09:26.968 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.969 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.970 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.971 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.972 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.973 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.974 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.975 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:09:26.977 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.978 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.979 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.980 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.981 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.982 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.983 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.984 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.985 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.986 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.987 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.988 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.989 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.990 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.991 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.992 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.993 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.994 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.995 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:26.996 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:26.997 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-04-25T19:09:26.998 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:26.999 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.000 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.001 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.002 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.003 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.004 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.005 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.006 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-04-25T19:09:27.007 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-04-25T19:09:27.008 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.009 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.010 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.011 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.012 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.013 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.014 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.015 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.016 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.017 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.018 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.019 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.020 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.021 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.022 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.023 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.024 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.025 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.026 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.027 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.028 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.029 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.030 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-25T19:09:27.031 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.032 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.033 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.034 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.035 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.036 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.037 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.038 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.039 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.040 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.041 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.042 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.043 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.044 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.045 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.046 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.047 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.048 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.049 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.050 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.051 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.052 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.053 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.054 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.055 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.056 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.057 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.058 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.059 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.060 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.061 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.062 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.063 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.064 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.065 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.066 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.067 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.068 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T19:09:27.069 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-04-25T19:09:27.070 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.071 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T19:09:27.072 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.073 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.074 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.075 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.076 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.077 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.080 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.082 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.083 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.084 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.085 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.086 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.087 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T19:09:27.088 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.089 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-04-25T19:09:27.090 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.091 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.092 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.093 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.095 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.096 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.097 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.098 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.099 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.100 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T19:09:27.101 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.102 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.104 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.105 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.106 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.107 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.108 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.110 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.111 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-04-25T19:09:27.112 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-04-25T19:09:27.113 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-04-25T19:09:27.114 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-04-25T19:09:27.115 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.116 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.117 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.118 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.119 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.120 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.121 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.122 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.123 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.124 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.125 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.126 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.127 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.128 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.129 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.130 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.131 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.132 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.133 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.134 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.135 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.136 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.137 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.138 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.139 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.140 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.141 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.142 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-04-25T19:09:27.143 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-04-25T19:09:27.144 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.145 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.146 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.147 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.148 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.149 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.150 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.151 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.152 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.153 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.154 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.155 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.156 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.157 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.158 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.159 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-04-25T19:09:27.160 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.161 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-04-25T19:09:27.162 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-04-25T19:09:27.163 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.164 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.165 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.166 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.167 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.168 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.169 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.170 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.171 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.172 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-04-25T19:09:27.173 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.174 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-04-25T19:09:27.175 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T19:09:27.176 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.177 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.178 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.179 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:27.181 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.182 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.183 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.184 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.185 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.186 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.187 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.188 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.189 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:27.190 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "modules": [ 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard", 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "iostat", 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "nfs", 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "restful", 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "snap_schedule" 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "available_modules": [ 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "alerts", 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.191 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.192 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.193 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.194 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.195 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.196 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_destination": { 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_destination", 2024-04-25T19:09:27.197 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email address to send alerts to", 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_from_name": { 2024-04-25T19:09:27.198 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_from_name", 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "Ceph", 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Email From: name", 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_host": { 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_host", 2024-04-25T19:09:27.199 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP server", 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_password": { 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_password", 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.200 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Password to authenticate with", 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_port": { 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_port", 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.201 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "465", 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP port", 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_sender": { 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_sender", 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.202 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SMTP envelope sender", 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_ssl": { 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_ssl", 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.203 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "smtp_user": { 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "smtp_user", 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.204 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "User to authenticate as", 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "balancer", 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.205 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "active": { 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "active", 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.206 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_time": { 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_time", 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0000", 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.207 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "begin_weekday": { 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "begin_weekday", 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.208 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_max_iterations": { 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_max_iterations", 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "25", 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "250", 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.209 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_metrics": { 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_metrics", 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "crush_compat_step": { 2024-04-25T19:09:27.210 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crush_compat_step", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.5", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0.001", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "0.999", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "aggressiveness of optimization", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "end_time": { 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_time", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.211 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2359", 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "end_weekday": { 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "end_weekday", 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:27.212 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "0", 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "6", 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.213 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.214 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.215 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.216 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.217 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "min_score": { 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_score", 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.218 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "upmap", 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "crush-compat", 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "none", 2024-04-25T19:09:27.219 INFO:teuthology.orchestra.run.smithi045.stdout: "read", 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap", 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap-read" 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Balancer mode", 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_ids": { 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_ids", 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.220 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.221 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_deviation": { 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_deviation", 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.222 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "upmap_max_optimizations": { 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "upmap_max_optimizations", 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.223 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm", 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_down_multiplier": { 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_down_multiplier", 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:27.224 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_refresh_rate": { 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_refresh_rate", 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.225 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "20", 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "agent_starting_port": { 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "agent_starting_port", 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.226 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4721", 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_ptrace": { 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_ptrace", 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.227 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_interval": { 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_interval", 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.228 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "autotune_memory_target_ratio": { 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.229 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0.7", 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm_log_destination": { 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cephadm_log_destination", 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.230 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "file", 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "file,syslog", 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "syslog" 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "cgroups_split": { 2024-04-25T19:09:27.231 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cgroups_split", 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "config_checks_enabled": { 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_checks_enabled", 2024-04-25T19:09:27.232 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "config_dashboard": { 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_dashboard", 2024-04-25T19:09:27.233 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_alertmanager": { 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_alertmanager", 2024-04-25T19:09:27.234 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_base": { 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_base", 2024-04-25T19:09:27.235 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Container image name, without the tag", 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_elasticsearch": { 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_elasticsearch", 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.236 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "elasticsearch container image", 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_grafana": { 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_grafana", 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.237 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_haproxy": { 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_haproxy", 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.238 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "HAproxy container image", 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_agent": { 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_agent", 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.239 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger agent container image", 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_collector": { 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_collector", 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.240 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger collector container image", 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_jaeger_query": { 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_jaeger_query", 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T19:09:27.241 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Jaeger query container image", 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_keepalived": { 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_keepalived", 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T19:09:27.242 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Keepalived container image", 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_loki": { 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_loki", 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.243 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Loki container image", 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_node_exporter": { 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_node_exporter", 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.244 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_nvmeof": { 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_nvmeof", 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.245 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Nvme-of container image", 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_prometheus": { 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_prometheus", 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.246 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Prometheus container image", 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_promtail": { 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_promtail", 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.247 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Promtail container image", 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_samba": { 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_samba", 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.248 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Samba/SMB container image", 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "container_image_snmp_gateway": { 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_image_snmp_gateway", 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "SNMP Gateway container image", 2024-04-25T19:09:27.249 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "container_init": { 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "container_init", 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.250 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "daemon_cache_timeout": { 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "daemon_cache_timeout", 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.251 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "default_cephadm_command_timeout": { 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "900", 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.252 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_registry": { 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "default_registry", 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "docker.io", 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.253 INFO:teuthology.orchestra.run.smithi045.stdout: "device_cache_timeout": { 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_cache_timeout", 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1800", 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache device inventory", 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "device_enhanced_scan": { 2024-04-25T19:09:27.254 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_enhanced_scan", 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "facts_cache_timeout": { 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "facts_cache_timeout", 2024-04-25T19:09:27.255 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "seconds to cache host facts data", 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "grafana_dashboards_path": { 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "grafana_dashboards_path", 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.256 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "host_check_interval": { 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "host_check_interval", 2024-04-25T19:09:27.257 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to perform a host check", 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "hw_monitoring": { 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hw_monitoring", 2024-04-25T19:09:27.258 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "inventory_list_all": { 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "inventory_list_all", 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.259 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.260 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.261 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "log_refresh_metadata": { 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_refresh_metadata", 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.262 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.263 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.264 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.265 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.266 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "*", 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.267 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "max_count_per_host": { 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_count_per_host", 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.268 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of daemons per service per host", 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max_osd_draining_count": { 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_osd_draining_count", 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T19:09:27.269 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "migration_current": { 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "migration_current", 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "internal - do not modify", 2024-04-25T19:09:27.270 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "mode": { 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mode", 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "root", 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm-package", 2024-04-25T19:09:27.271 INFO:teuthology.orchestra.run.smithi045.stdout: "root" 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "oob_default_addr": { 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "oob_default_addr", 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "169.254.1.1", 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.272 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "prometheus_alerts_path": { 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus_alerts_path", 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.273 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_insecure": { 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_insecure", 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.274 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_password": { 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_password", 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T19:09:27.275 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_url": { 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_url", 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T19:09:27.276 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "registry_username": { 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "registry_username", 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.277 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "secure_monitoring_stack": { 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "secure_monitoring_stack", 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.278 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "service_discovery_port": { 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "service_discovery_port", 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8765", 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "cephadm service discovery port", 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.279 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "ssh_config_file": { 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssh_config_file", 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.280 INFO:teuthology.orchestra.run.smithi045.stdout: "use_agent": { 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_agent", 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "use_repo_digest": { 2024-04-25T19:09:27.281 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "use_repo_digest", 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_failed_host_check": { 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_failed_host_check", 2024-04-25T19:09:27.282 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_daemons": { 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_daemons", 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.283 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_on_stray_hosts": { 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_on_stray_hosts", 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.284 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crash", 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.285 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.286 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.287 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.288 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.289 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "retain_interval": { 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retain_interval", 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.290 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "31536000", 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_recent_interval": { 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_recent_interval", 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.291 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1209600", 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dashboard", 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.292 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.293 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.294 INFO:teuthology.orchestra.run.smithi045.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_ENABLED": { 2024-04-25T19:09:27.295 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T19:09:27.296 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.297 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.298 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.299 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.300 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.301 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.302 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.303 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.304 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.305 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T19:09:27.306 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.307 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_URL": { 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_URL", 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.309 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.310 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.311 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.312 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.313 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "[]", 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.314 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "{}", 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.315 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.316 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.317 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.318 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.319 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T19:09:27.320 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T19:09:27.321 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T19:09:27.322 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.323 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.324 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.325 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.326 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8", 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.327 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "45", 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.328 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.329 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "admin", 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.330 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.331 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.332 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.333 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.334 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.335 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.336 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "cross_origin_url": { 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cross_origin_url", 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.337 INFO:teuthology.orchestra.run.smithi045.stdout: "crt_file": { 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "crt_file", 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "debug": { 2024-04-25T19:09:27.338 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "debug", 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Enable/disable debug options", 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "jwt_token_ttl": { 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "jwt_token_ttl", 2024-04-25T19:09:27.339 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "28800", 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.340 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.341 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.342 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.343 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.344 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.345 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "motd": { 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "motd", 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.346 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "The message of the day", 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect_resolve_ip_addr": { 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.347 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.348 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8080", 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.349 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.350 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl_server_port": { 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl_server_port", 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8443", 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.351 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "redirect", 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "redirect" 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.352 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.353 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "url_prefix": { 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url_prefix", 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.354 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_monitoring": { 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_monitoring", 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.355 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "monitor device health metrics", 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.356 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.357 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.358 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.359 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.360 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "mark_out_threshold": { 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mark_out_threshold", 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "2419200", 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.361 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "pool_name": { 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pool_name", 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "device_health_metrics", 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.362 INFO:teuthology.orchestra.run.smithi045.stdout: "retention_period": { 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "retention_period", 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15552000", 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long to retain device health metrics", 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.363 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_frequency": { 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_frequency", 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.364 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "self_heal": { 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "self_heal", 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.365 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.366 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "warn_threshold": { 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "warn_threshold", 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7257600", 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.367 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "diskprediction_local", 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.368 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.369 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.370 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.371 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.372 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "predict_interval": { 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predict_interval", 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "86400", 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.373 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "predictor_model": { 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "predictor_model", 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "prophetstor", 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.374 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "600", 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.375 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "influx", 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": false, 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "influxdb python module not found", 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "batch_size": { 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "batch_size", 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5000", 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.376 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "database": { 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "database", 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "ceph", 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.377 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "hostname": { 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "hostname", 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.378 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server hostname", 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "30", 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "5", 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.379 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.380 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.381 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.382 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.383 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.385 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "password": { 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "password", 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.386 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "password of InfluxDB server user", 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "port": { 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "port", 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "8086", 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.387 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "InfluxDB server port", 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "ssl": { 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ssl", 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "false", 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:09:27.388 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "threads": { 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threads", 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "32", 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.389 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "username": { 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "username", 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "username of InfluxDB server user", 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.390 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "verify_ssl": { 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "verify_ssl", 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "true", 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.391 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "insights", 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.392 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.393 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.394 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.395 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.396 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "iostat", 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.397 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.398 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.399 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.400 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.401 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "k8sevents", 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_event_retention_days": { 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "ceph_event_retention_days", 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.402 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "7", 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "config_check_secs": { 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "config_check_secs", 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.403 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "10", 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.404 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.406 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.407 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.408 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "localpool", 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "failure_domain": { 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "failure_domain", 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.409 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "host", 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "failure domain for any created local pool", 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.410 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.411 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.412 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.413 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.414 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min_size": { 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "min_size", 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.415 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default min_size for any created local pool", 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "num_rep": { 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "num_rep", 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3", 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.416 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default replica count for any created local pool", 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_num": { 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_num", 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "128", 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.417 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "prefix": { 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prefix", 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.418 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "name prefix for any created local pool", 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "subtree": { 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "subtree", 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "rack", 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.419 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mds_autoscaler", 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.420 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.421 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.422 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.423 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.424 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirroring", 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.425 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.426 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.427 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.428 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.429 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "nfs", 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.430 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.431 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.432 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.433 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.434 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.435 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "fail_fs": { 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "fail_fs", 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.436 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.437 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.438 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.439 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.440 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator": { 2024-04-25T19:09:27.441 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "orchestrator", 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "cephadm", 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "rook", 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "test_orchestrator" 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Orchestrator backend", 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.442 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_perf_query", 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.443 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.444 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.445 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.446 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.447 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "osd_support", 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.448 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.449 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.450 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.451 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.452 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "pg_autoscaler", 2024-04-25T19:09:27.453 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.454 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.455 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.456 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.457 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:27.458 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "threshold": { 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "threshold", 2024-04-25T19:09:27.459 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "3.0", 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1.0", 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "scaling threshold", 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.460 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "progress", 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_pg_recovery_event": { 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_pg_recovery_event", 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.461 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.462 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.463 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.464 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.465 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.466 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "max_completed_events": { 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_completed_events", 2024-04-25T19:09:27.467 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "50", 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "number of past completed events to remember", 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "sleep_interval": { 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "sleep_interval", 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.468 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "5", 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "how long the module is going to sleep", 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "prometheus", 2024-04-25T19:09:27.469 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "cache": { 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "cache", 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.470 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "exclude_perf_counters": { 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "exclude_perf_counters", 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.471 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.472 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.473 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.474 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.475 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools": { 2024-04-25T19:09:27.476 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools", 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T19:09:27.477 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "300", 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "scrape_interval": { 2024-04-25T19:09:27.478 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "scrape_interval", 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15.0", 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T19:09:27.479 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "::", 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T19:09:27.480 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "9283", 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "stale_cache_strategy": { 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stale_cache_strategy", 2024-04-25T19:09:27.481 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "log", 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_behaviour": { 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_behaviour", 2024-04-25T19:09:27.482 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "default", 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "default", 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "error" 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "standby_error_status_code": { 2024-04-25T19:09:27.483 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "standby_error_status_code", 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "500", 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "400", 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "599", 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.484 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.485 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.486 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.487 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.488 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.489 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_snap_create": { 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_snap_create", 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10", 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.490 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "mirror_snapshot_schedule": { 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.491 INFO:teuthology.orchestra.run.smithi045.stdout: "trash_purge_schedule": { 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "trash_purge_schedule", 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.492 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "restful", 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "enable_auth": { 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enable_auth", 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.493 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "key_file": { 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "key_file", 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.494 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.495 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.496 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.497 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.498 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.499 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "server_addr": { 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_addr", 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.500 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "server_port": { 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "server_port", 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.501 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rgw", 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.502 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.503 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.504 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.505 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.506 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rook", 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.507 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.508 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.509 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.510 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.511 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "storage_class": { 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "storage_class", 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "local", 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.512 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "selftest", 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.513 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.514 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.515 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.516 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.517 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "roption1": { 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption1", 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.518 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "roption2": { 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "roption2", 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.519 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "xyz", 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption1": { 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption1", 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.520 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption2": { 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption2", 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.521 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption3": { 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption3", 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "float", 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.522 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption4": { 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption4", 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.523 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption5": { 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption5", 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.524 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption6": { 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption6", 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.525 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "rwoption7": { 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rwoption7", 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "1", 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "42", 2024-04-25T19:09:27.526 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "testkey": { 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testkey", 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.527 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "testlkey": { 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testlkey", 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.528 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "testnewline": { 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "testnewline", 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.529 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "allow_m_granularity": { 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "allow_m_granularity", 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.530 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "dump_on_update": { 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "dump_on_update", 2024-04-25T19:09:27.531 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "dump database to debug log on update", 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.532 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.533 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.534 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.535 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.536 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "stats", 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.537 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.538 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.539 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.540 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.541 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "status", 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.542 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.543 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.544 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.545 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.546 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telegraf", 2024-04-25T19:09:27.547 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "address": { 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "address", 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.548 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "15", 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.549 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.550 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.551 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.552 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.553 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "telemetry", 2024-04-25T19:09:27.554 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_basic": { 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_basic", 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.555 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_crash": { 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_crash", 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.556 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_device": { 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_device", 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_ident": { 2024-04-25T19:09:27.557 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_ident", 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "channel_perf": { 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "channel_perf", 2024-04-25T19:09:27.558 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "contact": { 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "contact", 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.559 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "description": { 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "description", 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.560 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "device_url": { 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "device_url", 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.561 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "enabled": { 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "enabled", 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.562 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "24", 2024-04-25T19:09:27.563 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "8", 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "last_opt_revision": { 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "last_opt_revision", 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "1", 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.564 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard": { 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard", 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.565 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "leaderboard_description": { 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "leaderboard_description", 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.566 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.567 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.568 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.569 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.570 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "organization": { 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "organization", 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.571 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "proxy": { 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "proxy", 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.572 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "url": { 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "url", 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.573 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "test_orchestrator", 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.574 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.575 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.576 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.577 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.578 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.580 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.581 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.582 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.584 INFO:teuthology.orchestra.run.smithi045.stdout: "max_concurrent_clones": { 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "max_concurrent_clones", 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "4", 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "periodic_async_work": { 2024-04-25T19:09:27.585 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "periodic_async_work", 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Periodically check for async work", 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_delay": { 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_delay", 2024-04-25T19:09:27.586 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "0", 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "snapshot_clone_no_wait": { 2024-04-25T19:09:27.587 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "True", 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.589 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix", 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "can_run": true, 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "error_string": "", 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "module_options": { 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "discovery_interval": { 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "discovery_interval", 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "uint", 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "100", 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.590 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "identifier": { 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "identifier", 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.591 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "interval": { 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "interval", 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "secs", 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "60", 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.592 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "log_level": { 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_level", 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.593 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster": { 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster", 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.594 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_cluster_level": { 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_cluster_level", 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.595 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "info", 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [ 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "", 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "critical", 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "debug", 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "error", 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "info", 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "warning" 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "log_to_file": { 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "log_to_file", 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "bool", 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 1, 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "False", 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_host": { 2024-04-25T19:09:27.597 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_host", 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "", 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_port": { 2024-04-25T19:09:27.598 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_port", 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "int", 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "10051", 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "zabbix_sender": { 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "zabbix_sender", 2024-04-25T19:09:27.599 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "str", 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "level": "advanced", 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "flags": 0, 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "min": "", 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "max": "", 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "enum_allowed": [], 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "desc": "", 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "long_desc": "", 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "tags": [], 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "see_also": [] 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.600 INFO:teuthology.orchestra.run.smithi045.stdout: "services": { 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "dashboard": "https://172.21.15.45:8443/" 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "always_on_modules": { 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "octopus": [ 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-25T19:09:27.601 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "pacific": [ 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "quincy": [ 2024-04-25T19:09:27.602 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "reef": [ 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-25T19:09:27.603 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: ], 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "squid": [ 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "balancer", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "crash", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "devicehealth", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "orchestrator", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "pg_autoscaler", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "progress", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "rbd_support", 2024-04-25T19:09:27.604 INFO:teuthology.orchestra.run.smithi045.stdout: "status", 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "telemetry", 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "volumes" 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "last_failure_osd_epoch": 95, 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "active_clients": [ 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "devicehealth", 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 271400840 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.605 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "libcephsqlite", 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2619946372 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "rbd_support", 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T19:09:27.606 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2071830246 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2880576498 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.607 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "snap_schedule", 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 3265212237 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "name": "volumes", 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: "addrvec": [ 2024-04-25T19:09:27.608 INFO:teuthology.orchestra.run.smithi045.stdout: { 2024-04-25T19:09:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "type": "v2", 2024-04-25T19:09:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "addr": "172.21.15.45:0", 2024-04-25T19:09:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: "nonce": 2099612135 2024-04-25T19:09:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:27.609 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-25T19:09:27.609 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:09:27.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:26 smithi062 ceph-mon[32844]: pgmap v315: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T19:09:27.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:26 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3491427083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T19:09:27.610 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:26 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/907725964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T19:09:27.611 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:26 smithi079 ceph-mon[33002]: pgmap v315: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 204 B/s wr, 9 op/s 2024-04-25T19:09:27.611 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:26 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3491427083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T19:09:27.611 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:26 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/907725964' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T19:09:27.611 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:27 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3944617017' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.371+0000 7f2b7e800640 1 -- 172.21.15.45:0/3944617017 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b600a3d10 msgr2=0x7f2b600a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.371+0000 7f2b7e800640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b600a3d10 0x7f2b600a61d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009bd0 tx=0x7f2b6c005f90 comp rx=0 tx=0).stop 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.371+0000 7f2b7e800640 1 -- 172.21.15.45:0/3944617017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78103900 msgr2=0x7f2b78198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.371+0000 7f2b7e800640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78103900 0x7f2b78198b80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400e7c0 tx=0x7f2b6400ec80 comp rx=0 tx=0).stop 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.371+0000 7f2b7e800640 1 -- 172.21.15.45:0/3944617017 shutdown_connections 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.371+0000 7f2b7e800640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f2b600a3d10 0x7f2b600a61d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.371+0000 7f2b7e800640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b7810c400 0x7f2b7819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.371+0000 7f2b7e800640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f2b78103900 0x7f2b78198b80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.371+0000 7f2b7e800640 1 --2- 172.21.15.45:0/3944617017 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b78102fe0 0x7f2b78198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.371+0000 7f2b7e800640 1 -- 172.21.15.45:0/3944617017 >> 172.21.15.45:0/3944617017 conn(0x7f2b780fc7e0 msgr2=0x7f2b780fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.372+0000 7f2b7e800640 1 -- 172.21.15.45:0/3944617017 shutdown_connections 2024-04-25T19:09:27.612 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:09:26.372+0000 7f2b7e800640 1 -- 172.21.15.45:0/3944617017 wait complete. 2024-04-25T19:09:27.618 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T19:09:27.618 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T19:09:27.619 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:27 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3944617017' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T19:09:27.619 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:27 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3944617017' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T19:09:28.100 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:09:28.413 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:28 smithi045 ceph-mon[27776]: pgmap v316: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 193 B/s wr, 8 op/s 2024-04-25T19:09:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:28 smithi079 ceph-mon[33002]: pgmap v316: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 193 B/s wr, 8 op/s 2024-04-25T19:09:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:28 smithi062 ceph-mon[32844]: pgmap v316: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 193 B/s wr, 8 op/s 2024-04-25T19:09:30.236 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:30 smithi045 ceph-mon[27776]: pgmap v317: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T19:09:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:30 smithi079 ceph-mon[33002]: pgmap v317: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T19:09:30.545 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 277, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 357, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 78295, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 369491, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 19, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 5, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.127629857, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.067918262, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.052851374, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.005260111, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 351, 2024-04-25T19:09:30.546 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.051580343, 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000146952 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 389, 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000164915, 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000423 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3568, 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 4400 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4487, 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2576, 2024-04-25T19:09:30.547 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3898176, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 863555, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 12, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 5, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.589159246, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.258638886, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.465291780, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.008233443, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2577, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.204275656, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000079268 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6851, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.002929547, 2024-04-25T19:09:30.548 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000427 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 165472 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 355, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 449, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1840497, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 487406, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 7, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 6, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.152299883, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.051506757, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.121088959, 2024-04-25T19:09:30.549 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.013431536, 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 449, 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.093155041, 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000207472 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1825920, 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 459568 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.550 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 173, 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.057640478, 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000333181 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-25T19:09:30.551 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 58, 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.006785457, 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000116990 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 135, 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 42, 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 42, 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.097884641, 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002330586 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 93, 2024-04-25T19:09:30.552 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 11, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 19, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 2, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 7, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 10, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 13, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 168, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 1, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 1, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 4, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 493, 2024-04-25T19:09:30.553 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 300, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 93, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 65, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 0, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 4, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 38, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 4, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 18, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-04-25T19:09:30.554 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 4, 2024-04-25T19:09:30.555 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 234, 2024-04-25T19:09:30.555 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 6, 2024-04-25T19:09:30.555 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:09:30.555 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-04-25T19:09:30.555 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:09:30.555 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:09:30.555 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 0, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 15, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 25, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 19, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 6, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 7, 2024-04-25T19:09:30.556 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 36 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 6, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 531, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 296, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 303, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 6, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 16, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 16, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 0, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-04-25T19:09:30.557 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_path": 2, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_inode": 27, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 4, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-04-25T19:09:30.558 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 95, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 95, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 2, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 2, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 2, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 4194304, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 4374401, 2024-04-25T19:09:30.559 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 0, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 53, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.137768193, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002599399 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 0 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 24, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 133, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 109, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 23, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 272, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 249, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 19, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 205, 2024-04-25T19:09:30.560 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 186, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 1, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 31, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 30, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 54272, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 223608 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 168, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 9, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 135, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 20, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 64, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2, 2024-04-25T19:09:30.561 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.007583022, 2024-04-25T19:09:30.562 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.003791511 2024-04-25T19:09:30.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.562 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-04-25T19:09:30.562 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T19:09:30.562 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.002189502, 2024-04-25T19:09:30.562 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.002189502 2024-04-25T19:09:30.562 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.563 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-04-25T19:09:30.563 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.563 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.563 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.563 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.563 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-04-25T19:09:30.563 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.564 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.564 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.564 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-04-25T19:09:30.564 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 15, 2024-04-25T19:09:30.564 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.013537245, 2024-04-25T19:09:30.564 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000902483 2024-04-25T19:09:30.564 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.565 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-04-25T19:09:30.565 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.565 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.565 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.565 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.565 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-04-25T19:09:30.565 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T19:09:30.566 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000168755, 2024-04-25T19:09:30.566 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000168755 2024-04-25T19:09:30.566 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.566 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-04-25T19:09:30.566 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.566 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.567 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.567 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.567 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-04-25T19:09:30.567 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.567 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.567 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.568 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-04-25T19:09:30.568 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.568 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.568 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.568 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.569 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-04-25T19:09:30.569 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.569 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.569 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.569 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.570 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-04-25T19:09:30.570 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.570 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.571 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.571 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.571 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-04-25T19:09:30.571 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.571 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.571 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.572 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-04-25T19:09:30.572 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.572 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.572 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.572 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1, 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000321506, 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000321506 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 6, 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.008872236, 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.001478706 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.573 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-04-25T19:09:30.574 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-04-25T19:09:30.575 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-04-25T19:09:30.576 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-04-25T19:09:30.577 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 16, 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.065212375, 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.004075773 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-04-25T19:09:30.578 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 1, 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 4, 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 3, 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 1, 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 28, 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 28, 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 577, 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:09:30.579 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T19:09:30.580 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T19:09:30.581 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 287079, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 94, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T19:09:30.582 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 8224, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 171, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 105653, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 408, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.583 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 156, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 197820, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 30, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 156, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 156, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.817987412, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.005243509 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 156, 2024-04-25T19:09:30.584 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 196 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 156, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 6, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 150, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 0, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 10, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 6, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 59, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 23, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-04-25T19:09:30.585 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-04-25T19:09:30.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2024-04-25T19:09:30.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-04-25T19:09:30.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 28, 2024-04-25T19:09:30.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-04-25T19:09:30.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-04-25T19:09:30.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-04-25T19:09:30.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 0, 2024-04-25T19:09:30.586 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 20, 2024-04-25T19:09:30.587 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:09:30.587 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-04-25T19:09:30.587 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:09:30.587 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-04-25T19:09:30.587 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-04-25T19:09:30.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-04-25T19:09:30.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:09:30.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-04-25T19:09:30.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:09:30.588 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-04-25T19:09:30.588 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-04-25T19:09:30.588 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-04-25T19:09:30.588 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-04-25T19:09:30.589 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-04-25T19:09:30.589 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-04-25T19:09:30.589 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-04-25T19:09:30.589 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-04-25T19:09:30.589 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-04-25T19:09:30.589 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-04-25T19:09:30.589 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-04-25T19:09:30.589 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-04-25T19:09:30.589 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-04-25T19:09:30.590 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-04-25T19:09:30.590 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-04-25T19:09:30.590 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-04-25T19:09:30.590 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-04-25T19:09:30.590 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 104, 2024-04-25T19:09:30.590 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-04-25T19:09:30.590 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 12, 2024-04-25T19:09:30.590 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 12, 2024-04-25T19:09:30.590 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 12, 2024-04-25T19:09:30.591 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-04-25T19:09:30.591 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-04-25T19:09:30.591 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 40, 2024-04-25T19:09:30.591 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 0, 2024-04-25T19:09:30.591 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 10 2024-04-25T19:09:30.591 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.591 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-04-25T19:09:30.591 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 1, 2024-04-25T19:09:30.592 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:09:30.592 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-04-25T19:09:30.592 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-04-25T19:09:30.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.592 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-04-25T19:09:30.592 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 29 2024-04-25T19:09:30.592 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.592 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-04-25T19:09:30.593 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-04-25T19:09:30.593 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 8, 2024-04-25T19:09:30.593 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-04-25T19:09:30.593 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 8, 2024-04-25T19:09:30.593 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 16, 2024-04-25T19:09:30.593 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 16, 2024-04-25T19:09:30.593 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-04-25T19:09:30.593 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.593 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:09:30.594 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T19:09:30.594 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T19:09:30.594 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T19:09:30.594 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 5115, 2024-04-25T19:09:30.594 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 5337370, 2024-04-25T19:09:30.594 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:30.594 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 5115, 2024-04-25T19:09:30.594 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T19:09:30.594 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T19:09:30.594 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 5115, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 5337370, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:30.595 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 156, 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 200204, 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 156, 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 200204, 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.596 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 156, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 156, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 156, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 156, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:30.597 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 16, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 2384, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 16, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 6, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 2384, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.598 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x55c86bea0460": { 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 95, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 180097, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 95, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 53, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 180097, 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T19:09:30.599 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:30.600 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:30.600 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:30.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:30.600 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:30.600 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:09:30.613 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:30 smithi062 ceph-mon[32844]: pgmap v317: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T19:09:31.644 INFO:tasks.check_counter:Daemon mds.a mds.exported=4 2024-04-25T19:09:31.644 INFO:tasks.check_counter:Daemon mds.a mds.imported=4 2024-04-25T19:09:31.644 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=6 2024-04-25T19:09:31.644 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=531 2024-04-25T19:09:31.644 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=6 2024-04-25T19:09:31.644 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:09:31.644 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:09:31.644 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-04-25T19:09:31.644 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T19:09:31.645 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T19:09:32.097 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:09:32.408 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:32 smithi045 ceph-mon[27776]: pgmap v318: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T19:09:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:32 smithi079 ceph-mon[33002]: pgmap v318: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T19:09:32.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:32 smithi062 ceph-mon[32844]: pgmap v318: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-25T19:09:34.399 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:34 smithi045 ceph-mon[27776]: pgmap v319: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T19:09:34.401 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:09:34.401 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:09:34.401 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.005007910, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001050494, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:34.402 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 6, 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 185, 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3360, 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 188864, 2024-04-25T19:09:34.403 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.038915275, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.027585337, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.005606657, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 183, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.029871848, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000163234 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:34.404 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 188864 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 251, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 239, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1736336, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 147744, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.114467981, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.027562204, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.113675239, 2024-04-25T19:09:34.405 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.011338930, 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 238, 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.024845700, 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000104393 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1736336, 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 147744 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.406 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:09:34.407 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T19:09:34.408 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 149745, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 31, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T19:09:34.409 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T19:09:34.410 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 253, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1700906, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 253, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T19:09:34.411 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T19:09:34.412 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 253, 2024-04-25T19:09:34.412 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1700906, 2024-04-25T19:09:34.412 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T19:09:34.412 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:34.412 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:34.412 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:34.412 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:34.412 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:34.412 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:09:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:34 smithi079 ceph-mon[33002]: pgmap v319: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T19:09:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:34 smithi062 ceph-mon[32844]: pgmap v319: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T19:09:35.364 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T19:09:35.364 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T19:09:35.364 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T19:09:35.364 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T19:09:35.364 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T19:09:35.364 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T19:09:35.365 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T19:09:35.365 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-25T19:09:35.365 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T19:09:35.365 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T19:09:35.828 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:09:36.397 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:36 smithi045 ceph-mon[27776]: pgmap v320: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T19:09:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:36 smithi079 ceph-mon[33002]: pgmap v320: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T19:09:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:36 smithi062 ceph-mon[32844]: pgmap v320: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T19:09:38.119 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:09:38.119 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:09:38.119 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 247, 2024-04-25T19:09:38.119 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 235, 2024-04-25T19:09:38.119 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1735248, 2024-04-25T19:09:38.119 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 145184, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.108086190, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.037258872, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.075845595, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.011028220, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 234, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.025119946, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000107350 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:38.120 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1735248, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 145184 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 6, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 182, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3360, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 188336, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.039000513, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.027634557, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.005879074, 2024-04-25T19:09:38.121 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 180, 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.026388843, 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000146604 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 188336 2024-04-25T19:09:38.122 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001349139, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001159371, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:38.123 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.124 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:09:38.125 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:09:38.126 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148714, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 30, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T19:09:38.127 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T19:09:38.128 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 249, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1700386, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 249, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 249, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1700386, 2024-04-25T19:09:38.129 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T19:09:38.130 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:38.130 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:38.130 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:38.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:38.130 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:38.130 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:09:38.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:38 smithi079 ceph-mon[33002]: pgmap v321: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:38.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:38 smithi045 ceph-mon[27776]: pgmap v321: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:38.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:38 smithi062 ceph-mon[32844]: pgmap v321: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:38.951 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T19:09:38.951 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T19:09:38.951 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-25T19:09:38.951 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-25T19:09:38.951 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T19:09:38.951 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T19:09:38.951 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T19:09:38.951 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-25T19:09:38.951 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T19:09:38.951 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T19:09:39.425 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:09:40.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:40 smithi045 ceph-mon[27776]: pgmap v322: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:40.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:40 smithi079 ceph-mon[33002]: pgmap v322: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:40.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:40 smithi062 ceph-mon[32844]: pgmap v322: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:41.681 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.004329822, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000999518, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:41.682 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 6, 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 188, 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 3360, 2024-04-25T19:09:41.683 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 189392, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.037405586, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.027576408, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.004234006, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 186, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.033635204, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000180834 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:41.684 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 189392 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 255, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 243, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1738048, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 150000, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.107964469, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.038592971, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.077701795, 2024-04-25T19:09:41.685 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.011806109, 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 242, 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.026617277, 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000109988 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1738048, 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 150000 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-25T19:09:41.686 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:09:41.687 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:09:41.688 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 149745, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 31, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:09:41.689 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-25T19:09:41.690 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 257, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1702063, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 257, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 257, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1702063, 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-25T19:09:41.691 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-25T19:09:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-25T19:09:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-25T19:09:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:41.692 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-25T19:09:41.692 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-25T19:09:42.398 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:42 smithi045 ceph-mon[27776]: pgmap v323: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:42.416 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:42 smithi062 ceph-mon[32844]: pgmap v323: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:42 smithi079 ceph-mon[33002]: pgmap v323: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:42.461 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T19:09:42.461 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T19:09:42.461 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T19:09:42.461 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T19:09:42.461 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T19:09:42.461 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T19:09:42.461 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T19:09:42.461 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-25T19:09:42.462 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T19:09:42.462 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T19:09:42.719 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T19:09:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:44 smithi079 ceph-mon[33002]: pgmap v324: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:44 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:09:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:44 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:44.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:44 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:44.440 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:44 smithi062 ceph-mon[32844]: pgmap v324: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:44.440 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:44 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:09:44.440 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:44 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:44.440 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:44 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:44.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:44 smithi045 ceph-mon[27776]: pgmap v324: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:44.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:44 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T19:09:44.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:44 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:44.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:44 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:44.687 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T19:09:44.687 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:09:44.687 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 33005, 2024-04-25T19:09:44.687 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 34191, 2024-04-25T19:09:44.687 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 8620233, 2024-04-25T19:09:44.687 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 92320235, 2024-04-25T19:09:44.687 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 16, 2024-04-25T19:09:44.687 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-25T19:09:44.687 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 2.393194194, 2024-04-25T19:09:44.687 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.857413448, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.633672336, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.752223418, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 34189, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.703488772, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000079074 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 7678, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.001377627, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000179 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 350560, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 89472 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.688 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 24503, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 24446, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 8888399, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 68385765, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 97, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 7, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 1.687864130, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.506277125, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.493681880, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.581385468, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 24445, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 4.178072853, 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000170917 2024-04-25T19:09:44.689 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 378, 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000068562, 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000181 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1931024, 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 502480 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 63344, 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 116720, 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 19793731, 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 88155573, 2024-04-25T19:09:44.690 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 12, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 6, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 4.613978257, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 1.856510398, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 2.416427257, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.566316485, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 116718, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 4.230449298, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000036245 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 106358, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.021520830, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000202 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.691 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 44832, 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 173888 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 89802, 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 31.535297390, 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000351164 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-25T19:09:44.692 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8737, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2.121837634, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000242856 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 22365, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 22301, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 22301, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 145.826169247, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.006538996 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 64, 2024-04-25T19:09:44.693 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 1156, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 22, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 9507, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 5193, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 4304, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 10, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 67, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 46, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 2, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 3, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 5, 2024-04-25T19:09:44.694 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 25518, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 22070, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 58, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 1, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 4, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 7, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 37, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 7, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 186, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 4, 2024-04-25T19:09:44.695 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 234, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 6, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 14, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 9545, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 18901, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 7861, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 16071, 2024-04-25T19:09:44.696 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 13275, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 351, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 14442 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 429, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 3, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 2, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 2, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 557, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 60, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 5094, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 5031, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 3, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-25T19:09:44.697 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 2, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 1359, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 7, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 22, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-25T19:09:44.698 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 50475, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 50445, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 50445, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 30, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 5066, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 100, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 99, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 99, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 1, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 1, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 8, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 175868853, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 175975970, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 0, 2024-04-25T19:09:44.699 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 26045, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 548.554574119, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.021061799 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 0 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 4739, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 9806, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 5067, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 232, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 263, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 31, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 9507, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 15000, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 5493, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 3, 2024-04-25T19:09:44.700 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 7246, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 7243, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 114716, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 231800 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 40440, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 158, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 22365, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 43, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 352, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 5054, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 75.892935924, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.015016409 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-25T19:09:44.701 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 167, 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.297880359, 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001783714 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3, 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.003352392, 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001117464 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2318, 2024-04-25T19:09:44.702 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.039275826, 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000448350 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 1, 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000187406, 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000187406 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.703 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.704 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-25T19:09:44.704 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.704 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.704 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.704 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-25T19:09:44.704 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.704 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.704 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.704 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 10, 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.002033352, 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000203335 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 72, 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 12.468067137, 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.173167599 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.705 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 10, 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.096588312, 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.009658831 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 404, 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.118278812, 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000292769 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 458, 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.359127707, 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002967527 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-25T19:09:44.706 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 79, 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.602351043, 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.007624696 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 66, 2024-04-25T19:09:44.707 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 3.302532861, 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.050038376 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4, 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.026971075, 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.006742768 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8665, 2024-04-25T19:09:44.708 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 41.364708873, 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.004773769 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8, 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.003847488, 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000480936 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.709 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 8, 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.022752422, 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.002844052 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 5, 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.003356525, 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000671305 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-25T19:09:44.710 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4969, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 9.221921733, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.001855890 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 2, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 6, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 4, 2024-04-25T19:09:44.711 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 2, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 4286, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 2143, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 601, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 360, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 360, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:09:44.712 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:09:44.713 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 972721, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 4878, 2024-04-25T19:09:44.714 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 88, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 1, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 8224, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 171, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 18153397, 2024-04-25T19:09:44.715 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 187531, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 75644, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 186122954, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 31, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 75644, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 75644, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 960.199389743, 2024-04-25T19:09:44.716 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.012693662 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 75644, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 89023 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 75644, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 66, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 75578, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 902, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 5486, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 66, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 26111, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 186, 2024-04-25T19:09:44.717 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 6, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 5694, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 10972, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-25T19:09:44.718 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 36715, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-25T19:09:44.719 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 104, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 12, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 12, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 12, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 2582, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 0, 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 303 2024-04-25T19:09:44.720 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 1, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 1411 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 60, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 10, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 5086, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 5031, 2024-04-25T19:09:44.721 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 120848, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 26355341, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 120848, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 120848, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 26355341, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.722 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 75644, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 186845733, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 75644, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 186845733, 2024-04-25T19:09:44.723 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 75644, 2024-04-25T19:09:44.724 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 75644, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 75644, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 75644, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 5031, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 722779, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 5031, 2024-04-25T19:09:44.725 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 66, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 722779, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x563bc1940460": { 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 50475, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 171781666, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:44.726 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 50475, 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 26045, 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 171781666, 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:44.727 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T19:09:45.466 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:45 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:45.466 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:45 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:45.494 INFO:tasks.check_counter:Daemon mds.b mds.exported=7 2024-04-25T19:09:45.494 INFO:tasks.check_counter:Daemon mds.b mds.imported=7 2024-04-25T19:09:45.494 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=429 2024-04-25T19:09:45.494 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=3 2024-04-25T19:09:45.494 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-25T19:09:45.494 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T19:09:45.494 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2024-04-25T19:09:45.494 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-04-25T19:09:45.494 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T19:09:45.495 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T19:09:45.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:45 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:45.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:45 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:45.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:45 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:45.713 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:45 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:46.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:46 smithi062 ceph-mon[32844]: pgmap v325: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:46.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:46 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:46.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:46 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:46.399 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T19:09:46.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:46 smithi079 ceph-mon[33002]: pgmap v325: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:46.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:46 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:46.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:46 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:46.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:46 smithi045 ceph-mon[27776]: pgmap v325: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:46.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:46 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:46.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:46 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:47.574 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: Unable to set osd_memory_target on smithi079 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:47.575 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:47 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-04-25T19:09:47.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: Unable to set osd_memory_target on smithi079 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T19:09:47.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:47.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:47 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:47.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi079 to 176.2M 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: Unable to set osd_memory_target on smithi079 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:47.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:47 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:48.256 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T19:09:48.256 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 0, 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2, 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 0, 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 0, 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.001135614, 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000128931, 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:48.257 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.258 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 6, 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 187, 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3360, 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 189216, 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.011932507, 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.007872435, 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001557344, 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:48.259 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 185, 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.027400811, 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000148112 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 189216 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:09:48.260 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 254, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 242, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1737776, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 149360, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.046941815, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.013720121, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.036663141, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.006101151, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 241, 2024-04-25T19:09:48.261 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.012213133, 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000050676 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1737776, 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 149360 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-25T19:09:48.262 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:09:48.263 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:09:48.264 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:09:48.265 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148670, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:09:48.266 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-25T19:09:48.267 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 256, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1701933, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 256, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 256, 2024-04-25T19:09:48.268 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1701933, 2024-04-25T19:09:48.269 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:48.269 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:48.269 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:48.269 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:48.269 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:48.269 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:48.269 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T19:09:48.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:48 smithi079 ceph-mon[33002]: pgmap v326: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:48.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:48 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:48 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:48 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:48 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:48 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:48 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:48 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:48 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:48 smithi045 ceph-mon[27776]: pgmap v326: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:48.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:48 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:48 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:48 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:48 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:48 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:48 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:48 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:48 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:48 smithi062 ceph-mon[32844]: pgmap v326: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:48.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:48 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:48 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:48 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:48 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:48 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:48 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:48 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.865 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:48 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:48.987 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-25T19:09:48.988 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-25T19:09:48.988 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-04-25T19:09:48.988 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-04-25T19:09:48.988 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-25T19:09:48.988 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-25T19:09:48.988 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-25T19:09:48.988 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.e 2024-04-25T19:09:48.988 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T19:09:48.988 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T19:09:49.241 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T19:09:49.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:49 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-25T19:09:49.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:49 smithi062 ceph-mon[32844]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T19:09:49.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:49 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:49.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:49 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:49.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:49 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-25T19:09:49.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:49 smithi079 ceph-mon[33002]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T19:09:49.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:49 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:49.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:49 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:49.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:49 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi045 to 177.5M 2024-04-25T19:09:49.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:49 smithi045 ceph-mon[27776]: Unable to set osd_memory_target on smithi045 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T19:09:49.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:49 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:49.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:49 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:50.420 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: pgmap v327: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:50.420 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.420 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.420 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.420 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.420 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.420 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.420 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.420 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.421 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:09:50.421 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:09:50.421 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:50 smithi062 ceph-mon[32844]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: pgmap v327: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:50.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:09:50.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:09:50.685 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:50 smithi079 ceph-mon[33002]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: pgmap v327: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T19:09:50.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T19:09:50.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T19:09:50.715 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:50 smithi045 ceph-mon[27776]: from='mgr.25049 ' entity='mgr.x' 2024-04-25T19:09:51.027 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T19:09:51.028 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:09:51.028 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2477, 2024-04-25T19:09:51.028 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 2479, 2024-04-25T19:09:51.028 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 879526, 2024-04-25T19:09:51.028 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 787693, 2024-04-25T19:09:51.028 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-04-25T19:09:51.028 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2024-04-25T19:09:51.028 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.176593313, 2024-04-25T19:09:51.028 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.056173051, 2024-04-25T19:09:51.028 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.043941540, 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.061898325, 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2477, 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.069005028, 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000027858 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:09:51.029 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 8, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 188, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3916, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 478170, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 6, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 4, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.011015426, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.006609868, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.002449731, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000059358, 2024-04-25T19:09:51.030 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 186, 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.037276767, 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000200412 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 477472 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 283, 2024-04-25T19:09:51.031 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 259, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1809964, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 151982, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 3, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 2, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.048837751, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.013955839, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.039148364, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.005897604, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 258, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.013735103, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000053236 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:51.032 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1809696, 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 151664 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 1, 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-MDSRank": { 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T19:09:51.033 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 4195, 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.094645979, 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000022561 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: "finisher-PurgeQueue": { 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: "queue_len": 0, 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: "complete_latency": { 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: "mds": { 2024-04-25T19:09:51.034 INFO:teuthology.orchestra.run.smithi062.stdout: "request": 0, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "reply": 0, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "reply_latency": { 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_reply": 0, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "forward": 0, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_complete": 0, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_fetch_keys": 0, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_commit": 0, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_split": 0, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_merge": 0, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes": 17, 2024-04-25T19:09:51.035 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_top": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_bottom": 17, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pin_tail": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_pinned": 17, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_expired": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "inodes_with_caps": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "caps": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "subtrees": 1, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_hit": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_forward": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_discover": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_remote_ino": 0, 2024-04-25T19:09:51.036 INFO:teuthology.orchestra.run.smithi062.stdout: "traverse_lock": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "load_cent": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "q": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "exported": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "exported_inodes": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "imported": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "imported_inodes": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_dir_fetch": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "openino_peer_discover": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rfiles": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rbytes": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "root_rsnaps": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_set_tag": 0, 2024-04-25T19:09:51.037 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "scrub_file_inodes": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_cap_release": 0, 2024-04-25T19:09:51.038 INFO:teuthology.orchestra.run.smithi062.stdout: "process_request_cap_release": 0 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_cache": { 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_update_receipt": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_try_discover": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_send_discover": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "dir_handle_discover": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_delayed": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_created": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_enqueued": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_reintegrated": 0, 2024-04-25T19:09:51.039 INFO:teuthology.orchestra.run.smithi062.stdout: "strays_migrated": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_processing": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_started": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "recovery_completed": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_path": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_quiesce_inode": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_exportdir": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_flush": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_fragstats": 0, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "ireq_inodestats": 0 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.040 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_log": { 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "evlrg": 0, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "evadd": 0, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "evex": 0, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "evtrm": 0, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "ev": 95, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "evexg": 0, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "evexd": 0, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "segadd": 0, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "segex": 0, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "segtrm": 0, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "seg": 2, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "segmjr": 2, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "segexg": 0, 2024-04-25T19:09:51.041 INFO:teuthology.orchestra.run.smithi062.stdout: "segexd": 0, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "expos": 4194304, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "wrpos": 4374401, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "rdpos": 4372786, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "jlat": { 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "replayed": 95 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_mem": { 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "ino": 4, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "ino+": 64, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "ino-": 60, 2024-04-25T19:09:51.042 INFO:teuthology.orchestra.run.smithi062.stdout: "dir": 2, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "dir+": 27, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "dir-": 25, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "dn": 17, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "dn+": 81, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "dn-": 64, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "cap": 0, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "cap+": 0, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "cap-": 0, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "rss": 53316, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "heap": 215416 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_server": { 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_client_request": 0, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "dispatch_server_request": 0, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_request": 0, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_client_session": 0, 2024-04-25T19:09:51.043 INFO:teuthology.orchestra.run.smithi062.stdout: "handle_peer_request": 0, 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "req_create_latency": { 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getattr_latency": { 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getfilelock_latency": { 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.044 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "req_link_latency": { 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookup_latency": { 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookuphash_latency": { 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.045 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupino_latency": { 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupname_latency": { 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupparent_latency": { 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.046 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lookupsnap_latency": { 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "req_lssnap_latency": { 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mkdir_latency": { 2024-04-25T19:09:51.047 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mknod_latency": { 2024-04-25T19:09:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.048 INFO:teuthology.orchestra.run.smithi062.stdout: "req_mksnap_latency": { 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "req_open_latency": { 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "req_readdir_latency": { 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.049 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rename_latency": { 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "req_renamesnap_latency": { 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "req_snapdiff_latency": { 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.050 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmdir_latency": { 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmsnap_latency": { 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "req_rmxattr_latency": { 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.051 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setattr_latency": { 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setdirlayout_latency": { 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setfilelock_latency": { 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.052 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setlayout_latency": { 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "req_setxattr_latency": { 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "req_symlink_latency": { 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.053 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "req_unlink_latency": { 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "req_getvxattr_latency": { 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.054 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_sessions": { 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "session_count": 1, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "session_add": 4, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "session_remove": 3, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_open": 3, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "sessions_stale": 0, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "total_load": 0, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "average_load": 0, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "avg_session_uptime": 5291, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "mdthresh_evicted": 0 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-25T19:09:51.055 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-25T19:09:51.056 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:09:51.057 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 182099, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 49, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 8224, 2024-04-25T19:09:51.058 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 171, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 25704, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 101, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "objecter": { 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "op_active": 0, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "op_laggy": 0, 2024-04-25T19:09:51.059 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send": 2480, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "op_send_bytes": 21, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "op_resend": 3, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "op_reply": 2480, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "op_latency": { 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2480, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 1.227981445, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000495153 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "op_inflight": 0, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "oplen_avg": { 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 2480, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 2481 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "op": 2480, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "op_r": 2480, 2024-04-25T19:09:51.060 INFO:teuthology.orchestra.run.smithi062.stdout: "op_w": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "op_rmw": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "op_pg": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_stat": 1598, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_create": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_read": 880, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_write": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writefull": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_writesame": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_append": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_zero": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_truncate": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_delete": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_mapext": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_sparse_read": 0, 2024-04-25T19:09:51.061 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_clonerange": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_getxattr": 1, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_setxattr": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_rmxattr": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_call": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_watch": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_notify": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "osdop_other": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_active": 0, 2024-04-25T19:09:51.062 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_send": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_resend": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "linger_ping": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_active": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_send": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "poolop_resend": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_active": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_send": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "poolstat_resend": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_active": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_send": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "statfs_resend": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "command_active": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "command_send": 0, 2024-04-25T19:09:51.063 INFO:teuthology.orchestra.run.smithi062.stdout: "command_resend": 0, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "map_epoch": 104, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "map_full": 0, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "map_inc": 12, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_sessions": 4, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_open": 4, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_session_close": 0, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_laggy": 0, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_wr": 0, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_rd": 2, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_del": 0 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "oft": { 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_objs": 0, 2024-04-25T19:09:51.064 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_updates": 0, 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "omap_total_removes": 0 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "osd-slow-ops": { 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "slow_ops_count": 0 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "purge_queue": { 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops": 0, 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing": 0, 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executing_high_water": 0, 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed_ops": 0, 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_executed": 0, 2024-04-25T19:09:51.065 INFO:teuthology.orchestra.run.smithi062.stdout: "pq_item_in_journal": 0 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 2764, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 2427164, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 2764, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2764, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2427164, 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:51.066 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_bytes": { 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:51.067 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 2480, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 180102, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2480, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 180102, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-objecter_ops": { 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 1024, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T19:09:51.068 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 2480, 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 2480, 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 2480, 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 2480, 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:09:51.069 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.070 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-write_buf_throttle-0x562fad11a6e0": { 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 3758096384, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 0, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 0, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 0, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 0, 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:51.071 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:51.072 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:51.072 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T19:09:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:51 smithi079 ceph-mon[33002]: Adjusting osd_memory_target on smithi062 to 176.2M 2024-04-25T19:09:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:51 smithi079 ceph-mon[33002]: Unable to set osd_memory_target on smithi062 to 184794624: error parsing value: Value '184794624' is below minimum 939524096 2024-04-25T19:09:51.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:51 smithi079 ceph-mon[33002]: pgmap v328: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:51.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:51 smithi045 ceph-mon[27776]: Adjusting osd_memory_target on smithi062 to 176.2M 2024-04-25T19:09:51.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:51 smithi045 ceph-mon[27776]: Unable to set osd_memory_target on smithi062 to 184794624: error parsing value: Value '184794624' is below minimum 939524096 2024-04-25T19:09:51.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:51 smithi045 ceph-mon[27776]: pgmap v328: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:51.796 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:51 smithi062 ceph-mon[32844]: Adjusting osd_memory_target on smithi062 to 176.2M 2024-04-25T19:09:51.796 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:51 smithi062 ceph-mon[32844]: Unable to set osd_memory_target on smithi062 to 184794624: error parsing value: Value '184794624' is below minimum 939524096 2024-04-25T19:09:51.796 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:51 smithi062 ceph-mon[32844]: pgmap v328: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:51.823 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T19:09:51.823 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T19:09:51.823 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T19:09:51.823 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-25T19:09:51.823 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-04-25T19:09:51.823 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:09:51.823 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:09:51.823 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-25T19:09:51.823 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T19:09:51.824 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T19:09:52.069 INFO:teuthology.orchestra.run.smithi062.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T19:09:53.913 INFO:teuthology.orchestra.run.smithi062.stdout:{ 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 259, 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 247, 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 1739136, 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 152560, 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 4, 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 0, 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.051644241, 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.014911028, 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.040236587, 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.006109102, 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:53.914 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 246, 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.012816943, 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000052101 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 1739136, 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 152560 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 2, 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 189, 2024-04-25T19:09:53.915 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 320, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 189472, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 5, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 3, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.011016528, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.007510758, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.001232350, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 186, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.037701488, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000202696 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:53.916 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 189472 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_messages": 4, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages": 4, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_bytes": 3040, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_bytes": 448, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_created_connections": 2, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_active_connections": 1, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_total_time": 0.000744132, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_send_time": 0.000037580, 2024-04-25T19:09:53.917 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_recv_time": 0.000668443, 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 3, 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000786059, 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000262019 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.918 INFO:teuthology.orchestra.run.smithi062.stdout: "cct": { 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "total_workers": 0, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "unhealthy_workers": 0 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "mempool": { 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "bloom_filter_items": 0, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:09:53.919 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_extent_items": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_blob_items": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:09:53.920 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_txc_items": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluestore_writing_items": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_bytes": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_items": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_bytes": 148714, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_anon_items": 30, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "buffer_meta_items": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_bytes": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_items": 0, 2024-04-25T19:09:53.921 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_mapbl_items": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "osd_pglog_items": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_bytes": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_items": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_bytes": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "pgmap_items": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_bytes": 928, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "mds_co_items": 2, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_bytes": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_1_items": 0, 2024-04-25T19:09:53.922 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_bytes": 0, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "unittest_2_items": 0 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: }, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "val": 0, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "max": 104857600, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "get_started": 0, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "get": 261, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "get_sum": 1702583, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "get_or_fail_success": 261, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "take": 0, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "take_sum": 0, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "put": 261, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "put_sum": 1702583, 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "wait": { 2024-04-25T19:09:53.923 INFO:teuthology.orchestra.run.smithi062.stdout: "avgcount": 0, 2024-04-25T19:09:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: "sum": 0.000000000, 2024-04-25T19:09:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: "avgtime": 0.000000000 2024-04-25T19:09:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:53.924 INFO:teuthology.orchestra.run.smithi062.stdout: } 2024-04-25T19:09:53.924 INFO:teuthology.orchestra.run.smithi062.stdout:} 2024-04-25T19:09:54.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:54 smithi062 ceph-mon[32844]: pgmap v329: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:54.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:54 smithi079 ceph-mon[33002]: pgmap v329: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:54.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:54 smithi045 ceph-mon[27776]: pgmap v329: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:54.584 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T19:09:54.584 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T19:09:54.584 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-25T19:09:54.584 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-25T19:09:54.584 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T19:09:54.584 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T19:09:54.584 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T19:09:54.584 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-25T19:09:54.584 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T19:09:54.584 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T19:09:54.833 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T19:09:56.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:56 smithi079 ceph-mon[33002]: pgmap v330: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:56.364 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:56 smithi062 ceph-mon[32844]: pgmap v330: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:56.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:56 smithi045 ceph-mon[27776]: pgmap v330: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:09:56.833 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-25T19:09:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:09:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 587, 2024-04-25T19:09:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 589, 2024-04-25T19:09:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 33694607, 2024-04-25T19:09:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 186676, 2024-04-25T19:09:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 8, 2024-04-25T19:09:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-04-25T19:09:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.068472640, 2024-04-25T19:09:56.833 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.014359512, 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.032515852, 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.016512814, 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 587, 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.021764417, 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000037077 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.834 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 398, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 460, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 46062053, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 320040, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 6, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 5, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.053188115, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.012435582, 2024-04-25T19:09:56.835 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.029388246, 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.008221603, 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 458, 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.018379740, 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000040130 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 193728 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.836 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 282, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 275, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 12831782, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 108929, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 5, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 4, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.032835044, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.010070748, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.015034944, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.006471732, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 275, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.012972880, 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000047174 2024-04-25T19:09:56.837 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 204336, 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 52960 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-04-25T19:09:56.838 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1938, 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.105216888, 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000054291 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-04-25T19:09:56.839 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 0, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 0, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 0, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 0, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 13866, 2024-04-25T19:09:56.840 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 5394, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 8472, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 66, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 2, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-04-25T19:09:56.841 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 4, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 234, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 6, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:09:56.842 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 0, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 0 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-04-25T19:09:56.843 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-04-25T19:09:56.844 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_path": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_inode": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "evlrg": 0, 2024-04-25T19:09:56.845 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 0, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 23870, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 0, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 2, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "segmjr": 17, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 175868853, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 175975970, 2024-04-25T19:09:56.846 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 175975858, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 23870 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 4667, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 9161, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 4494, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 222, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 222, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 0, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 13866, 2024-04-25T19:09:56.847 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 13866, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 0, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 0, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 0, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 0, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 75048, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 215392 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 0, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 0, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 0, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 0, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.848 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.849 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.850 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.851 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-04-25T19:09:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-04-25T19:09:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.852 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.853 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-04-25T19:09:56.854 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-04-25T19:09:56.855 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.856 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.857 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.858 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 2, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 2, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 0, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 0, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-04-25T19:09:56.859 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 2195, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:09:56.860 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:09:56.861 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 405775, 2024-04-25T19:09:56.862 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 4830, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 8224, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 171, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-04-25T19:09:56.863 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 20783722, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 170347, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 1160, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 26, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 4, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 1160, 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-04-25T19:09:56.864 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1160, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.932979748, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000804292 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1160, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1161 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 1160, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 1160, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 0, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 624, 2024-04-25T19:09:56.865 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 0, 2024-04-25T19:09:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 533, 2024-04-25T19:09:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 0, 2024-04-25T19:09:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 0, 2024-04-25T19:09:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-04-25T19:09:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-04-25T19:09:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-04-25T19:09:56.866 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-04-25T19:09:56.869 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 2, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-04-25T19:09:56.870 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-04-25T19:09:56.871 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 104, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 8, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 10, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 10, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 0, 2024-04-25T19:09:56.872 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 0, 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 2, 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 0 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 0, 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-04-25T19:09:56.873 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 0, 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed_ops": 0, 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 0, 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:09:56.874 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 1263, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 92466544, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 1263, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 1263, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 92466544, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-04-25T19:09:56.875 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 1160, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 92059586, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 1160, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 92059586, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.876 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 1160, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 1160, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 1160, 2024-04-25T19:09:56.877 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1160, 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-04-25T19:09:56.878 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x5587891ba6e0": { 2024-04-25T19:09:56.879 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:56.880 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:56.881 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-25T19:09:57.616 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T19:09:57.617 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T19:09:57.617 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T19:09:57.617 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-25T19:09:57.617 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-25T19:09:57.617 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:09:57.617 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:09:57.617 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-25T19:09:57.617 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T19:09:57.617 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T19:09:57.863 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T19:09:58.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:09:58 smithi079 ceph-mon[33002]: pgmap v331: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:58.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:09:58 smithi045 ceph-mon[27776]: pgmap v331: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:58.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:09:58 smithi062 ceph-mon[32844]: pgmap v331: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:09:59.577 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-25T19:09:59.577 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:09:59.577 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 879, 2024-04-25T19:09:59.577 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 881, 2024-04-25T19:09:59.577 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 1059928, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 279586, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 6, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 2, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.080191333, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.022252241, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.023382299, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.026744289, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 879, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.028701739, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000032652 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.578 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 769, 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 952, 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 191080, 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 723871, 2024-04-25T19:09:59.579 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 7, 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 5, 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.083454237, 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.024973961, 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.019995048, 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.030479792, 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 950, 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.051145003, 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000053836 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.580 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 481232 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 1137, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 1113, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 2096720, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 424842, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 3, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 2, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.141779433, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.041641075, 2024-04-25T19:09:59.581 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.065203740, 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.037015699, 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1112, 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.053865743, 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000048440 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 1810784, 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 154224 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.582 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 4162, 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.117404791, 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000028208 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.583 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 0, 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 0, 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 0, 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-04-25T19:09:59.584 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 109, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 109, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 109, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 1, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 0, 2024-04-25T19:09:59.585 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 0, 2024-04-25T19:09:59.586 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:09:59.587 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 0 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-04-25T19:09:59.588 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_path": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_inode": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-04-25T19:09:59.589 INFO:teuthology.orchestra.run.smithi079.stdout: "evlrg": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 438, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 2, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "segmjr": 2, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-04-25T19:09:59.590 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 4194304, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 5023695, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 5021607, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 438 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 34, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 181, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 147, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 3, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 18, 2024-04-25T19:09:59.591 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 15, 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 109, 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 259, 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 150, 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 0, 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 0, 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 0, 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 51664, 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 215416 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 0, 2024-04-25T19:09:59.592 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 0, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 0, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 0, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 0, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.593 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.594 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.595 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.596 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.597 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.598 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.599 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.600 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.601 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-04-25T19:09:59.602 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:59.603 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 2, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 1, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 1, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 0, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 0, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 2201, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-04-25T19:09:59.604 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:09:59.605 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-04-25T19:09:59.606 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 191382, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 85, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 8224, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 171, 2024-04-25T19:09:59.607 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 165148, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 1345, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 2493, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 21, 2024-04-25T19:09:59.608 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 3, 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 2493, 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2493, 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 2.186952633, 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000877237 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2493, 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 2494 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 2493, 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 2493, 2024-04-25T19:09:59.609 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 1612, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 879, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-04-25T19:09:59.610 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 1, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-04-25T19:09:59.611 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 104, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 12, 2024-04-25T19:09:59.612 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 5, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 5, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 0, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 0, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 2, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 0 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 0, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 0 2024-04-25T19:09:59.613 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 0, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed_ops": 0, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 0, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 2781, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 3079748, 2024-04-25T19:09:59.614 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 2781, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 2781, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 3079748, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-04-25T19:09:59.615 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 2493, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 829396, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 2493, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 829396, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:59.616 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 2493, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 2493, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 2493, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 2493, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.617 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-04-25T19:09:59.618 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x563e45f9e6e0": { 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:09:59.619 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 0, 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 0, 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:09:59.620 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-25T19:10:00.304 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:00 smithi079 ceph-mon[33002]: pgmap v332: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:10:00.304 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:00 smithi079 ceph-mon[33002]: overall HEALTH_OK 2024-04-25T19:10:00.331 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T19:10:00.331 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T19:10:00.331 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T19:10:00.331 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T19:10:00.331 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T19:10:00.331 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:10:00.332 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:10:00.332 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-25T19:10:00.332 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T19:10:00.332 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T19:10:00.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:00 smithi045 ceph-mon[27776]: pgmap v332: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:10:00.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:00 smithi045 ceph-mon[27776]: overall HEALTH_OK 2024-04-25T19:10:00.570 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T19:10:00.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:00 smithi062 ceph-mon[32844]: pgmap v332: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:10:00.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:00 smithi062 ceph-mon[32844]: overall HEALTH_OK 2024-04-25T19:10:02.256 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:02 smithi079 ceph-mon[33002]: pgmap v333: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:10:02.264 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-25T19:10:02.264 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 4586, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 2757, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 3869881, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 1510456, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 9, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 3, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.211544629, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.082698099, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.159703384, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.008597823, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2759, 2024-04-25T19:10:02.265 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.095018915, 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000034439 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 6871, 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000987386, 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000143 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:10:02.266 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 105, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 303, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 51630, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 562768, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 10, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 6, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.019298897, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.009112084, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.006678969, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.001944434, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 301, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.051981900, 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000172697 2024-04-25T19:10:02.267 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 92, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000012975, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000141 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 482672 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 940, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 1231, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 2033940, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 725200, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 17, 2024-04-25T19:10:02.268 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 10, 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.100481934, 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.033707394, 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.082383499, 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.006756433, 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1226, 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.034362223, 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000028027 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1322, 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000224063, 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000169 2024-04-25T19:10:02.269 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 1824112, 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 159216 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 1, 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-MDSRank": { 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 252, 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.122945941, 2024-04-25T19:10:02.270 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000487880 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "finisher-PurgeQueue": { 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "queue_len": 0, 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "complete_latency": { 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 122, 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.008168979, 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000066958 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": { 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "request": 430, 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "reply": 331, 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "reply_latency": { 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 331, 2024-04-25T19:10:02.271 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.332117853, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001003377 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_reply": 0, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "forward": 99, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_complete": 0, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_fetch_keys": 0, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_commit": 11, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_split": 0, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_merge": 0, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes": 59, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_top": 1, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_bottom": 1, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pin_tail": 57, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_pinned": 57, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_expired": 267, 2024-04-25T19:10:02.272 INFO:teuthology.orchestra.run.smithi079.stdout: "inodes_with_caps": 0, 2024-04-25T19:10:02.273 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": 0, 2024-04-25T19:10:02.273 INFO:teuthology.orchestra.run.smithi079.stdout: "subtrees": 2, 2024-04-25T19:10:02.273 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse": 1112, 2024-04-25T19:10:02.273 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_hit": 614, 2024-04-25T19:10:02.273 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_forward": 98, 2024-04-25T19:10:02.273 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_discover": 54, 2024-04-25T19:10:02.273 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_dir_fetch": 0, 2024-04-25T19:10:02.273 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_remote_ino": 0, 2024-04-25T19:10:02.273 INFO:teuthology.orchestra.run.smithi079.stdout: "traverse_lock": 3, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "load_cent": 0, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "q": 0, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "exported": 3, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "exported_inodes": 148, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "imported": 3, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "imported_inodes": 19, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_dir_fetch": 0, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_backtrace_fetch": 0, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "openino_peer_discover": 0, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rfiles": 4, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rbytes": 234, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "root_rsnaps": 6, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T19:10:02.274 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_set_tag": 0, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_inotable_repaired": 0, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_inodes": 0, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "scrub_file_inodes": 0, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_inode_file_caps": 0, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_revoke": 30, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_grant": 154, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_trunc": 2, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps": 133, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_caps_dirty": 104, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_cap_release": 7, 2024-04-25T19:10:02.275 INFO:teuthology.orchestra.run.smithi079.stdout: "process_request_cap_release": 185 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_cache": { 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update": 7, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_update_receipt": 462, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_try_discover": 256, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_send_discover": 267, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "dir_handle_discover": 9, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays": 1, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_delayed": 0, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "num_strays_enqueuing": 0, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_created": 62, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_enqueued": 61, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_reintegrated": 0, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "strays_migrated": 0, 2024-04-25T19:10:02.276 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_processing": 0, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_enqueued": 0, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "num_recovering_prioritized": 0, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_started": 0, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "recovery_completed": 0, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_path": 2, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_quiesce_inode": 38, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_exportdir": 3, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_flush": 0, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragmentdir": 0, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_fragstats": 0, 2024-04-25T19:10:02.277 INFO:teuthology.orchestra.run.smithi079.stdout: "ireq_inodestats": 0 2024-04-25T19:10:02.278 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.278 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_log": { 2024-04-25T19:10:02.278 INFO:teuthology.orchestra.run.smithi079.stdout: "evlrg": 0, 2024-04-25T19:10:02.278 INFO:teuthology.orchestra.run.smithi079.stdout: "evadd": 438, 2024-04-25T19:10:02.278 INFO:teuthology.orchestra.run.smithi079.stdout: "evex": 0, 2024-04-25T19:10:02.278 INFO:teuthology.orchestra.run.smithi079.stdout: "evtrm": 0, 2024-04-25T19:10:02.278 INFO:teuthology.orchestra.run.smithi079.stdout: "ev": 438, 2024-04-25T19:10:02.278 INFO:teuthology.orchestra.run.smithi079.stdout: "evexg": 0, 2024-04-25T19:10:02.278 INFO:teuthology.orchestra.run.smithi079.stdout: "evexd": 0, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "segadd": 2, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "segex": 0, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "segtrm": 0, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "seg": 2, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "segmjr": 2, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "segexg": 0, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "segexd": 0, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "expos": 4194304, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "wrpos": 5023695, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "rdpos": 0, 2024-04-25T19:10:02.279 INFO:teuthology.orchestra.run.smithi079.stdout: "jlat": { 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 95, 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.239104450, 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.002516888 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "replayed": 0 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_mem": { 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "ino": 46, 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "ino+": 244, 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "ino-": 198, 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "dir": 15, 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "dir+": 238, 2024-04-25T19:10:02.280 INFO:teuthology.orchestra.run.smithi079.stdout: "dir-": 223, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "dn": 59, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "dn+": 402, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "dn-": 343, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "cap": 0, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "cap+": 82, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "cap-": 82, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "rss": 56228, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "heap": 223608 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_server": { 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_client_request": 614, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "dispatch_server_request": 9, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_request": 430, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_client_session": 12, 2024-04-25T19:10:02.281 INFO:teuthology.orchestra.run.smithi079.stdout: "handle_peer_request": 130, 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "req_create_latency": { 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 44, 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.087078881, 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001979065 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getattr_latency": { 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1, 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000195320, 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000195320 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getfilelock_latency": { 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.282 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "req_link_latency": { 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookup_latency": { 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 187, 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.058791187, 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000314391 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookuphash_latency": { 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.283 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupino_latency": { 2024-04-25T19:10:02.284 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.284 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.284 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.284 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupname_latency": { 2024-04-25T19:10:02.284 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.284 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.284 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.284 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupparent_latency": { 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lookupsnap_latency": { 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "req_lssnap_latency": { 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.285 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.286 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.286 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mkdir_latency": { 2024-04-25T19:10:02.286 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1, 2024-04-25T19:10:02.286 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.002368847, 2024-04-25T19:10:02.286 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.002368847 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mknod_latency": { 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "req_mksnap_latency": { 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "req_open_latency": { 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 13, 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.003176865, 2024-04-25T19:10:02.287 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000244374 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "req_readdir_latency": { 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 7, 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.004637601, 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000662514 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rename_latency": { 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 13, 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.033424433, 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.002571110 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "req_renamesnap_latency": { 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.288 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "req_snapdiff_latency": { 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmdir_latency": { 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1, 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.036603780, 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.036603780 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmsnap_latency": { 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.289 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "req_rmxattr_latency": { 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setattr_latency": { 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 2, 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.009634606, 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.004817303 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setdirlayout_latency": { 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.290 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setfilelock_latency": { 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setlayout_latency": { 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "req_setxattr_latency": { 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.291 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "req_symlink_latency": { 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 1, 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000991125, 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000991125 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "req_unlink_latency": { 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 61, 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.095215208, 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.001560905 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_revoke_eviction": 0, 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "cap_acquisition_throttle": 0, 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "req_getvxattr_latency": { 2024-04-25T19:10:02.292 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_sessions": { 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "session_count": 1, 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "session_add": 2, 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "session_remove": 1, 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_open": 1, 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "sessions_stale": 0, 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "total_load": 122, 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "average_load": 122, 2024-04-25T19:10:02.293 INFO:teuthology.orchestra.run.smithi079.stdout: "avg_session_uptime": 537, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "mdthresh_evicted": 0 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:10:02.294 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:10:02.295 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 289591, 2024-04-25T19:10:02.296 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 120, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 8224, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 171, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 171854, 2024-04-25T19:10:02.297 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 1208, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "objecter": { 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "op_active": 0, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "op_laggy": 0, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send": 229, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "op_send_bytes": 856752, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "op_resend": 31, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "op_reply": 229, 2024-04-25T19:10:02.298 INFO:teuthology.orchestra.run.smithi079.stdout: "op_latency": { 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 229, 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 1.076976746, 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.004702955 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "op_inflight": 0, 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "oplen_avg": { 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 229, 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 266 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "op": 229, 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "op_r": 3, 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "op_w": 226, 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "op_rmw": 0, 2024-04-25T19:10:02.299 INFO:teuthology.orchestra.run.smithi079.stdout: "op_pg": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_stat": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_create": 10, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_read": 3, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_write": 98, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writefull": 16, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_writesame": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_append": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_zero": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_truncate": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_delete": 73, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_mapext": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_sparse_read": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_clonerange": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_getxattr": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_setxattr": 20, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_cmpxattr": 0, 2024-04-25T19:10:02.300 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_rmxattr": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_resetxattrs": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_call": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_watch": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_notify": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_pgls_filter": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "osdop_other": 2, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_active": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_send": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_resend": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "linger_ping": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_active": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_send": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "poolop_resend": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_active": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_send": 0, 2024-04-25T19:10:02.301 INFO:teuthology.orchestra.run.smithi079.stdout: "poolstat_resend": 0, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_active": 0, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_send": 0, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "statfs_resend": 0, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "command_active": 0, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "command_send": 0, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "command_resend": 0, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "map_epoch": 104, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "map_full": 0, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "map_inc": 12, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_sessions": 12, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_open": 12, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_session_close": 0, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_laggy": 0, 2024-04-25T19:10:02.302 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_wr": 35, 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_rd": 0, 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_del": 9 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "oft": { 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_objs": 1, 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_kv_pairs": 0, 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_updates": 0, 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "omap_total_removes": 0 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "osd-slow-ops": { 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "slow_ops_count": 40 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "purge_queue": { 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops": 0, 2024-04-25T19:10:02.303 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_ops_high_water": 10, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing": 0, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executing_high_water": 10, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed_ops": 61, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_executed": 61, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "pq_item_in_journal": 0 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 5627, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 5429738, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 5627, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-04-25T19:10:02.304 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 5627, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 5429738, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_bytes": { 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-04-25T19:10:02.305 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 229, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 865745, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 229, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 865745, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-objecter_ops": { 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 1024, 2024-04-25T19:10:02.306 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 0, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 0, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 0, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 229, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 229, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 229, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 229, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle": { 2024-04-25T19:10:02.307 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 61, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 8993, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 61, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 3, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 8993, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:02.308 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-write_buf_throttle-0x557560e5c460": { 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 3758096384, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 438, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 829391, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 438, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 95, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 829391, 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:10:02.309 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:02.310 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:02.310 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:02.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:02.310 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:02.310 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-25T19:10:02.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:02 smithi045 ceph-mon[27776]: pgmap v333: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:10:02.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:02 smithi062 ceph-mon[32844]: pgmap v333: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T19:10:03.054 INFO:tasks.check_counter:Daemon mds.i mds.exported=3 2024-04-25T19:10:03.054 INFO:tasks.check_counter:Daemon mds.i mds.imported=3 2024-04-25T19:10:03.055 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=7 2024-04-25T19:10:03.055 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=462 2024-04-25T19:10:03.055 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T19:10:03.055 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T19:10:03.055 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T19:10:03.055 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-25T19:10:03.055 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T19:10:03.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T19:10:03.294 INFO:teuthology.orchestra.run.smithi079.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T19:10:04.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:04 smithi079 ceph-mon[33002]: pgmap v334: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:10:04.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:04 smithi045 ceph-mon[27776]: pgmap v334: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:10:04.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:04 smithi062 ceph-mon[32844]: pgmap v334: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T19:10:04.989 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-25T19:10:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T19:10:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 0, 2024-04-25T19:10:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 2, 2024-04-25T19:10:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 0, 2024-04-25T19:10:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 0, 2024-04-25T19:10:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 4, 2024-04-25T19:10:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 0, 2024-04-25T19:10:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.000886885, 2024-04-25T19:10:04.989 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.000142830, 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:04.990 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 2, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 192, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 320, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 190000, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 5, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 3, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.010820015, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.007358822, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.001235117, 2024-04-25T19:10:04.991 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 189, 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.039698548, 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000210045 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 190000 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T19:10:04.992 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_messages": 267, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages": 253, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_bytes": 1743264, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_bytes": 155568, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_created_connections": 2, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_active_connections": 1, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_total_time": 0.049085635, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_send_time": 0.014335723, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_recv_time": 0.038812671, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_running_fast_dispatch_time": 0.005812464, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 253, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.014945211, 2024-04-25T19:10:04.993 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000059071 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_handle_ack_lat": { 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_recv_encrypted_bytes": 1743264, 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "msgr_send_encrypted_bytes": 155568 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "cct": { 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "total_workers": 0, 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "unhealthy_workers": 0 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.994 INFO:teuthology.orchestra.run.smithi079.stdout: "mempool": { 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_bytes": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bloom_filter_items": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_alloc_items": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_data_items": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_other_items": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T19:10:04.995 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_bytes": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_extent_items": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_bytes": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_blob_items": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_fsck_items": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_bytes": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_txc_items": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T19:10:04.996 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_bytes": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "bluestore_writing_items": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_bytes": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_items": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_reader_items": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "bluefs_file_writer_items": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_bytes": 148709, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_anon_items": 31, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_bytes": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "buffer_meta_items": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_bytes": 0, 2024-04-25T19:10:04.997 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_items": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_bytes": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_mapbl_items": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_bytes": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_pglog_items": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_bytes": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_items": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_mapping_items": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_bytes": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "pgmap_items": 0, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_bytes": 928, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_co_items": 2, 2024-04-25T19:10:04.998 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_bytes": 0, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_1_items": 0, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_bytes": 0, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "unittest_2_items": 0 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "val": 0, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "max": 104857600, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "get_started": 0, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "get": 265, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "get_sum": 1703103, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_fail": 0, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "get_or_fail_success": 265, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "take": 0, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "take_sum": 0, 2024-04-25T19:10:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: "put": 265, 2024-04-25T19:10:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "put_sum": 1703103, 2024-04-25T19:10:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "wait": { 2024-04-25T19:10:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "avgcount": 0, 2024-04-25T19:10:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "sum": 0.000000000, 2024-04-25T19:10:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: "avgtime": 0.000000000 2024-04-25T19:10:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-25T19:10:05.000 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-25T19:10:05.796 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T19:10:05.796 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T19:10:05.797 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-25T19:10:05.797 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-25T19:10:05.797 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T19:10:05.797 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T19:10:05.797 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T19:10:05.797 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-25T19:10:05.797 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T19:10:05.808 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T19:10:05.808 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T19:10:05.808 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:10:05.871 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T19:10:05.871 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:10:05.927 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-25T19:10:06.398 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:10:06.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:06 smithi079 ceph-mon[33002]: pgmap v335: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T19:10:06.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:06 smithi045 ceph-mon[27776]: pgmap v335: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T19:10:06.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:06 smithi062 ceph-mon[32844]: pgmap v335: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T19:10:08.337 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:08 smithi045 ceph-mon[27776]: pgmap v336: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:10:08.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:08 smithi079 ceph-mon[33002]: pgmap v336: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:10:08.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.506+0000 7f7418a00700 1 -- 172.21.15.45:0/2998771988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7414103b30 msgr2=0x7f7414103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:08.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.506+0000 7f7418a00700 1 --2- 172.21.15.45:0/2998771988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7414103b30 0x7f7414103f10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7404009990 tx=0x7f7404009ca0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.506+0000 7f7418a00700 1 -- 172.21.15.45:0/2998771988 shutdown_connections 2024-04-25T19:10:08.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.507+0000 7f7418a00700 1 --2- 172.21.15.45:0/2998771988 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741410cf60 0x7f741410f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.507+0000 7f7418a00700 1 --2- 172.21.15.45:0/2998771988 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7414103b30 0x7f7414103f10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.507+0000 7f7418a00700 1 --2- 172.21.15.45:0/2998771988 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7414104450 0x7f741410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.507+0000 7f7418a00700 1 -- 172.21.15.45:0/2998771988 >> 172.21.15.45:0/2998771988 conn(0x7f74140fd360 msgr2=0x7f74140ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:08.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.507+0000 7f7418a00700 1 -- 172.21.15.45:0/2998771988 shutdown_connections 2024-04-25T19:10:08.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.507+0000 7f7418a00700 1 -- 172.21.15.45:0/2998771988 wait complete. 2024-04-25T19:10:08.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.508+0000 7f7418a00700 1 Processor -- start 2024-04-25T19:10:08.509 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.508+0000 7f7418a00700 1 -- start start 2024-04-25T19:10:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.508+0000 7f7418a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7414103b30 0x7f7414101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.508+0000 7f7418a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7414104450 0x7f7414101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.508+0000 7f7418a00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741410cf60 0x7f7414102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7418a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7414112ba0 con 0x7f7414103b30 2024-04-25T19:10:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7418a00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f7414112a20 con 0x7f741410cf60 2024-04-25T19:10:08.510 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7418a00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7414112d20 con 0x7f7414104450 2024-04-25T19:10:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7413400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7414103b30 0x7f7414101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7412a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7414104450 0x7f7414101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7412a00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7414104450 0x7f7414101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48284/0 (socket says 172.21.15.45:48284) 2024-04-25T19:10:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7412a00700 1 -- 172.21.15.45:0/3492081349 learned_addr learned my addr 172.21.15.45:0/3492081349 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:10:08.511 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7413e00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741410cf60 0x7f7414102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7412a00700 1 -- 172.21.15.45:0/3492081349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741410cf60 msgr2=0x7f7414102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7412a00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741410cf60 0x7f7414102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7412a00700 1 -- 172.21.15.45:0/3492081349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7414103b30 msgr2=0x7f7414101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7412a00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7414103b30 0x7f7414101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.509+0000 7f7412a00700 1 -- 172.21.15.45:0/3492081349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74141a75f0 con 0x7f7414104450 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.510+0000 7f7412a00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7414104450 0x7f7414101cd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f740000e9c0 tx=0x7f740000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.510+0000 7f740be00700 1 -- 172.21.15.45:0/3492081349 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400004a30 con 0x7f7414104450 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.510+0000 7f740be00700 1 -- 172.21.15.45:0/3492081349 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f740000cc70 con 0x7f7414104450 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.510+0000 7f7418a00700 1 -- 172.21.15.45:0/3492081349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74141a78e0 con 0x7f7414104450 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.510+0000 7f740be00700 1 -- 172.21.15.45:0/3492081349 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000054f0 con 0x7f7414104450 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.510+0000 7f7418a00700 1 -- 172.21.15.45:0/3492081349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74141a7e20 con 0x7f7414104450 2024-04-25T19:10:08.512 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.511+0000 7f7418a00700 1 -- 172.21.15.45:0/3492081349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7414069730 con 0x7f7414104450 2024-04-25T19:10:08.515 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.514+0000 7f740be00700 1 -- 172.21.15.45:0/3492081349 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f740000c7c0 con 0x7f7414104450 2024-04-25T19:10:08.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.515+0000 7f740be00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73f80a3db0 0x7f73f80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:08.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.515+0000 7f740be00700 1 -- 172.21.15.45:0/3492081349 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8571+0+0 (secure 0 0 0) 0x7f74000d0980 con 0x7f7414104450 2024-04-25T19:10:08.516 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.515+0000 7f7413400700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73f80a3db0 0x7f73f80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:08.517 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.516+0000 7f7413400700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73f80a3db0 0x7f73f80a6270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7404006010 tx=0x7f740400b260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:08.518 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.517+0000 7f740be00700 1 -- 172.21.15.45:0/3492081349 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f740000ce20 con 0x7f7414104450 2024-04-25T19:10:08.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:08 smithi062 ceph-mon[32844]: pgmap v336: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:10:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.765+0000 7f7418a00700 1 -- 172.21.15.45:0/3492081349 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7414102c70 con 0x7f7414104450 2024-04-25T19:10:08.767 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.766+0000 7f740be00700 1 -- 172.21.15.45:0/3492081349 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v104) ==== 81+0+943 (secure 0 0 0) 0x7f740009cf90 con 0x7f7414104450 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/943541413 2024-04-26T18:59:18.746957+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2599503580 2024-04-26T18:59:18.746957+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/943541413 2024-04-26T18:59:18.746957+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3590060181 2024-04-26T18:59:18.746957+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/4010951391 2024-04-26T18:59:18.746957+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1117554794 2024-04-26T18:47:16.094556+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3367087840 2024-04-26T18:59:18.746957+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/3065212608 2024-04-26T18:47:16.094556+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/2122448921 2024-04-26T18:46:42.822772+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1563847530 2024-04-26T18:47:16.094556+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/817529633 2024-04-26T18:47:16.094556+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/1502518510 2024-04-26T18:46:42.822772+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6800/1404432740 2024-04-26T18:46:42.822772+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/3065212608 2024-04-26T18:47:16.094556+0000 2024-04-25T19:10:08.768 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:0/3157265170 2024-04-26T18:46:42.822772+0000 2024-04-25T19:10:08.769 INFO:teuthology.orchestra.run.smithi045.stdout:172.21.15.45:6801/1404432740 2024-04-26T18:46:42.822772+0000 2024-04-25T19:10:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 -- 172.21.15.45:0/3492081349 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73f80a3db0 msgr2=0x7f73f80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73f80a3db0 0x7f73f80a6270 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7404006010 tx=0x7f740400b260 comp rx=0 tx=0).stop 2024-04-25T19:10:08.770 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 -- 172.21.15.45:0/3492081349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7414104450 msgr2=0x7f7414101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7414104450 0x7f7414101cd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f740000e9c0 tx=0x7f740000ecd0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 -- 172.21.15.45:0/3492081349 shutdown_connections 2024-04-25T19:10:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f73f80a3db0 0x7f73f80a6270 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f741410cf60 0x7f7414102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7414103b30 0x7f7414101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 --2- 172.21.15.45:0/3492081349 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7414104450 0x7f7414101cd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 -- 172.21.15.45:0/3492081349 >> 172.21.15.45:0/3492081349 conn(0x7f74140fd360 msgr2=0x7f74140fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 -- 172.21.15.45:0/3492081349 shutdown_connections 2024-04-25T19:10:08.771 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:08.769+0000 7f7418a00700 1 -- 172.21.15.45:0/3492081349 wait complete. 2024-04-25T19:10:08.772 INFO:teuthology.orchestra.run.smithi045.stderr:listed 16 entries 2024-04-25T19:10:09.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:09 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3492081349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T19:10:09.242 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:09 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3492081349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T19:10:09.581 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T19:10:09.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:10:09.581 DEBUG:teuthology.orchestra.run.smithi045:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T19:10:09.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:09 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3492081349' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T19:10:09.801 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T19:10:09.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:10:09.801 DEBUG:teuthology.orchestra.run.smithi045:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T19:10:09.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:10:09.883 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T19:10:09.961 INFO:teuthology.orchestra.run.smithi045.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T19:10:09.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:10:09.961 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T19:10:10.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:10:10.043 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-25T19:10:10.133 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T19:10:10.133 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-25T19:10:10.133 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T19:10:10.172 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/343e484e20aebf1f48612fc0ed20af9a41ae639e3a4d0c266803f13f6b1fddba/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VEA3II4ED5JIZXE6YTJRMX2LE2:/var/lib/containers/storage/overlay/l/COCSE6OYMXB2SNKJEBDZ364OGT,upperdir=/var/lib/containers/storage/overlay/343e484e20aebf1f48612fc0ed20af9a41ae639e3a4d0c266803f13f6b1fddba/diff,workdir=/var/lib/containers/storage/overlay/343e484e20aebf1f48612fc0ed20af9a41ae639e3a4d0c266803f13f6b1fddba/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/9add4876a99f91b71f448e15d9f427a4f78685b8390a0cb7da2f9f51a8760f16/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VEA3II4ED5JIZXE6YTJRMX2LE2:/var/lib/containers/storage/overlay/l/COCSE6OYMXB2SNKJEBDZ364OGT,upperdir=/var/lib/containers/storage/overlay/9add4876a99f91b71f448e15d9f427a4f78685b8390a0cb7da2f9f51a8760f16/diff,workdir=/var/lib/containers/storage/overlay/9add4876a99f91b71f448e15d9f427a4f78685b8390a0cb7da2f9f51a8760f16/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/a56df39309c06cf74a37adae2b20f0d445687e937e432a0352568fa3489b258a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VEA3II4ED5JIZXE6YTJRMX2LE2:/var/lib/containers/storage/overlay/l/COCSE6OYMXB2SNKJEBDZ364OGT,upperdir=/var/lib/containers/storage/overlay/a56df39309c06cf74a37adae2b20f0d445687e937e432a0352568fa3489b258a/diff,workdir=/var/lib/containers/storage/overlay/a56df39309c06cf74a37adae2b20f0d445687e937e432a0352568fa3489b258a/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f08e87bd7cd3843498d4a2f2ca2304739f219782a025254aa26a07d19cfa5095/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VEA3II4ED5JIZXE6YTJRMX2LE2:/var/lib/containers/storage/overlay/l/COCSE6OYMXB2SNKJEBDZ364OGT,upperdir=/var/lib/containers/storage/overlay/f08e87bd7cd3843498d4a2f2ca2304739f219782a025254aa26a07d19cfa5095/diff,workdir=/var/lib/containers/storage/overlay/f08e87bd7cd3843498d4a2f2ca2304739f219782a025254aa26a07d19cfa5095/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/31bba1671b98b073ff5ef9230305de605866f7326947c12c7dc9669883e5e45a/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VEA3II4ED5JIZXE6YTJRMX2LE2:/var/lib/containers/storage/overlay/l/COCSE6OYMXB2SNKJEBDZ364OGT,upperdir=/var/lib/containers/storage/overlay/31bba1671b98b073ff5ef9230305de605866f7326947c12c7dc9669883e5e45a/diff,workdir=/var/lib/containers/storage/overlay/31bba1671b98b073ff5ef9230305de605866f7326947c12c7dc9669883e5e45a/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d673524257068215ad0c0c1ee2bde1ac7c0029c676535ea4241528f1e9941d38/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/VEA3II4ED5JIZXE6YTJRMX2LE2:/var/lib/containers/storage/overlay/l/COCSE6OYMXB2SNKJEBDZ364OGT,upperdir=/var/lib/containers/storage/overlay/d673524257068215ad0c0c1ee2bde1ac7c0029c676535ea4241528f1e9941d38/diff,workdir=/var/lib/containers/storage/overlay/d673524257068215ad0c0c1ee2bde1ac7c0029c676535ea4241528f1e9941d38/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/726a8cc023132fb08250f206f03e4d1e7260ce8f42b8fbb33c09048c4bf40d72/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VEA3II4ED5JIZXE6YTJRMX2LE2:/var/lib/containers/storage/overlay/l/COCSE6OYMXB2SNKJEBDZ364OGT,upperdir=/var/lib/containers/storage/overlay/726a8cc023132fb08250f206f03e4d1e7260ce8f42b8fbb33c09048c4bf40d72/diff,workdir=/var/lib/containers/storage/overlay/726a8cc023132fb08250f206f03e4d1e7260ce8f42b8fbb33c09048c4bf40d72/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/18904252779e25a8aeef4b6eb9917dce9d69dcab9a7b90252d35b8ff1005c341/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VEA3II4ED5JIZXE6YTJRMX2LE2:/var/lib/containers/storage/overlay/l/COCSE6OYMXB2SNKJEBDZ364OGT,upperdir=/var/lib/containers/storage/overlay/18904252779e25a8aeef4b6eb9917dce9d69dcab9a7b90252d35b8ff1005c341/diff,workdir=/var/lib/containers/storage/overlay/18904252779e25a8aeef4b6eb9917dce9d69dcab9a7b90252d35b8ff1005c341/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/dead3f8cc064ffedd42e27e2e64e9376dd9e46fb7bbc7f9cfabae9c8e6871c74/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VEA3II4ED5JIZXE6YTJRMX2LE2:/var/lib/containers/storage/overlay/l/COCSE6OYMXB2SNKJEBDZ364OGT,upperdir=/var/lib/containers/storage/overlay/dead3f8cc064ffedd42e27e2e64e9376dd9e46fb7bbc7f9cfabae9c8e6871c74/diff,workdir=/var/lib/containers/storage/overlay/dead3f8cc064ffedd42e27e2e64e9376dd9e46fb7bbc7f9cfabae9c8e6871c74/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/dfe6b7b3ed3d8b2fec83dad4d4e0e527f5ca8f8391c0f9ebb54b16bff7ea1036/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/VEA3II4ED5JIZXE6YTJRMX2LE2:/var/lib/containers/storage/overlay/l/COCSE6OYMXB2SNKJEBDZ364OGT,upperdir=/var/lib/containers/storage/overlay/dfe6b7b3ed3d8b2fec83dad4d4e0e527f5ca8f8391c0f9ebb54b16bff7ea1036/diff,workdir=/var/lib/containers/storage/overlay/dfe6b7b3ed3d8b2fec83dad4d4e0e527f5ca8f8391c0f9ebb54b16bff7ea1036/work,redirect_dir=on,uuid=on,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 2024-04-25T19:10:10.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:10:10.173 DEBUG:teuthology.orchestra.run.smithi045:> ip netns list 2024-04-25T19:10:10.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T19:10:10.264 DEBUG:teuthology.orchestra.run.smithi045:> sudo ip link delete ceph-brx 2024-04-25T19:10:10.327 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:10 smithi045 ceph-mon[27776]: pgmap v337: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:10:10.344 INFO:teuthology.orchestra.run.smithi045.stderr:Cannot find device "ceph-brx" 2024-04-25T19:10:10.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:10:10.347 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T19:10:10.361 INFO:tasks.cephadm:Teardown begin 2024-04-25T19:10:10.362 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T19:10:10.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:10:10.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:10 smithi079 ceph-mon[33002]: pgmap v337: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:10:10.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:10 smithi062 ceph-mon[32844]: pgmap v337: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T19:10:10.833 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:10:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:12 smithi079 ceph-mon[33002]: pgmap v338: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-25T19:10:12.184 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:12 smithi079 ceph-mon[33002]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T19:10:12.415 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:12 smithi062 ceph-mon[32844]: pgmap v338: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-25T19:10:12.415 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:12 smithi062 ceph-mon[32844]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T19:10:12.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:12 smithi045 ceph-mon[27776]: pgmap v338: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-25T19:10:12.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:12 smithi045 ceph-mon[27776]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T19:10:12.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.974+0000 7fe09ba00700 1 -- 172.21.15.45:0/3703149742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe09406bab0 msgr2=0x7fe09410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:12.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.974+0000 7fe09ba00700 1 --2- 172.21.15.45:0/3703149742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe09406bab0 0x7fe09410f920 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00b0a0 tx=0x7fe08c00b3b0 comp rx=0 tx=0).stop 2024-04-25T19:10:12.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.974+0000 7fe09ba00700 1 -- 172.21.15.45:0/3703149742 shutdown_connections 2024-04-25T19:10:12.976 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.974+0000 7fe09ba00700 1 --2- 172.21.15.45:0/3703149742 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe09406bab0 0x7fe09410f920 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.974+0000 7fe09ba00700 1 --2- 172.21.15.45:0/3703149742 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe094108a50 0x7fe094108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.974+0000 7fe09ba00700 1 --2- 172.21.15.45:0/3703149742 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe09406b110 0x7fe09406b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.974+0000 7fe09ba00700 1 -- 172.21.15.45:0/3703149742 >> 172.21.15.45:0/3703149742 conn(0x7fe0940788f0 msgr2=0x7fe094078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.975+0000 7fe09ba00700 1 -- 172.21.15.45:0/3703149742 shutdown_connections 2024-04-25T19:10:12.977 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.975+0000 7fe09ba00700 1 -- 172.21.15.45:0/3703149742 wait complete. 2024-04-25T19:10:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.975+0000 7fe09ba00700 1 Processor -- start 2024-04-25T19:10:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe09ba00700 1 -- start start 2024-04-25T19:10:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe09ba00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe09406b110 0x7fe094199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe09ba00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe09406bab0 0x7fe094199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe09ba00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe094108a50 0x7fe09419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe09ba00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe094112c40 con 0x7fe094108a50 2024-04-25T19:10:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe09ba00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fe094112ac0 con 0x7fe09406bab0 2024-04-25T19:10:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe09ba00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe094112dc0 con 0x7fe09406b110 2024-04-25T19:10:12.979 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe093e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe09406bab0 0x7fe094199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe099000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe09406b110 0x7fe094199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe093e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe09406bab0 0x7fe094199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:48032/0 (socket says 172.21.15.45:48032) 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe093e00700 1 -- 172.21.15.45:0/2232430670 learned_addr learned my addr 172.21.15.45:0/2232430670 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.976+0000 7fe099000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe09406b110 0x7fe094199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:48292/0 (socket says 172.21.15.45:48292) 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe099a00700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe094108a50 0x7fe09419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe093e00700 1 -- 172.21.15.45:0/2232430670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe09406b110 msgr2=0x7fe094199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe093e00700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe09406b110 0x7fe094199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe093e00700 1 -- 172.21.15.45:0/2232430670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe094108a50 msgr2=0x7fe09419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe093e00700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe094108a50 0x7fe09419da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe093e00700 1 -- 172.21.15.45:0/2232430670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe09419e140 con 0x7fe09406bab0 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe099000700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe09406b110 0x7fe094199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe093e00700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe09406bab0 0x7fe094199670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe08000e9c0 tx=0x7fe08000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe091600700 1 -- 172.21.15.45:0/2232430670 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe08000cbd0 con 0x7fe09406bab0 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe09ba00700 1 -- 172.21.15.45:0/2232430670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe09419e430 con 0x7fe09406bab0 2024-04-25T19:10:12.980 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe091600700 1 -- 172.21.15.45:0/2232430670 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe08000ee60 con 0x7fe09406bab0 2024-04-25T19:10:12.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.977+0000 7fe091600700 1 -- 172.21.15.45:0/2232430670 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0800055c0 con 0x7fe09406bab0 2024-04-25T19:10:12.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.978+0000 7fe09ba00700 1 -- 172.21.15.45:0/2232430670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0941a5cb0 con 0x7fe09406bab0 2024-04-25T19:10:12.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.978+0000 7fe09ba00700 1 -- 172.21.15.45:0/2232430670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe094104190 con 0x7fe09406bab0 2024-04-25T19:10:12.981 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.980+0000 7fe091600700 1 -- 172.21.15.45:0/2232430670 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fe080004750 con 0x7fe09406bab0 2024-04-25T19:10:12.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.982+0000 7fe091600700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe07c0a3db0 0x7fe07c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:12.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.982+0000 7fe091600700 1 -- 172.21.15.45:0/2232430670 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8615+0+0 (secure 0 0 0) 0x7fe0800d0b80 con 0x7fe09406bab0 2024-04-25T19:10:12.983 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.982+0000 7fe099000700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe07c0a3db0 0x7fe07c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:12.984 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.983+0000 7fe099000700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe07c0a3db0 0x7fe07c0a6270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe088009fd0 tx=0x7fe088009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:12.987 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:12.986+0000 7fe091600700 1 -- 172.21.15.45:0/2232430670 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe08009d160 con 0x7fe09406bab0 2024-04-25T19:10:13.256 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.255+0000 7fe09ba00700 1 -- 172.21.15.45:0/2232430670 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe09402df80 con 0x7fe09406bab0 2024-04-25T19:10:13.257 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.256+0000 7fe091600700 1 -- 172.21.15.45:0/2232430670 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11853 (secure 0 0 0) 0x7fe08009ca60 con 0x7fe09406bab0 2024-04-25T19:10:13.257 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:10:13.258 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T19:10:12.144200+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25789":{"gid":25789,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/620692737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":620692737},{"type":"v1","addr":"172.21.15.79:6837","nonce":620692737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24512]},"id":1}]} 2024-04-25T19:10:13.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.258+0000 7fe09ba00700 1 -- 172.21.15.45:0/2232430670 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe07c0a3db0 msgr2=0x7fe07c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:13.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.258+0000 7fe09ba00700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe07c0a3db0 0x7fe07c0a6270 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe088009fd0 tx=0x7fe088009260 comp rx=0 tx=0).stop 2024-04-25T19:10:13.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.258+0000 7fe09ba00700 1 -- 172.21.15.45:0/2232430670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe09406bab0 msgr2=0x7fe094199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:13.260 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.258+0000 7fe09ba00700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe09406bab0 0x7fe094199670 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe08000e9c0 tx=0x7fe08000ecd0 comp rx=0 tx=0).stop 2024-04-25T19:10:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.259+0000 7fe09ba00700 1 -- 172.21.15.45:0/2232430670 shutdown_connections 2024-04-25T19:10:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.259+0000 7fe09ba00700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fe07c0a3db0 0x7fe07c0a6270 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.259+0000 7fe09ba00700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe094108a50 0x7fe09419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.259+0000 7fe09ba00700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe09406b110 0x7fe094199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.259+0000 7fe09ba00700 1 --2- 172.21.15.45:0/2232430670 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fe09406bab0 0x7fe094199670 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.259+0000 7fe09ba00700 1 -- 172.21.15.45:0/2232430670 >> 172.21.15.45:0/2232430670 conn(0x7fe0940788f0 msgr2=0x7fe0940ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.259+0000 7fe09ba00700 1 -- 172.21.15.45:0/2232430670 shutdown_connections 2024-04-25T19:10:13.261 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:13.259+0000 7fe09ba00700 1 -- 172.21.15.45:0/2232430670 wait complete. 2024-04-25T19:10:13.262 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 37 2024-04-25T19:10:13.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:13 smithi045 ceph-mon[27776]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:10:13.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:13 smithi045 ceph-mon[27776]: pgmap v340: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-25T19:10:13.596 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:13 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2232430670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:10:13.748 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:13 smithi062 ceph-mon[32844]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:10:13.748 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:13 smithi062 ceph-mon[32844]: pgmap v340: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-25T19:10:13.748 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:13 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2232430670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:10:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:13 smithi079 ceph-mon[33002]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T19:10:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:13 smithi079 ceph-mon[33002]: pgmap v340: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 27 op/s 2024-04-25T19:10:13.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:13 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2232430670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:10:14.194 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T19:10:14.674 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:10:16.214 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:16 smithi045 ceph-mon[27776]: pgmap v341: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T19:10:16.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:16 smithi079 ceph-mon[33002]: pgmap v341: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T19:10:16.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:16 smithi062 ceph-mon[32844]: pgmap v341: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T19:10:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.044+0000 7fdab5c00700 1 -- 172.21.15.45:0/1203324272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdab0109c80 msgr2=0x7fdab010dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.044+0000 7fdab5c00700 1 --2- 172.21.15.45:0/1203324272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdab0109c80 0x7fdab010dcd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0098f0 tx=0x7fda9c009c00 comp rx=0 tx=0).stop 2024-04-25T19:10:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.044+0000 7fdab5c00700 1 -- 172.21.15.45:0/1203324272 shutdown_connections 2024-04-25T19:10:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.044+0000 7fdab5c00700 1 --2- 172.21.15.45:0/1203324272 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab010e3b0 0x7fdab0115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.044+0000 7fdab5c00700 1 --2- 172.21.15.45:0/1203324272 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab01092d0 0x7fdab01096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.044+0000 7fdab5c00700 1 --2- 172.21.15.45:0/1203324272 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdab0109c80 0x7fdab010dcd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.044+0000 7fdab5c00700 1 -- 172.21.15.45:0/1203324272 >> 172.21.15.45:0/1203324272 conn(0x7fdab0076210 msgr2=0x7fdab0107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.044+0000 7fdab5c00700 1 -- 172.21.15.45:0/1203324272 shutdown_connections 2024-04-25T19:10:17.046 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.044+0000 7fdab5c00700 1 -- 172.21.15.45:0/1203324272 wait complete. 2024-04-25T19:10:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.046+0000 7fdab5c00700 1 Processor -- start 2024-04-25T19:10:17.047 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.046+0000 7fdab5c00700 1 -- start start 2024-04-25T19:10:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.046+0000 7fdab5c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdab01092d0 0x7fdab01a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.046+0000 7fdab5c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab0109c80 0x7fdab01a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.046+0000 7fdab5c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab010e3b0 0x7fdab019f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.046+0000 7fdab5c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdab0119270 con 0x7fdab010e3b0 2024-04-25T19:10:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdab5c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01190f0 con 0x7fdab01092d0 2024-04-25T19:10:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdab5c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdab01193f0 con 0x7fdab0109c80 2024-04-25T19:10:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdaaf400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab010e3b0 0x7fdab019f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:17.048 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdaaf400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab010e3b0 0x7fdab019f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34392/0 (socket says 172.21.15.45:34392) 2024-04-25T19:10:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdaaf400700 1 -- 172.21.15.45:0/3733299730 learned_addr learned my addr 172.21.15.45:0/3733299730 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:10:17.049 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdaaea00700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdab01092d0 0x7fdab01a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdaae000700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab0109c80 0x7fdab01a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdaaf400700 1 -- 172.21.15.45:0/3733299730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab0109c80 msgr2=0x7fdab01a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdaaf400700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab0109c80 0x7fdab01a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdaaf400700 1 -- 172.21.15.45:0/3733299730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdab01092d0 msgr2=0x7fdab01a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdaaf400700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdab01092d0 0x7fdab01a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.047+0000 7fdaaf400700 1 -- 172.21.15.45:0/3733299730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab019ff50 con 0x7fdab010e3b0 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.048+0000 7fdaaea00700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdab01092d0 0x7fdab01a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.048+0000 7fdaae000700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab0109c80 0x7fdab01a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.048+0000 7fdaaf400700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab010e3b0 0x7fdab019f670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800ef30 tx=0x7fdaa800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.048+0000 7fdaa7400700 1 -- 172.21.15.45:0/3733299730 <== mon.0 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80092e0 con 0x7fdab010e3b0 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.048+0000 7fdaa7400700 1 -- 172.21.15.45:0/3733299730 <== mon.0 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa8004bb0 con 0x7fdab010e3b0 2024-04-25T19:10:17.050 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.048+0000 7fdaa7400700 1 -- 172.21.15.45:0/3733299730 <== mon.0 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8004020 con 0x7fdab010e3b0 2024-04-25T19:10:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.048+0000 7fdab5c00700 1 -- 172.21.15.45:0/3733299730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab01a01e0 con 0x7fdab010e3b0 2024-04-25T19:10:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.048+0000 7fdab5c00700 1 -- 172.21.15.45:0/3733299730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab01ac520 con 0x7fdab010e3b0 2024-04-25T19:10:17.051 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.050+0000 7fdab5c00700 1 -- 172.21.15.45:0/3733299730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab010a7b0 con 0x7fdab010e3b0 2024-04-25T19:10:17.053 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.052+0000 7fdaa7400700 1 -- 172.21.15.45:0/3733299730 <== mon.0 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fdaa801b030 con 0x7fdab010e3b0 2024-04-25T19:10:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.053+0000 7fdaa7400700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fda940a3e30 0x7fda940a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:17.054 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.053+0000 7fdaa7400700 1 -- 172.21.15.45:0/3733299730 <== mon.0 v2:172.21.15.45:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8571+0+0 (secure 0 0 0) 0x7fdaa80d0c40 con 0x7fdab010e3b0 2024-04-25T19:10:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.054+0000 7fdaaea00700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fda940a3e30 0x7fda940a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:17.055 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.054+0000 7fdaaea00700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fda940a3e30 0x7fda940a62f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009750 tx=0x7fdaa0009310 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:17.058 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.056+0000 7fdaa7400700 1 -- 172.21.15.45:0/3733299730 <== mon.0 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdaa8022020 con 0x7fdab010e3b0 2024-04-25T19:10:17.337 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.335+0000 7fdab5c00700 1 -- 172.21.15.45:0/3733299730 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdab01a0ba0 con 0x7fdab010e3b0 2024-04-25T19:10:17.338 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.337+0000 7fdaa7400700 1 -- 172.21.15.45:0/3733299730 <== mon.0 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11853 (secure 0 0 0) 0x7fdaa8010040 con 0x7fdab010e3b0 2024-04-25T19:10:17.339 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:10:17.339 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14364,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/4023030153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":4023030153},{"type":"v1","addr":"172.21.15.45:6835","nonce":4023030153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14400,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/3080448067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":3080448067},{"type":"v1","addr":"172.21.15.45:6839","nonce":3080448067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24434,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6833/1072807299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6832","nonce":1072807299},{"type":"v1","addr":"172.21.15.62:6833","nonce":1072807299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24445,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6833/990407858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":990407858},{"type":"v1","addr":"172.21.15.79:6833","nonce":990407858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24452,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/496487622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":496487622},{"type":"v1","addr":"172.21.15.45:6837","nonce":496487622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24464,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.62:6835/3037892802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6834","nonce":3037892802},{"type":"v1","addr":"172.21.15.62:6835","nonce":3037892802}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T18:55:26.025536+0000","modified":"2024-04-25T19:10:12.144200+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24524,"mds_1":24512,"mds_2":24505},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6835/1201787807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":1201787807},{"type":"v1","addr":"172.21.15.79:6835","nonce":1201787807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24485":{"gid":24485,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.62:6837/2985890701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6836","nonce":2985890701},{"type":"v1","addr":"172.21.15.62:6837","nonce":2985890701}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.79:6839/235958219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6838","nonce":235958219},{"type":"v1","addr":"172.21.15.79:6839","nonce":235958219}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24512":{"gid":24512,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.45:6841/2490735053","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6840","nonce":2490735053},{"type":"v1","addr":"172.21.15.45:6841","nonce":2490735053}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24524":{"gid":24524,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":7,"addr":"172.21.15.62:6839/3288398937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6838","nonce":3288398937},{"type":"v1","addr":"172.21.15.62:6839","nonce":3288398937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25789":{"gid":25789,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.79:6837/620692737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6836","nonce":620692737},{"type":"v1","addr":"172.21.15.79:6837","nonce":620692737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24524,"qdb_cluster":[24524,24505,24512]},"id":1}]} 2024-04-25T19:10:17.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 -- 172.21.15.45:0/3733299730 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fda940a3e30 msgr2=0x7fda940a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:17.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fda940a3e30 0x7fda940a62f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009750 tx=0x7fdaa0009310 comp rx=0 tx=0).stop 2024-04-25T19:10:17.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 -- 172.21.15.45:0/3733299730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab010e3b0 msgr2=0x7fdab019f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:17.341 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab010e3b0 0x7fdab019f670 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800ef30 tx=0x7fdaa800c3a0 comp rx=0 tx=0).stop 2024-04-25T19:10:17.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 -- 172.21.15.45:0/3733299730 shutdown_connections 2024-04-25T19:10:17.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fda940a3e30 0x7fda940a62f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:17.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fdab01092d0 0x7fdab01a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:17.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdab010e3b0 0x7fdab019f670 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:17.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 --2- 172.21.15.45:0/3733299730 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdab0109c80 0x7fdab01a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:17.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 -- 172.21.15.45:0/3733299730 >> 172.21.15.45:0/3733299730 conn(0x7fdab0076210 msgr2=0x7fdab01086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:17.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 -- 172.21.15.45:0/3733299730 shutdown_connections 2024-04-25T19:10:17.342 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:17.340+0000 7fdab5c00700 1 -- 172.21.15.45:0/3733299730 wait complete. 2024-04-25T19:10:17.343 INFO:teuthology.orchestra.run.smithi045.stderr:dumped fsmap epoch 37 2024-04-25T19:10:17.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:17 smithi045 ceph-mon[27776]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T19:10:17.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:17 smithi045 ceph-mon[27776]: pgmap v343: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T19:10:17.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:17 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3733299730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:10:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:17 smithi062 ceph-mon[32844]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T19:10:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:17 smithi062 ceph-mon[32844]: pgmap v343: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T19:10:17.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:17 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3733299730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:10:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:17 smithi079 ceph-mon[33002]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T19:10:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:17 smithi079 ceph-mon[33002]: pgmap v343: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T19:10:17.934 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:17 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3733299730' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T19:10:18.106 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T19:10:18.577 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:10:20.182 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:20 smithi045 ceph-mon[27776]: pgmap v344: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 383 B/s wr, 4 op/s 2024-04-25T19:10:20.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:20 smithi079 ceph-mon[33002]: pgmap v344: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 383 B/s wr, 4 op/s 2024-04-25T19:10:20.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:20 smithi062 ceph-mon[32844]: pgmap v344: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 383 B/s wr, 4 op/s 2024-04-25T19:10:20.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.748+0000 7fbc1f600700 1 -- 172.21.15.45:0/1706943661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc181092d0 msgr2=0x7fbc181096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:20.750 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.748+0000 7fbc1f600700 1 --2- 172.21.15.45:0/1706943661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc181092d0 0x7fbc181096b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009920 tx=0x7fbc0c009c30 comp rx=0 tx=0).stop 2024-04-25T19:10:20.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.749+0000 7fbc1f600700 1 -- 172.21.15.45:0/1706943661 shutdown_connections 2024-04-25T19:10:20.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.749+0000 7fbc1f600700 1 --2- 172.21.15.45:0/1706943661 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc1810e3b0 0x7fbc18115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:20.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.749+0000 7fbc1f600700 1 --2- 172.21.15.45:0/1706943661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc181092d0 0x7fbc181096b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:20.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.749+0000 7fbc1f600700 1 --2- 172.21.15.45:0/1706943661 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc18109c80 0x7fbc1810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:20.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.749+0000 7fbc1f600700 1 -- 172.21.15.45:0/1706943661 >> 172.21.15.45:0/1706943661 conn(0x7fbc18076210 msgr2=0x7fbc18107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:20.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.749+0000 7fbc1f600700 1 -- 172.21.15.45:0/1706943661 shutdown_connections 2024-04-25T19:10:20.751 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.749+0000 7fbc1f600700 1 -- 172.21.15.45:0/1706943661 wait complete. 2024-04-25T19:10:20.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.750+0000 7fbc1f600700 1 Processor -- start 2024-04-25T19:10:20.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.750+0000 7fbc1f600700 1 -- start start 2024-04-25T19:10:20.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.750+0000 7fbc1f600700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc181092d0 0x7fbc181a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:20.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.750+0000 7fbc1f600700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc18109c80 0x7fbc181a5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:20.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.750+0000 7fbc1f600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810e3b0 0x7fbc1819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:20.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.750+0000 7fbc1f600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbc18119290 con 0x7fbc1810e3b0 2024-04-25T19:10:20.752 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.750+0000 7fbc1f600700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7fbc18119110 con 0x7fbc18109c80 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.750+0000 7fbc1f600700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbc18119410 con 0x7fbc181092d0 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc1d600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810e3b0 0x7fbc1819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc1d600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810e3b0 0x7fbc1819f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:34400/0 (socket says 172.21.15.45:34400) 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc1d600700 1 -- 172.21.15.45:0/3542139673 learned_addr learned my addr 172.21.15.45:0/3542139673 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc1cc00700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc181092d0 0x7fbc181a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc17e00700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc18109c80 0x7fbc181a5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc1cc00700 1 -- 172.21.15.45:0/3542139673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc18109c80 msgr2=0x7fbc181a5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc1cc00700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc18109c80 0x7fbc181a5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc1cc00700 1 -- 172.21.15.45:0/3542139673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810e3b0 msgr2=0x7fbc1819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc1cc00700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810e3b0 0x7fbc1819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:20.753 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc1cc00700 1 -- 172.21.15.45:0/3542139673 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1819ff70 con 0x7fbc181092d0 2024-04-25T19:10:20.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.751+0000 7fbc1cc00700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc181092d0 0x7fbc181a5570 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c000c00 tx=0x7fbc0c00f950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:20.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.752+0000 7fbc15600700 1 -- 172.21.15.45:0/3542139673 <== mon.1 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c027070 con 0x7fbc181092d0 2024-04-25T19:10:20.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.752+0000 7fbc1f600700 1 -- 172.21.15.45:0/3542139673 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc181a0200 con 0x7fbc181092d0 2024-04-25T19:10:20.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.752+0000 7fbc15600700 1 -- 172.21.15.45:0/3542139673 <== mon.1 v2:172.21.15.79:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc0c0054f0 con 0x7fbc181092d0 2024-04-25T19:10:20.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.752+0000 7fbc15600700 1 -- 172.21.15.45:0/3542139673 <== mon.1 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c004930 con 0x7fbc181092d0 2024-04-25T19:10:20.754 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.752+0000 7fbc1f600700 1 -- 172.21.15.45:0/3542139673 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc181ac320 con 0x7fbc181092d0 2024-04-25T19:10:20.755 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.753+0000 7fbc1f600700 1 -- 172.21.15.45:0/3542139673 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc1810a7b0 con 0x7fbc181092d0 2024-04-25T19:10:20.756 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.755+0000 7fbc15600700 1 -- 172.21.15.45:0/3542139673 <== mon.1 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7fbc0c02a070 con 0x7fbc181092d0 2024-04-25T19:10:20.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.757+0000 7fbc15600700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc000a3db0 0x7fbc000a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:20.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.757+0000 7fbc15600700 1 -- 172.21.15.45:0/3542139673 <== mon.1 v2:172.21.15.79:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8571+0+0 (secure 0 0 0) 0x7fbc0c0d1530 con 0x7fbc181092d0 2024-04-25T19:10:20.758 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.757+0000 7fbc17e00700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc000a3db0 0x7fbc000a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:20.759 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.758+0000 7fbc17e00700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc000a3db0 0x7fbc000a6270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc181a0e00 tx=0x7fbc04009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:20.760 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.759+0000 7fbc15600700 1 -- 172.21.15.45:0/3542139673 <== mon.1 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc0c031080 con 0x7fbc181092d0 2024-04-25T19:10:20.993 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.992+0000 7fbc1f600700 1 -- 172.21.15.45:0/3542139673 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc1802df80 con 0x7fbc181092d0 2024-04-25T19:10:20.995 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.994+0000 7fbc15600700 1 -- 172.21.15.45:0/3542139673 <== mon.1 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+23123 (secure 0 0 0) 0x7fbc0c013030 con 0x7fbc181092d0 2024-04-25T19:10:20.996 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-25T19:10:20.996 INFO:teuthology.orchestra.run.smithi045.stdout:{"epoch":106,"fsid":"ea2833ba-0333-11ef-bc93-c7b262605968","created":"2024-04-25T18:46:09.147410+0000","modified":"2024-04-25T19:10:16.492101+0000","last_up_change":"2024-04-25T18:54:09.903298+0000","last_in_change":"2024-04-25T18:53:49.961093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T18:50:00.259702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T18:55:17.591864+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T18:55:22.070218+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ee46df06-8c1b-43ba-a8d7-c8c7a23e8995","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6803","nonce":4137498586}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6805","nonce":4137498586}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6809","nonce":4137498586}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":4137498586},{"type":"v1","addr":"172.21.15.45:6807","nonce":4137498586}]},"public_addr":"172.21.15.45:6803/4137498586","cluster_addr":"172.21.15.45:6805/4137498586","heartbeat_back_addr":"172.21.15.45:6809/4137498586","heartbeat_front_addr":"172.21.15.45:6807/4137498586","state":["exists","up"]},{"osd":1,"uuid":"550faceb-9f8f-4beb-8dd2-e0c6b4968c5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6800","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6801","nonce":237724307}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6802","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6803","nonce":237724307}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6806","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6807","nonce":237724307}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6804","nonce":237724307},{"type":"v1","addr":"172.21.15.62:6805","nonce":237724307}]},"public_addr":"172.21.15.62:6801/237724307","cluster_addr":"172.21.15.62:6803/237724307","heartbeat_back_addr":"172.21.15.62:6807/237724307","heartbeat_front_addr":"172.21.15.62:6805/237724307","state":["exists","up"]},{"osd":2,"uuid":"30f605cc-9a0c-4410-8d6d-299c2413482b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6801","nonce":2930472364}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6803","nonce":2930472364}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6807","nonce":2930472364}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2930472364},{"type":"v1","addr":"172.21.15.79:6805","nonce":2930472364}]},"public_addr":"172.21.15.79:6801/2930472364","cluster_addr":"172.21.15.79:6803/2930472364","heartbeat_back_addr":"172.21.15.79:6807/2930472364","heartbeat_front_addr":"172.21.15.79:6805/2930472364","state":["exists","up"]},{"osd":3,"uuid":"2cab3d66-da4b-4960-bf80-8d198d0eac8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6811","nonce":3944994121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6813","nonce":3944994121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6817","nonce":3944994121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":3944994121},{"type":"v1","addr":"172.21.15.45:6815","nonce":3944994121}]},"public_addr":"172.21.15.45:6811/3944994121","cluster_addr":"172.21.15.45:6813/3944994121","heartbeat_back_addr":"172.21.15.45:6817/3944994121","heartbeat_front_addr":"172.21.15.45:6815/3944994121","state":["exists","up"]},{"osd":4,"uuid":"ac287cf8-20dd-477f-8877-966a1d4e269b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6808","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6809","nonce":2453949627}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6810","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6811","nonce":2453949627}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6814","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6815","nonce":2453949627}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6812","nonce":2453949627},{"type":"v1","addr":"172.21.15.62:6813","nonce":2453949627}]},"public_addr":"172.21.15.62:6809/2453949627","cluster_addr":"172.21.15.62:6811/2453949627","heartbeat_back_addr":"172.21.15.62:6815/2453949627","heartbeat_front_addr":"172.21.15.62:6813/2453949627","state":["exists","up"]},{"osd":5,"uuid":"c258a3e7-ef41-4ad5-a9b0-689eb2761171","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6809","nonce":188682646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6811","nonce":188682646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6815","nonce":188682646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":188682646},{"type":"v1","addr":"172.21.15.79:6813","nonce":188682646}]},"public_addr":"172.21.15.79:6809/188682646","cluster_addr":"172.21.15.79:6811/188682646","heartbeat_back_addr":"172.21.15.79:6815/188682646","heartbeat_front_addr":"172.21.15.79:6813/188682646","state":["exists","up"]},{"osd":6,"uuid":"9b1c752c-c246-4c8c-adee-e8948e1c2978","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6819","nonce":401127899}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6821","nonce":401127899}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6825","nonce":401127899}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":401127899},{"type":"v1","addr":"172.21.15.45:6823","nonce":401127899}]},"public_addr":"172.21.15.45:6819/401127899","cluster_addr":"172.21.15.45:6821/401127899","heartbeat_back_addr":"172.21.15.45:6825/401127899","heartbeat_front_addr":"172.21.15.45:6823/401127899","state":["exists","up"]},{"osd":7,"uuid":"eb5a094c-22a9-4e39-9014-57094699afec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6816","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6817","nonce":3708428772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6818","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6819","nonce":3708428772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6822","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6823","nonce":3708428772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6820","nonce":3708428772},{"type":"v1","addr":"172.21.15.62:6821","nonce":3708428772}]},"public_addr":"172.21.15.62:6817/3708428772","cluster_addr":"172.21.15.62:6819/3708428772","heartbeat_back_addr":"172.21.15.62:6823/3708428772","heartbeat_front_addr":"172.21.15.62:6821/3708428772","state":["exists","up"]},{"osd":8,"uuid":"b9e1829d-cafc-47cf-ba9b-1dc60e0c6421","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6817","nonce":2481950968}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6819","nonce":2481950968}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6823","nonce":2481950968}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":2481950968},{"type":"v1","addr":"172.21.15.79:6821","nonce":2481950968}]},"public_addr":"172.21.15.79:6817/2481950968","cluster_addr":"172.21.15.79:6819/2481950968","heartbeat_back_addr":"172.21.15.79:6823/2481950968","heartbeat_front_addr":"172.21.15.79:6821/2481950968","state":["exists","up"]},{"osd":9,"uuid":"f783937a-cdd7-4310-a6b4-00cf934db047","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":70,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6827","nonce":2318794689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6829","nonce":2318794689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6833","nonce":2318794689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":2318794689},{"type":"v1","addr":"172.21.15.45:6831","nonce":2318794689}]},"public_addr":"172.21.15.45:6827/2318794689","cluster_addr":"172.21.15.45:6829/2318794689","heartbeat_back_addr":"172.21.15.45:6833/2318794689","heartbeat_front_addr":"172.21.15.45:6831/2318794689","state":["exists","up"]},{"osd":10,"uuid":"bf2abd19-2916-4e72-ace4-4fc3a72ef06c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6824","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6825","nonce":2698516716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6826","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6827","nonce":2698516716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6830","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6831","nonce":2698516716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.62:6828","nonce":2698516716},{"type":"v1","addr":"172.21.15.62:6829","nonce":2698516716}]},"public_addr":"172.21.15.62:6825/2698516716","cluster_addr":"172.21.15.62:6827/2698516716","heartbeat_back_addr":"172.21.15.62:6831/2698516716","heartbeat_front_addr":"172.21.15.62:6829/2698516716","state":["exists","up"]},{"osd":11,"uuid":"197c940c-1050-4bff-b88c-86b6a4da0cf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":84,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6825","nonce":1965921727}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6827","nonce":1965921727}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6831","nonce":1965921727}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":1965921727},{"type":"v1","addr":"172.21.15.79:6829","nonce":1965921727}]},"public_addr":"172.21.15.79:6825/1965921727","cluster_addr":"172.21.15.79:6827/1965921727","heartbeat_back_addr":"172.21.15.79:6831/1965921727","heartbeat_front_addr":"172.21.15.79:6829/1965921727","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:07.329026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:32.511086+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:49:56.791478+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:28.738817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:50:53.580689+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:17.692607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:51:51.357883+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:16.991962+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:52:42.035990+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:15.869920+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:53:42.580069+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T18:54:07.762551+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"3.19","mappings":[{"from":3,"to":8}]},{"pgid":"3.1b","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":4,"to":7}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.45:6801/943541413":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/2599503580":"2024-04-26T18:59:18.746957+0000","172.21.15.45:6800/943541413":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/3590060181":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/4010951391":"2024-04-26T18:59:18.746957+0000","172.21.15.45:0/1117554794":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3367087840":"2024-04-26T18:59:18.746957+0000","172.21.15.45:6800/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/2122448921":"2024-04-26T18:46:42.822772+0000","172.21.15.45:0/1563847530":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/817529633":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/1502518510":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6800/1404432740":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/3065212608":"2024-04-26T18:47:16.094556+0000","172.21.15.45:0/3157265170":"2024-04-26T18:46:42.822772+0000","172.21.15.45:6801/1404432740":"2024-04-26T18:46:42.822772+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T19:10:20.999 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.997+0000 7fbc1f600700 1 -- 172.21.15.45:0/3542139673 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc000a3db0 msgr2=0x7fbc000a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.997+0000 7fbc1f600700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc000a3db0 0x7fbc000a6270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc181a0e00 tx=0x7fbc04009260 comp rx=0 tx=0).stop 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.997+0000 7fbc1f600700 1 -- 172.21.15.45:0/3542139673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc181092d0 msgr2=0x7fbc181a5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.997+0000 7fbc1f600700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc181092d0 0x7fbc181a5570 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c000c00 tx=0x7fbc0c00f950 comp rx=0 tx=0).stop 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.997+0000 7fbc1f600700 1 -- 172.21.15.45:0/3542139673 shutdown_connections 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.997+0000 7fbc1f600700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7fbc000a3db0 0x7fbc000a6270 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.997+0000 7fbc1f600700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbc1810e3b0 0x7fbc1819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.997+0000 7fbc1f600700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc181092d0 0x7fbc181a5570 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.997+0000 7fbc1f600700 1 --2- 172.21.15.45:0/3542139673 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7fbc18109c80 0x7fbc181a5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.997+0000 7fbc1f600700 1 -- 172.21.15.45:0/3542139673 >> 172.21.15.45:0/3542139673 conn(0x7fbc18076210 msgr2=0x7fbc18076e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.998+0000 7fbc1f600700 1 -- 172.21.15.45:0/3542139673 shutdown_connections 2024-04-25T19:10:21.000 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:20.998+0000 7fbc1f600700 1 -- 172.21.15.45:0/3542139673 wait complete. 2024-04-25T19:10:21.386 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:21 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3542139673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:10:21.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:21 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3542139673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:10:21.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:21 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3542139673' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T19:10:21.904 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-25T19:10:22.356 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:10:22.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:22 smithi079 ceph-mon[33002]: pgmap v345: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 413 B/s wr, 7 op/s 2024-04-25T19:10:22.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:22 smithi045 ceph-mon[27776]: pgmap v345: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 413 B/s wr, 7 op/s 2024-04-25T19:10:22.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:22 smithi062 ceph-mon[32844]: pgmap v345: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 413 B/s wr, 7 op/s 2024-04-25T19:10:24.326 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:24 smithi045 ceph-mon[27776]: pgmap v346: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 409 B/s wr, 7 op/s 2024-04-25T19:10:24.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-mon[33002]: pgmap v346: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 409 B/s wr, 7 op/s 2024-04-25T19:10:24.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.498+0000 7f21e1c00700 1 -- 172.21.15.45:0/1530822045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc10f440 msgr2=0x7f21dc111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:24.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.498+0000 7f21e1c00700 1 --2- 172.21.15.45:0/1530822045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc10f440 0x7f21dc111830 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009f90 tx=0x7f21d400b330 comp rx=0 tx=0).stop 2024-04-25T19:10:24.500 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.498+0000 7f21e1c00700 1 -- 172.21.15.45:0/1530822045 shutdown_connections 2024-04-25T19:10:24.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.498+0000 7f21e1c00700 1 --2- 172.21.15.45:0/1530822045 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc074c70 0x7f21dc075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:24.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.498+0000 7f21e1c00700 1 --2- 172.21.15.45:0/1530822045 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc10f440 0x7f21dc111830 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:24.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.498+0000 7f21e1c00700 1 --2- 172.21.15.45:0/1530822045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21dc075590 0x7f21dc10ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:24.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.498+0000 7f21e1c00700 1 -- 172.21.15.45:0/1530822045 >> 172.21.15.45:0/1530822045 conn(0x7f21dc079ba0 msgr2=0x7f21dc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:24.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.499+0000 7f21e1c00700 1 -- 172.21.15.45:0/1530822045 shutdown_connections 2024-04-25T19:10:24.501 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.499+0000 7f21e1c00700 1 -- 172.21.15.45:0/1530822045 wait complete. 2024-04-25T19:10:24.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.500+0000 7f21e1c00700 1 Processor -- start 2024-04-25T19:10:24.502 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.500+0000 7f21e1c00700 1 -- start start 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.500+0000 7f21e1c00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc074c70 0x7f21dc19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.500+0000 7f21e1c00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc075590 0x7f21dc19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.500+0000 7f21e1c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21dc10f440 0x7f21dc19fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.500+0000 7f21e1c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc114e20 con 0x7f21dc10f440 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.500+0000 7f21e1c00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc114ca0 con 0x7f21dc075590 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.500+0000 7f21e1c00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f21dc114fa0 con 0x7f21dc074c70 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21daa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc074c70 0x7f21dc19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21da000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc075590 0x7f21dc19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21da000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc075590 0x7f21dc19b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:35486/0 (socket says 172.21.15.45:35486) 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21daa00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc074c70 0x7f21dc19b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.45:35660/0 (socket says 172.21.15.45:35660) 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21da000700 1 -- 172.21.15.45:0/2103312986 learned_addr learned my addr 172.21.15.45:0/2103312986 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21db400700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21dc10f440 0x7f21dc19fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:24.503 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21da000700 1 -- 172.21.15.45:0/2103312986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc074c70 msgr2=0x7f21dc19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:24.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21da000700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc074c70 0x7f21dc19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:24.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21da000700 1 -- 172.21.15.45:0/2103312986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21dc10f440 msgr2=0x7f21dc19fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:24.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21da000700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21dc10f440 0x7f21dc19fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:24.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21da000700 1 -- 172.21.15.45:0/2103312986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21dc1a02c0 con 0x7f21dc075590 2024-04-25T19:10:24.504 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.501+0000 7f21daa00700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc074c70 0x7f21dc19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:24.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.502+0000 7f21da000700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc075590 0x7f21dc19b7f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f21c800ebf0 tx=0x7f21c800ef00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:24.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.502+0000 7f21d3400700 1 -- 172.21.15.45:0/2103312986 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c800cc90 con 0x7f21dc075590 2024-04-25T19:10:24.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.502+0000 7f21d3400700 1 -- 172.21.15.45:0/2103312986 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21c800ce30 con 0x7f21dc075590 2024-04-25T19:10:24.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.502+0000 7f21e1c00700 1 -- 172.21.15.45:0/2103312986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21dc1a05b0 con 0x7f21dc075590 2024-04-25T19:10:24.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.502+0000 7f21d3400700 1 -- 172.21.15.45:0/2103312986 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c80187e0 con 0x7f21dc075590 2024-04-25T19:10:24.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.502+0000 7f21e1c00700 1 -- 172.21.15.45:0/2103312986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21dc1a7e30 con 0x7f21dc075590 2024-04-25T19:10:24.505 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.503+0000 7f21e1c00700 1 -- 172.21.15.45:0/2103312986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21dc069730 con 0x7f21dc075590 2024-04-25T19:10:24.506 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.505+0000 7f21d3400700 1 -- 172.21.15.45:0/2103312986 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f21c801f020 con 0x7f21dc075590 2024-04-25T19:10:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.507+0000 7f21d3400700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21c00a3db0 0x7f21c00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.507+0000 7f21d3400700 1 -- 172.21.15.45:0/2103312986 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8571+0+0 (secure 0 0 0) 0x7f21c80d0ac0 con 0x7f21dc075590 2024-04-25T19:10:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.507+0000 7f21daa00700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21c00a3db0 0x7f21c00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:24.508 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.508+0000 7f21daa00700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21c00a3db0 0x7f21c00a6270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21cc011490 tx=0x7f21cc009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:24.513 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.512+0000 7f21d3400700 1 -- 172.21.15.45:0/2103312986 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21c8053ff0 con 0x7f21dc075590 2024-04-25T19:10:24.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:24 smithi062 ceph-mon[32844]: pgmap v346: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 409 B/s wr, 7 op/s 2024-04-25T19:10:24.794 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.792+0000 7f21e1c00700 1 -- 172.21.15.45:0/2103312986 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f21dc1a8000 con 0x7f21dc075590 2024-04-25T19:10:24.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.803+0000 7f21d3400700 1 -- 172.21.15.45:0/2103312986 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f21c809d0d0 con 0x7f21dc075590 2024-04-25T19:10:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.806+0000 7f21e1c00700 1 -- 172.21.15.45:0/2103312986 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21c00a3db0 msgr2=0x7f21c00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.806+0000 7f21e1c00700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21c00a3db0 0x7f21c00a6270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21cc011490 tx=0x7f21cc009260 comp rx=0 tx=0).stop 2024-04-25T19:10:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.806+0000 7f21e1c00700 1 -- 172.21.15.45:0/2103312986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc075590 msgr2=0x7f21dc19b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.806+0000 7f21e1c00700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc075590 0x7f21dc19b7f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f21c800ebf0 tx=0x7f21c800ef00 comp rx=0 tx=0).stop 2024-04-25T19:10:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.806+0000 7f21e1c00700 1 -- 172.21.15.45:0/2103312986 shutdown_connections 2024-04-25T19:10:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.806+0000 7f21e1c00700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f21c00a3db0 0x7f21c00a6270 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.806+0000 7f21e1c00700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f21dc10f440 0x7f21dc19fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.806+0000 7f21e1c00700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f21dc074c70 0x7f21dc19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:24.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.806+0000 7f21e1c00700 1 --2- 172.21.15.45:0/2103312986 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f21dc075590 0x7f21dc19b7f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:24.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.807+0000 7f21e1c00700 1 -- 172.21.15.45:0/2103312986 >> 172.21.15.45:0/2103312986 conn(0x7f21dc079ba0 msgr2=0x7f21dc0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:24.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.807+0000 7f21e1c00700 1 -- 172.21.15.45:0/2103312986 shutdown_connections 2024-04-25T19:10:24.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:24.807+0000 7f21e1c00700 1 -- 172.21.15.45:0/2103312986 wait complete. 2024-04-25T19:10:24.810 INFO:teuthology.orchestra.run.smithi045.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T19:10:25.114 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 19:10:24 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b[63904]: ignoring --setuser ceph since I am not root 2024-04-25T19:10:25.114 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 19:10:24 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b[63904]: ignoring --setgroup ceph since I am not root 2024-04-25T19:10:25.114 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 19:10:24 smithi062 ceph-mds[63908]: -- 172.21.15.62:0/84273148 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b494d35a40 con 0x55b494d08800 2024-04-25T19:10:25.114 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 19:10:24 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b[63904]: starting mds.b at 2024-04-25T19:10:25.115 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 19:10:24 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h[63594]: ignoring --setuser ceph since I am not root 2024-04-25T19:10:25.115 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 19:10:24 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h[63594]: ignoring --setgroup ceph since I am not root 2024-04-25T19:10:25.115 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 19:10:24 smithi062 ceph-mds[63598]: -- 172.21.15.62:0/3722754405 <== mon.2 v2:172.21.15.62:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558b4935a40 con 0x5558b4908c00 2024-04-25T19:10:25.115 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 19:10:24 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h[63594]: starting mds.h at 2024-04-25T19:10:25.136 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a[27771]: 2024-04-25T19:10:24.795+0000 7f5d59200700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T19:10:25.137 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c[63996]: ignoring --setuser ceph since I am not root 2024-04-25T19:10:25.137 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c[63996]: ignoring --setgroup ceph since I am not root 2024-04-25T19:10:25.137 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-mds[64000]: -- 172.21.15.79:0/3400150519 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56300aa7fa40 con 0x56300aa52800 2024-04-25T19:10:25.137 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c[63996]: starting mds.c at 2024-04-25T19:10:25.137 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f[63686]: ignoring --setuser ceph since I am not root 2024-04-25T19:10:25.137 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f[63686]: ignoring --setgroup ceph since I am not root 2024-04-25T19:10:25.137 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-mds[63690]: -- 172.21.15.79:0/4209248930 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56499314da40 con 0x564993120c00 2024-04-25T19:10:25.137 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f[63686]: starting mds.f at 2024-04-25T19:10:25.137 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i[64306]: ignoring --setuser ceph since I am not root 2024-04-25T19:10:25.138 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i[64306]: ignoring --setgroup ceph since I am not root 2024-04-25T19:10:25.138 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-mds[64310]: -- 172.21.15.79:0/1144594286 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5558a3bd9a40 con 0x5558a3bad000 2024-04-25T19:10:25.138 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:10:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i[64306]: starting mds.i at 2024-04-25T19:10:25.138 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:10:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a[67075]: ignoring --setuser ceph since I am not root 2024-04-25T19:10:25.138 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:10:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a[67075]: ignoring --setgroup ceph since I am not root 2024-04-25T19:10:25.138 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:10:24 smithi045 ceph-mds[67081]: -- 172.21.15.45:0/3544064282 <== mon.1 v2:172.21.15.79:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561fae60da40 con 0x561fae5e0800 2024-04-25T19:10:25.138 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:10:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a[67075]: starting mds.a at 2024-04-25T19:10:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:25 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2103312986' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:10:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:25 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:10:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:25 smithi079 ceph-mon[33002]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T19:10:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:25 smithi079 ceph-mon[33002]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T19:10:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:25 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T19:10:25.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:25 smithi079 ceph-mon[33002]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T19:10:25.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:25 smithi079 ceph-mon[33002]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T19:10:25.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:25 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T19:10:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:25 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2103312986' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:10:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:25 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:10:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:25 smithi045 ceph-mon[27776]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T19:10:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:25 smithi045 ceph-mon[27776]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T19:10:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:25 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T19:10:25.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:25 smithi045 ceph-mon[27776]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T19:10:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:25 smithi045 ceph-mon[27776]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T19:10:25.465 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:25 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T19:10:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:25 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2103312986' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:10:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:25 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T19:10:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:25 smithi062 ceph-mon[32844]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T19:10:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:25 smithi062 ceph-mon[32844]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T19:10:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:25 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T19:10:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:25 smithi062 ceph-mon[32844]: osdmap e107: 12 total, 12 up, 12 in 2024-04-25T19:10:25.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:25 smithi062 ceph-mon[32844]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T19:10:25.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:25 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T19:10:25.658 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T19:10:26.130 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:10:26.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: pgmap v348: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 119 B/s wr, 7 op/s 2024-04-25T19:10:26.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.79:6834/1851267717,v1:172.21.15.79:6835/1851267717] up:boot 2024-04-25T19:10:26.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.79:6836/3221265613,v1:172.21.15.79:6837/3221265613] up:boot 2024-04-25T19:10:26.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.79:6838/185810195,v1:172.21.15.79:6839/185810195] up:boot 2024-04-25T19:10:26.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.62:6836/3383876648,v1:172.21.15.62:6837/3383876648] up:boot 2024-04-25T19:10:26.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.62:6838/1365196329,v1:172.21.15.62:6839/1365196329] up:boot 2024-04-25T19:10:26.429 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: mds.? [v2:172.21.15.45:6840/743972667,v1:172.21.15.45:6841/743972667] up:boot 2024-04-25T19:10:26.430 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T19:10:26.430 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T19:10:26.430 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T19:10:26.430 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:10:26.430 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T19:10:26.430 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T19:10:26.430 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:26 smithi045 ceph-mon[27776]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T19:10:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: pgmap v348: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 119 B/s wr, 7 op/s 2024-04-25T19:10:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.79:6834/1851267717,v1:172.21.15.79:6835/1851267717] up:boot 2024-04-25T19:10:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.79:6836/3221265613,v1:172.21.15.79:6837/3221265613] up:boot 2024-04-25T19:10:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.79:6838/185810195,v1:172.21.15.79:6839/185810195] up:boot 2024-04-25T19:10:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.62:6836/3383876648,v1:172.21.15.62:6837/3383876648] up:boot 2024-04-25T19:10:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.62:6838/1365196329,v1:172.21.15.62:6839/1365196329] up:boot 2024-04-25T19:10:26.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: mds.? [v2:172.21.15.45:6840/743972667,v1:172.21.15.45:6841/743972667] up:boot 2024-04-25T19:10:26.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T19:10:26.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T19:10:26.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T19:10:26.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:10:26.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T19:10:26.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T19:10:26.435 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:26 smithi079 ceph-mon[33002]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T19:10:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: pgmap v348: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 119 B/s wr, 7 op/s 2024-04-25T19:10:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.79:6834/1851267717,v1:172.21.15.79:6835/1851267717] up:boot 2024-04-25T19:10:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.79:6836/3221265613,v1:172.21.15.79:6837/3221265613] up:boot 2024-04-25T19:10:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.79:6838/185810195,v1:172.21.15.79:6839/185810195] up:boot 2024-04-25T19:10:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.62:6836/3383876648,v1:172.21.15.62:6837/3383876648] up:boot 2024-04-25T19:10:26.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.62:6838/1365196329,v1:172.21.15.62:6839/1365196329] up:boot 2024-04-25T19:10:26.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: mds.? [v2:172.21.15.45:6840/743972667,v1:172.21.15.45:6841/743972667] up:boot 2024-04-25T19:10:26.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T19:10:26.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T19:10:26.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T19:10:26.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T19:10:26.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T19:10:26.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T19:10:26.615 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:26 smithi062 ceph-mon[32844]: from='mgr.25049 172.21.15.45:0/2667355375' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T19:10:28.228 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:28 smithi045 ceph-mon[27776]: pgmap v349: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T19:10:28.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:28 smithi079 ceph-mon[33002]: pgmap v349: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T19:10:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.523+0000 7f56d8800700 1 -- 172.21.15.45:0/906757494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0113900 msgr2=0x7f56d0115cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.526 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.523+0000 7f56d8800700 1 --2- 172.21.15.45:0/906757494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0113900 0x7f56d0115cf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009f90 tx=0x7f56cc00b330 comp rx=0 tx=0).stop 2024-04-25T19:10:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.523+0000 7f56d8800700 1 -- 172.21.15.45:0/906757494 shutdown_connections 2024-04-25T19:10:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.523+0000 7f56d8800700 1 --2- 172.21.15.45:0/906757494 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0113900 0x7f56d0115cf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.523+0000 7f56d8800700 1 --2- 172.21.15.45:0/906757494 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d0076f70 0x7f56d0077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.523+0000 7f56d8800700 1 --2- 172.21.15.45:0/906757494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d0077890 0x7f56d01133c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.523+0000 7f56d8800700 1 -- 172.21.15.45:0/906757494 >> 172.21.15.45:0/906757494 conn(0x7f56d0100750 msgr2=0x7f56d0102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.524+0000 7f56d8800700 1 -- 172.21.15.45:0/906757494 shutdown_connections 2024-04-25T19:10:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.524+0000 7f56d8800700 1 -- 172.21.15.45:0/906757494 wait complete. 2024-04-25T19:10:28.527 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.525+0000 7f56d8800700 1 Processor -- start 2024-04-25T19:10:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.526+0000 7f56d8800700 1 -- start start 2024-04-25T19:10:28.528 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.526+0000 7f56d8800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d0076f70 0x7f56d019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.526+0000 7f56d8800700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d0077890 0x7f56d019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d8800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0113900 0x7f56d01a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d8800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0119280 con 0x7f56d0076f70 2024-04-25T19:10:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d8800700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0119100 con 0x7f56d0113900 2024-04-25T19:10:28.529 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d8800700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f56d0119400 con 0x7f56d0077890 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d0076f70 0x7f56d019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d6800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0113900 0x7f56d01a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d6800700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0113900 0x7f56d01a3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:55666/0 (socket says 172.21.15.45:55666) 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d0076f70 0x7f56d019f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:49548/0 (socket says 172.21.15.45:49548) 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d6800700 1 -- 172.21.15.45:0/4022234674 learned_addr learned my addr 172.21.15.45:0/4022234674 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d5400700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d0077890 0x7f56d019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d6800700 1 -- 172.21.15.45:0/4022234674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d0077890 msgr2=0x7f56d019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d6800700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d0077890 0x7f56d019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d6800700 1 -- 172.21.15.45:0/4022234674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d0076f70 msgr2=0x7f56d019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d6800700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d0076f70 0x7f56d019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:28.530 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.527+0000 7f56d6800700 1 -- 172.21.15.45:0/4022234674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d01a46e0 con 0x7f56d0113900 2024-04-25T19:10:28.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.528+0000 7f56d6800700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0113900 0x7f56d01a3fa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00b6c0 tx=0x7f56cc00cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:28.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.528+0000 7f56caa00700 1 -- 172.21.15.45:0/4022234674 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc021070 con 0x7f56d0113900 2024-04-25T19:10:28.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.528+0000 7f56caa00700 1 -- 172.21.15.45:0/4022234674 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56cc00ce50 con 0x7f56d0113900 2024-04-25T19:10:28.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.528+0000 7f56caa00700 1 -- 172.21.15.45:0/4022234674 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56cc005040 con 0x7f56d0113900 2024-04-25T19:10:28.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.528+0000 7f56d8800700 1 -- 172.21.15.45:0/4022234674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d01a4970 con 0x7f56d0113900 2024-04-25T19:10:28.531 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.528+0000 7f56d8800700 1 -- 172.21.15.45:0/4022234674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d01ac250 con 0x7f56d0113900 2024-04-25T19:10:28.532 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.530+0000 7f56d8800700 1 -- 172.21.15.45:0/4022234674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d0069730 con 0x7f56d0113900 2024-04-25T19:10:28.533 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.531+0000 7f56caa00700 1 -- 172.21.15.45:0/4022234674 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f56cc024070 con 0x7f56d0113900 2024-04-25T19:10:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.533+0000 7f56caa00700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56b80a3e30 0x7f56b80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.533+0000 7f56caa00700 1 -- 172.21.15.45:0/4022234674 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8785+0+0 (secure 0 0 0) 0x7f56cc00f020 con 0x7f56d0113900 2024-04-25T19:10:28.535 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.533+0000 7f56d5e00700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56b80a3e30 0x7f56b80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:28.536 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.534+0000 7f56d5e00700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56b80a3e30 0x7f56b80a62f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009c60 tx=0x7f56c4009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:28.539 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.537+0000 7f56caa00700 1 -- 172.21.15.45:0/4022234674 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56cc02b020 con 0x7f56d0113900 2024-04-25T19:10:28.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:28 smithi062 ceph-mon[32844]: pgmap v349: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T19:10:28.820 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:28.818+0000 7f56d8800700 1 -- 172.21.15.45:0/4022234674 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f56d01a0a30 con 0x7f56d0113900 2024-04-25T19:10:29.165 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.161+0000 7f56caa00700 1 -- 172.21.15.45:0/4022234674 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) ==== 104+0+0 (secure 0 0 0) 0x7f56cc09e160 con 0x7f56d0113900 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 -- 172.21.15.45:0/4022234674 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56b80a3e30 msgr2=0x7f56b80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56b80a3e30 0x7f56b80a62f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009c60 tx=0x7f56c4009260 comp rx=0 tx=0).stop 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 -- 172.21.15.45:0/4022234674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0113900 msgr2=0x7f56d01a3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0113900 0x7f56d01a3fa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f56cc00b6c0 tx=0x7f56cc00cc30 comp rx=0 tx=0).stop 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 -- 172.21.15.45:0/4022234674 shutdown_connections 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f56b80a3e30 0x7f56b80a62f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f56d0113900 0x7f56d01a3fa0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f56d0076f70 0x7f56d019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 --2- 172.21.15.45:0/4022234674 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f56d0077890 0x7f56d019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 -- 172.21.15.45:0/4022234674 >> 172.21.15.45:0/4022234674 conn(0x7f56d0100750 msgr2=0x7f56d0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:29.168 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.165+0000 7f56d8800700 1 -- 172.21.15.45:0/4022234674 shutdown_connections 2024-04-25T19:10:29.169 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:29.166+0000 7f56d8800700 1 -- 172.21.15.45:0/4022234674 wait complete. 2024-04-25T19:10:29.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:29 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/4022234674' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:10:29.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:29 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:10:29.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:29 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/4022234674' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:10:29.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:29 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:10:29.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:29 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/4022234674' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:10:29.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:29 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T19:10:29.965 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T19:10:30.421 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:10:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:30 smithi079 ceph-mon[33002]: pgmap v350: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T19:10:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:30 smithi079 ceph-mon[33002]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T19:10:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:30 smithi079 ceph-mon[33002]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T19:10:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:30 smithi079 ceph-mon[33002]: Cluster is now healthy 2024-04-25T19:10:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:30 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T19:10:30.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:30 smithi079 ceph-mon[33002]: fsmap 12 up:standby 2024-04-25T19:10:30.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:30 smithi045 ceph-mon[27776]: pgmap v350: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T19:10:30.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:30 smithi045 ceph-mon[27776]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T19:10:30.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:30 smithi045 ceph-mon[27776]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T19:10:30.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:30 smithi045 ceph-mon[27776]: Cluster is now healthy 2024-04-25T19:10:30.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:30 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T19:10:30.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:30 smithi045 ceph-mon[27776]: fsmap 12 up:standby 2024-04-25T19:10:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:30 smithi062 ceph-mon[32844]: pgmap v350: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T19:10:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:30 smithi062 ceph-mon[32844]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T19:10:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:30 smithi062 ceph-mon[32844]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T19:10:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:30 smithi062 ceph-mon[32844]: Cluster is now healthy 2024-04-25T19:10:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:30 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T19:10:30.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:30 smithi062 ceph-mon[32844]: fsmap 12 up:standby 2024-04-25T19:10:32.379 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:32 smithi045 ceph-mon[27776]: pgmap v351: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-25T19:10:32.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:32 smithi079 ceph-mon[33002]: pgmap v351: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-25T19:10:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.557+0000 7f46f6000700 1 -- 172.21.15.45:0/3472994748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46f0108a50 msgr2=0x7f46f0108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.557+0000 7f46f6000700 1 --2- 172.21.15.45:0/3472994748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46f0108a50 0x7f46f0108e30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f46e000b0a0 tx=0x7f46e000b3b0 comp rx=0 tx=0).stop 2024-04-25T19:10:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.557+0000 7f46f6000700 1 -- 172.21.15.45:0/3472994748 shutdown_connections 2024-04-25T19:10:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.557+0000 7f46f6000700 1 --2- 172.21.15.45:0/3472994748 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f0103450 0x7f46f010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:32.560 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.557+0000 7f46f6000700 1 --2- 172.21.15.45:0/3472994748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46f0108a50 0x7f46f0108e30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.557+0000 7f46f6000700 1 --2- 172.21.15.45:0/3472994748 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46f0102a90 0x7f46f0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.557+0000 7f46f6000700 1 -- 172.21.15.45:0/3472994748 >> 172.21.15.45:0/3472994748 conn(0x7f46f0079ba0 msgr2=0x7f46f0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.558+0000 7f46f6000700 1 -- 172.21.15.45:0/3472994748 shutdown_connections 2024-04-25T19:10:32.561 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.558+0000 7f46f6000700 1 -- 172.21.15.45:0/3472994748 wait complete. 2024-04-25T19:10:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.559+0000 7f46f6000700 1 Processor -- start 2024-04-25T19:10:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.559+0000 7f46f6000700 1 -- start start 2024-04-25T19:10:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.559+0000 7f46f6000700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46f0102a90 0x7f46f0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.559+0000 7f46f6000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46f0103450 0x7f46f01996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:32.562 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.559+0000 7f46f6000700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f0108a50 0x7f46f019da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.559+0000 7f46f6000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f46f0112c30 con 0x7f46f0103450 2024-04-25T19:10:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.559+0000 7f46f6000700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7f46f0112ab0 con 0x7f46f0102a90 2024-04-25T19:10:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.559+0000 7f46f6000700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46f0112db0 con 0x7f46f0108a50 2024-04-25T19:10:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.559+0000 7f46ef400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46f0102a90 0x7f46f0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.559+0000 7f46e6a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46f0103450 0x7f46f01996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.560+0000 7f46ef400700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46f0102a90 0x7f46f0199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:55682/0 (socket says 172.21.15.45:55682) 2024-04-25T19:10:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.560+0000 7f46ef400700 1 -- 172.21.15.45:0/2245679413 learned_addr learned my addr 172.21.15.45:0/2245679413 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:10:32.563 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.560+0000 7f46efe00700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f0108a50 0x7f46f019da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:32.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.560+0000 7f46ef400700 1 -- 172.21.15.45:0/2245679413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f0108a50 msgr2=0x7f46f019da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:32.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.560+0000 7f46ef400700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f0108a50 0x7f46f019da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:32.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.560+0000 7f46ef400700 1 -- 172.21.15.45:0/2245679413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46f0103450 msgr2=0x7f46f01996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:32.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.560+0000 7f46ef400700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46f0103450 0x7f46f01996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:32.564 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.560+0000 7f46ef400700 1 -- 172.21.15.45:0/2245679413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f019e170 con 0x7f46f0102a90 2024-04-25T19:10:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.560+0000 7f46e6a00700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46f0103450 0x7f46f01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.560+0000 7f46ef400700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46f0102a90 0x7f46f0199160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f46e000d010 tx=0x7f46e0004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.561+0000 7f46ecc00700 1 -- 172.21.15.45:0/2245679413 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0021070 con 0x7f46f0102a90 2024-04-25T19:10:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.561+0000 7f46f6000700 1 -- 172.21.15.45:0/2245679413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f019e400 con 0x7f46f0102a90 2024-04-25T19:10:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.561+0000 7f46ecc00700 1 -- 172.21.15.45:0/2245679413 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e0007e30 con 0x7f46f0102a90 2024-04-25T19:10:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.561+0000 7f46ecc00700 1 -- 172.21.15.45:0/2245679413 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0004640 con 0x7f46f0102a90 2024-04-25T19:10:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.561+0000 7f46f6000700 1 -- 172.21.15.45:0/2245679413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f01a5ce0 con 0x7f46f0102a90 2024-04-25T19:10:32.565 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.562+0000 7f46f6000700 1 -- 172.21.15.45:0/2245679413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46f0069730 con 0x7f46f0102a90 2024-04-25T19:10:32.566 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.564+0000 7f46ecc00700 1 -- 172.21.15.45:0/2245679413 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7f46e0007980 con 0x7f46f0102a90 2024-04-25T19:10:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.565+0000 7f46ecc00700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46d00a3db0 0x7f46d00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.566+0000 7f46ecc00700 1 -- 172.21.15.45:0/2245679413 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8785+0+0 (secure 0 0 0) 0x7f46e000f020 con 0x7f46f0102a90 2024-04-25T19:10:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.566+0000 7f46e6a00700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46d00a3db0 0x7f46d00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:32.568 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.566+0000 7f46e6a00700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46d00a3db0 0x7f46d00a6270 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009680 tx=0x7f46d8009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:32.570 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.568+0000 7f46ecc00700 1 -- 172.21.15.45:0/2245679413 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46e002a020 con 0x7f46f0102a90 2024-04-25T19:10:32.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:32 smithi062 ceph-mon[32844]: pgmap v351: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-25T19:10:32.811 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:32.809+0000 7f46f6000700 1 -- 172.21.15.45:0/2245679413 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46f002df80 con 0x7f46f0102a90 2024-04-25T19:10:33.171 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.168+0000 7f46ecc00700 1 -- 172.21.15.45:0/2245679413 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7f46e0004000 con 0x7f46f0102a90 2024-04-25T19:10:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.174+0000 7f46f6000700 1 -- 172.21.15.45:0/2245679413 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46d00a3db0 msgr2=0x7f46d00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.174+0000 7f46f6000700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46d00a3db0 0x7f46d00a6270 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009680 tx=0x7f46d8009260 comp rx=0 tx=0).stop 2024-04-25T19:10:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.174+0000 7f46f6000700 1 -- 172.21.15.45:0/2245679413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46f0102a90 msgr2=0x7f46f0199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.174+0000 7f46f6000700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46f0102a90 0x7f46f0199160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f46e000d010 tx=0x7f46e0004d00 comp rx=0 tx=0).stop 2024-04-25T19:10:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.174+0000 7f46f6000700 1 -- 172.21.15.45:0/2245679413 shutdown_connections 2024-04-25T19:10:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.174+0000 7f46f6000700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7f46d00a3db0 0x7f46d00a6270 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.174+0000 7f46f6000700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7f46f0102a90 0x7f46f0199160 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f46e000d010 tx=0x7f46e0004d00 comp rx=0 tx=0).stop 2024-04-25T19:10:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.174+0000 7f46f6000700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46f0108a50 0x7f46f019da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.174+0000 7f46f6000700 1 --2- 172.21.15.45:0/2245679413 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f46f0103450 0x7f46f01996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:33.177 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.174+0000 7f46f6000700 1 -- 172.21.15.45:0/2245679413 >> 172.21.15.45:0/2245679413 conn(0x7f46f0079ba0 msgr2=0x7f46f01018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.175+0000 7f46f6000700 1 -- 172.21.15.45:0/2245679413 shutdown_connections 2024-04-25T19:10:33.178 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:33.175+0000 7f46f6000700 1 -- 172.21.15.45:0/2245679413 wait complete. 2024-04-25T19:10:33.179 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_metadata' removed 2024-04-25T19:10:33.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:33 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/2245679413' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:33.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:33 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:33.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:33 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/2245679413' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:33.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:33 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:33.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:33 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/2245679413' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:33.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:33 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:34.024 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid ea2833ba-0333-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T19:10:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:34 smithi079 ceph-mon[33002]: pgmap v352: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-25T19:10:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:34 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:10:34.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:34 smithi079 ceph-mon[33002]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T19:10:34.463 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:34 smithi045 ceph-mon[27776]: pgmap v352: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-25T19:10:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:34 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:10:34.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:34 smithi045 ceph-mon[27776]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T19:10:34.493 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T19:10:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:34 smithi062 ceph-mon[32844]: pgmap v352: 97 pgs: 97 active+clean; 88 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-25T19:10:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:34 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:10:34.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:34 smithi062 ceph-mon[32844]: osdmap e108: 12 total, 12 up, 12 in 2024-04-25T19:10:36.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:36 smithi079 ceph-mon[33002]: pgmap v354: 65 pgs: 65 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:10:36.451 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:36 smithi045 ceph-mon[27776]: pgmap v354: 65 pgs: 65 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:10:36.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:36 smithi062 ceph-mon[32844]: pgmap v354: 65 pgs: 65 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:10:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.797+0000 7feb5ac00700 1 -- 172.21.15.45:0/2649878625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb54074c70 msgr2=0x7feb54075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.797+0000 7feb5ac00700 1 --2- 172.21.15.45:0/2649878625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb54074c70 0x7feb54075050 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feb44009990 tx=0x7feb44009ca0 comp rx=0 tx=0).stop 2024-04-25T19:10:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.797+0000 7feb5ac00700 1 -- 172.21.15.45:0/2649878625 shutdown_connections 2024-04-25T19:10:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.797+0000 7feb5ac00700 1 --2- 172.21.15.45:0/2649878625 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb54074c70 0x7feb54075050 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.797+0000 7feb5ac00700 1 --2- 172.21.15.45:0/2649878625 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb5410f4a0 0x7feb54111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:36.800 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.797+0000 7feb5ac00700 1 --2- 172.21.15.45:0/2649878625 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb54075590 0x7feb5410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:36.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.797+0000 7feb5ac00700 1 -- 172.21.15.45:0/2649878625 >> 172.21.15.45:0/2649878625 conn(0x7feb54079ba0 msgr2=0x7feb54079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:36.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.797+0000 7feb5ac00700 1 -- 172.21.15.45:0/2649878625 shutdown_connections 2024-04-25T19:10:36.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.797+0000 7feb5ac00700 1 -- 172.21.15.45:0/2649878625 wait complete. 2024-04-25T19:10:36.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.799+0000 7feb5ac00700 1 Processor -- start 2024-04-25T19:10:36.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.799+0000 7feb5ac00700 1 -- start start 2024-04-25T19:10:36.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb5ac00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb54074c70 0x7feb5419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:36.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb5ac00700 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb54075590 0x7feb5419b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:36.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb5ac00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb5410f4a0 0x7feb5419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:36.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb5ac00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7feb54114ed0 con 0x7feb5410f4a0 2024-04-25T19:10:36.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb5ac00700 1 -- --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_getmap magic: 0 -- 0x7feb54114d50 con 0x7feb54074c70 2024-04-25T19:10:36.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb5ac00700 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feb54115050 con 0x7feb54075590 2024-04-25T19:10:36.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb53e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb54074c70 0x7feb5419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:36.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb53e00700 1 --2- >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb54074c70 0x7feb5419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.62:3300/0 says I am v2:172.21.15.45:40940/0 (socket says 172.21.15.45:40940) 2024-04-25T19:10:36.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb53e00700 1 -- 172.21.15.45:0/3575138101 learned_addr learned my addr 172.21.15.45:0/3575138101 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-25T19:10:36.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb53400700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb54075590 0x7feb5419b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:36.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.800+0000 7feb58c00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb5410f4a0 0x7feb5419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb53e00700 1 -- 172.21.15.45:0/3575138101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb54075590 msgr2=0x7feb5419b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb53e00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb54075590 0x7feb5419b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb53e00700 1 -- 172.21.15.45:0/3575138101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb5410f4a0 msgr2=0x7feb5419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb53e00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb5410f4a0 0x7feb5419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb53e00700 1 -- 172.21.15.45:0/3575138101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb541a0320 con 0x7feb54074c70 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb53400700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb54075590 0x7feb5419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb53e00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb54074c70 0x7feb5419b360 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feb44000c00 tx=0x7feb440057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb50c00700 1 -- 172.21.15.45:0/3575138101 <== mon.2 v2:172.21.15.62:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44027070 con 0x7feb54074c70 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb50c00700 1 -- 172.21.15.45:0/3575138101 <== mon.2 v2:172.21.15.62:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb44022470 con 0x7feb54074c70 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb50c00700 1 -- 172.21.15.45:0/3575138101 <== mon.2 v2:172.21.15.62:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb44017650 con 0x7feb54074c70 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.801+0000 7feb5ac00700 1 -- 172.21.15.45:0/3575138101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb541a05b0 con 0x7feb54074c70 2024-04-25T19:10:36.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.802+0000 7feb5ac00700 1 -- 172.21.15.45:0/3575138101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb541a7e90 con 0x7feb54074c70 2024-04-25T19:10:36.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.803+0000 7feb5ac00700 1 -- 172.21.15.45:0/3575138101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb54076360 con 0x7feb54074c70 2024-04-25T19:10:36.807 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.805+0000 7feb50c00700 1 -- 172.21.15.45:0/3575138101 <== mon.2 v2:172.21.15.62:3300/0 4 ==== mgrmap(e 27) ==== 138777+0+0 (secure 0 0 0) 0x7feb4402a070 con 0x7feb54074c70 2024-04-25T19:10:36.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.806+0000 7feb50c00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7feb380a3db0 0x7feb380a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T19:10:36.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.806+0000 7feb50c00700 1 -- 172.21.15.45:0/3575138101 <== mon.2 v2:172.21.15.62:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8289+0+0 (secure 0 0 0) 0x7feb440d13e0 con 0x7feb54074c70 2024-04-25T19:10:36.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.806+0000 7feb53400700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7feb380a3db0 0x7feb380a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T19:10:36.809 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.807+0000 7feb53400700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7feb380a3db0 0x7feb380a6270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feb541110f0 tx=0x7feb40009260 comp rx=0 tx=0).ready entity=mgr.25049 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T19:10:36.813 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:36.811+0000 7feb50c00700 1 -- 172.21.15.45:0/3575138101 <== mon.2 v2:172.21.15.62:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb44013070 con 0x7feb54074c70 2024-04-25T19:10:37.045 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.043+0000 7feb5ac00700 1 -- 172.21.15.45:0/3575138101 --> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7feb5402df80 con 0x7feb54074c70 2024-04-25T19:10:37.194 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.191+0000 7feb50c00700 1 -- 172.21.15.45:0/3575138101 <== mon.2 v2:172.21.15.62:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7feb4409db90 con 0x7feb54074c70 2024-04-25T19:10:37.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.198+0000 7feb5ac00700 1 -- 172.21.15.45:0/3575138101 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7feb380a3db0 msgr2=0x7feb380a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:37.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.198+0000 7feb5ac00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7feb380a3db0 0x7feb380a6270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feb541110f0 tx=0x7feb40009260 comp rx=0 tx=0).stop 2024-04-25T19:10:37.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.198+0000 7feb5ac00700 1 -- 172.21.15.45:0/3575138101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb54074c70 msgr2=0x7feb5419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T19:10:37.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.198+0000 7feb5ac00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb54074c70 0x7feb5419b360 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7feb44000c00 tx=0x7feb440057c0 comp rx=0 tx=0).stop 2024-04-25T19:10:37.202 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.200+0000 7feb5ac00700 1 -- 172.21.15.45:0/3575138101 shutdown_connections 2024-04-25T19:10:37.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.200+0000 7feb5ac00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.45:6800/1345298339,v1:172.21.15.45:6801/1345298339] conn(0x7feb380a3db0 0x7feb380a6270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:37.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.200+0000 7feb5ac00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.62:3300/0,v1:172.21.15.62:6789/0] conn(0x7feb54074c70 0x7feb5419b360 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:37.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.200+0000 7feb5ac00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7feb5410f4a0 0x7feb5419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:37.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.200+0000 7feb5ac00700 1 --2- 172.21.15.45:0/3575138101 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb54075590 0x7feb5419b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T19:10:37.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.200+0000 7feb5ac00700 1 -- 172.21.15.45:0/3575138101 >> 172.21.15.45:0/3575138101 conn(0x7feb54079ba0 msgr2=0x7feb540ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T19:10:37.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.200+0000 7feb5ac00700 1 -- 172.21.15.45:0/3575138101 shutdown_connections 2024-04-25T19:10:37.203 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-25T19:10:37.201+0000 7feb5ac00700 1 -- 172.21.15.45:0/3575138101 wait complete. 2024-04-25T19:10:37.206 INFO:teuthology.orchestra.run.smithi045.stderr:pool 'cephfs_data' removed 2024-04-25T19:10:37.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:37 smithi079 ceph-mon[33002]: from='client.? 172.21.15.45:0/3575138101' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:37.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:37 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:37.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:37 smithi045 ceph-mon[27776]: from='client.? 172.21.15.45:0/3575138101' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:37.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:37 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:37 smithi062 ceph-mon[32844]: from='client.? 172.21.15.45:0/3575138101' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:37.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:37 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T19:10:38.037 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:10:38.135 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:10:38.165 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:10:38.194 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T19:10:38.194 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T19:10:38.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:38 smithi045 ceph-mon[27776]: pgmap v355: 65 pgs: 65 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:10:38.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:38 smithi045 ceph-mon[27776]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:10:38.213 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:38 smithi045 ceph-mon[27776]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T19:10:38.253 DEBUG:teuthology.orchestra.run.smithi062:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T19:10:38.269 DEBUG:teuthology.orchestra.run.smithi079:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T19:10:38.284 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T19:10:38.285 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T19:10:38.285 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.a 2024-04-25T19:10:38.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:38 smithi062 ceph-mon[32844]: pgmap v355: 65 pgs: 65 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:10:38.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:38 smithi062 ceph-mon[32844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:10:38.614 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:38 smithi062 ceph-mon[32844]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T19:10:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:38 smithi079 ceph-mon[33002]: pgmap v355: 65 pgs: 65 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T19:10:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:38 smithi079 ceph-mon[33002]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T19:10:38.684 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:38 smithi079 ceph-mon[33002]: osdmap e109: 12 total, 12 up, 12 in 2024-04-25T19:10:38.714 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:38 smithi045 systemd[1]: Stopping Ceph mon.a for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:10:39.195 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:38 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a[27771]: 2024-04-25T19:10:38.864+0000 7f5d60000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:10:39.195 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:38 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a[27771]: 2024-04-25T19:10:38.864+0000 7f5d60000700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T19:10:39.195 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:38 smithi045 podman[112249]: 2024-04-25 19:10:38.950894814 +0000 UTC m=+0.240558340 container died c3ffc389232cce19ad2773b68cccb69a8a68ecdc6b27de49c81eccefb3ad6d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:10:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:39 smithi045 podman[112249]: 2024-04-25 19:10:39.381502757 +0000 UTC m=+0.671166308 container cleanup c3ffc389232cce19ad2773b68cccb69a8a68ecdc6b27de49c81eccefb3ad6d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T19:10:39.464 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:39 smithi045 bash[112249]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a 2024-04-25T19:10:39.920 INFO:journalctl@ceph.mon.a.smithi045.stdout:Apr 25 19:10:39 smithi045 podman[112262]: 2024-04-25 19:10:39.657279897 +0000 UTC m=+0.693118574 container remove c3ffc389232cce19ad2773b68cccb69a8a68ecdc6b27de49c81eccefb3ad6d10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:10:40.056 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T19:10:40.171 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:10:40.171 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T19:10:40.172 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T19:10:40.172 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.b 2024-04-25T19:10:40.501 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:40 smithi062 systemd[1]: Stopping Ceph mon.b for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:10:40.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-b[32840]: 2024-04-25T19:10:40.499+0000 7fa1f7e53700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:10:40.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-b[32840]: 2024-04-25T19:10:40.499+0000 7fa1f7e53700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T19:10:40.864 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:40 smithi062 podman[69992]: 2024-04-25 19:10:40.623390036 +0000 UTC m=+0.242817029 container died 24591487aabe27feffa0457c21ec120500e1ab6a92fd2de66e95b9774a9599de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T19:10:41.221 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:40 smithi062 podman[69992]: 2024-04-25 19:10:40.985938172 +0000 UTC m=+0.605365170 container cleanup 24591487aabe27feffa0457c21ec120500e1ab6a92fd2de66e95b9774a9599de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-b, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T19:10:41.221 INFO:journalctl@ceph.mon.b.smithi062.stdout:Apr 25 19:10:40 smithi062 bash[69992]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-b 2024-04-25T19:10:41.367 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T19:10:41.415 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:10:41.416 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T19:10:41.416 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T19:10:41.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.c 2024-04-25T19:10:41.710 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:41 smithi079 systemd[1]: Stopping Ceph mon.c for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:10:41.980 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-c[32997]: 2024-04-25T19:10:41.708+0000 7f68ac12b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:10:41.980 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-c[32997]: 2024-04-25T19:10:41.708+0000 7f68ac12b700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T19:10:41.980 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:41 smithi079 podman[70869]: 2024-04-25 19:10:41.780492545 +0000 UTC m=+0.175550569 container died 3fda186e9518c953bf91ef19710189f2fc7e6f6bb6559c71c189a5d877a770a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T19:10:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:42 smithi079 podman[70869]: 2024-04-25 19:10:42.192959885 +0000 UTC m=+0.588017906 container cleanup 3fda186e9518c953bf91ef19710189f2fc7e6f6bb6559c71c189a5d877a770a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T19:10:42.434 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:42 smithi079 bash[70869]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-c 2024-04-25T19:10:42.698 INFO:journalctl@ceph.mon.c.smithi079.stdout:Apr 25 19:10:42 smithi079 podman[70884]: 2024-04-25 19:10:42.44444347 +0000 UTC m=+0.666476046 container remove 3fda186e9518c953bf91ef19710189f2fc7e6f6bb6559c71c189a5d877a770a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T19:10:42.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T19:10:42.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:10:42.745 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T19:10:42.745 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T19:10:42.745 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.x 2024-04-25T19:10:43.214 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:10:42 smithi045 systemd[1]: Stopping Ceph mgr.x for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:10:43.714 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:10:43 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T19:10:43.325+0000 7f932b200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:10:43.714 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:10:43 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x[28018]: 2024-04-25T19:10:43.325+0000 7f932b200700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T19:10:43.714 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:10:43 smithi045 podman[112380]: 2024-04-25 19:10:43.430686193 +0000 UTC m=+0.261557823 container died ffacc8ad311ab730175a2f6385858bbc0add9a209f7f335b2cee9d66ba55bf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T19:10:44.114 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:10:43 smithi045 podman[112380]: 2024-04-25 19:10:43.894035747 +0000 UTC m=+0.724907394 container cleanup ffacc8ad311ab730175a2f6385858bbc0add9a209f7f335b2cee9d66ba55bf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:10:44.114 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:10:43 smithi045 bash[112380]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x 2024-04-25T19:10:44.464 INFO:journalctl@ceph.mgr.x.smithi045.stdout:Apr 25 19:10:44 smithi045 podman[112394]: 2024-04-25 19:10:44.11237799 +0000 UTC m=+0.665687323 container remove ffacc8ad311ab730175a2f6385858bbc0add9a209f7f335b2cee9d66ba55bf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T19:10:44.675 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T19:10:44.788 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:10:44.788 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T19:10:44.788 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T19:10:44.789 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.y 2024-04-25T19:10:45.112 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 19:10:44 smithi062 systemd[1]: Stopping Ceph mgr.y for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:10:45.364 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 19:10:45 smithi062 podman[70123]: 2024-04-25 19:10:45.110862533 +0000 UTC m=+0.139566277 container died 4cd562e72ce52b9a396539c52f784b1121a85e97bbc0cc526d08f107d508eef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T19:10:45.844 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 19:10:45 smithi062 podman[70123]: 2024-04-25 19:10:45.481927058 +0000 UTC m=+0.510630786 container cleanup 4cd562e72ce52b9a396539c52f784b1121a85e97bbc0cc526d08f107d508eef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:10:45.844 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 19:10:45 smithi062 bash[70123]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y 2024-04-25T19:10:45.844 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 19:10:45 smithi062 podman[70137]: 2024-04-25 19:10:45.633071283 +0000 UTC m=+0.515667743 container remove 4cd562e72ce52b9a396539c52f784b1121a85e97bbc0cc526d08f107d508eef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T19:10:45.844 INFO:journalctl@ceph.mgr.y.smithi062.stdout:Apr 25 19:10:45 smithi062 systemd[1]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-25T19:10:45.855 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T19:10:45.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:10:45.892 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T19:10:45.892 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T19:10:45.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.z 2024-04-25T19:10:46.184 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 19:10:45 smithi079 systemd[1]: Stopping Ceph mgr.z for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:10:46.476 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 19:10:46 smithi079 podman[71000]: 2024-04-25 19:10:46.217658097 +0000 UTC m=+0.136772103 container died 792d1af133808fda831b7a9ed83d412f9510de04e932e0d4bcdf7927857b8e7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:10:46.749 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 19:10:46 smithi079 podman[71000]: 2024-04-25 19:10:46.555069458 +0000 UTC m=+0.474183470 container cleanup 792d1af133808fda831b7a9ed83d412f9510de04e932e0d4bcdf7927857b8e7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-04-25T19:10:46.749 INFO:journalctl@ceph.mgr.z.smithi079.stdout:Apr 25 19:10:46 smithi079 bash[71000]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mgr-z 2024-04-25T19:10:46.973 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T19:10:47.009 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:10:47.009 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T19:10:47.009 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T19:10:47.009 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.0 2024-04-25T19:10:47.463 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:47 smithi045 systemd[1]: Stopping Ceph osd.0 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:10:47.964 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:47 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0[39895]: 2024-04-25T19:10:47.587+0000 7f33c4c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:10:47.964 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:47 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0[39895]: 2024-04-25T19:10:47.587+0000 7f33c4c00700 -1 osd.0 109 *** Got signal Terminated *** 2024-04-25T19:10:47.964 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:47 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0[39895]: 2024-04-25T19:10:47.587+0000 7f33c4c00700 -1 osd.0 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:10:53.339 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:53 smithi045 podman[112500]: 2024-04-25 19:10:53.035857102 +0000 UTC m=+5.574308454 container died c2b27313cd214b01f87e8b4fda30d596c57b486d7a4051f7f5b90c79dcdf603c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T19:10:53.714 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:53 smithi045 podman[112500]: 2024-04-25 19:10:53.464475216 +0000 UTC m=+6.002926631 container cleanup c2b27313cd214b01f87e8b4fda30d596c57b486d7a4051f7f5b90c79dcdf603c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T19:10:53.714 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:53 smithi045 bash[112500]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0 2024-04-25T19:10:54.214 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:53 smithi045 podman[112533]: 2024-04-25 19:10:53.715732201 +0000 UTC m=+0.664905597 container remove c2b27313cd214b01f87e8b4fda30d596c57b486d7a4051f7f5b90c79dcdf603c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T19:10:54.914 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:54 smithi045 podman[112621]: 2024-04-25 19:10:54.521530002 +0000 UTC m=+0.041908535 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:10:54.914 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:54 smithi045 podman[112621]: 2024-04-25 19:10:54.641527135 +0000 UTC m=+0.161905696 container create 0c0ee01c29bea4dadbe5bb4e4267ce5efe398b579260558ef9e427a49cb3b08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T19:10:54.914 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:54 smithi045 podman[112621]: 2024-04-25 19:10:54.911921981 +0000 UTC m=+0.432300609 container init 0c0ee01c29bea4dadbe5bb4e4267ce5efe398b579260558ef9e427a49cb3b08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T19:10:55.214 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:54 smithi045 podman[112621]: 2024-04-25 19:10:54.923898844 +0000 UTC m=+0.444277432 container start 0c0ee01c29bea4dadbe5bb4e4267ce5efe398b579260558ef9e427a49cb3b08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-25T19:10:55.214 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:55 smithi045 podman[112621]: 2024-04-25 19:10:55.00229884 +0000 UTC m=+0.522677462 container attach 0c0ee01c29bea4dadbe5bb4e4267ce5efe398b579260558ef9e427a49cb3b08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T19:10:55.714 INFO:journalctl@ceph.osd.0.smithi045.stdout:Apr 25 19:10:55 smithi045 podman[112621]: 2024-04-25 19:10:55.398908657 +0000 UTC m=+0.919287182 container died 0c0ee01c29bea4dadbe5bb4e4267ce5efe398b579260558ef9e427a49cb3b08f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T19:10:56.077 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T19:10:56.198 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:10:56.198 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T19:10:56.198 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T19:10:56.199 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.1 2024-04-25T19:10:56.488 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:10:56 smithi062 systemd[1]: Stopping Ceph osd.1 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:10:56.864 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:10:56 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1[37421]: 2024-04-25T19:10:56.486+0000 7f40ab451700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:10:56.864 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:10:56 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1[37421]: 2024-04-25T19:10:56.486+0000 7f40ab451700 -1 osd.1 109 *** Got signal Terminated *** 2024-04-25T19:10:56.864 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:10:56 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1[37421]: 2024-04-25T19:10:56.486+0000 7f40ab451700 -1 osd.1 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:11:02.147 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:11:01 smithi062 podman[70249]: 2024-04-25 19:11:01.866013951 +0000 UTC m=+5.473968689 container died b365222624b98438406ac9c5ed3dba067e3c374dc6eb76eaa540de9dd1287df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:11:02.480 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:11:02 smithi062 podman[70249]: 2024-04-25 19:11:02.236572205 +0000 UTC m=+5.844526941 container cleanup b365222624b98438406ac9c5ed3dba067e3c374dc6eb76eaa540de9dd1287df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T19:11:02.480 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:11:02 smithi062 bash[70249]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1 2024-04-25T19:11:02.814 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:11:02 smithi062 podman[70267]: 2024-04-25 19:11:02.47936519 +0000 UTC m=+0.610738514 container remove b365222624b98438406ac9c5ed3dba067e3c374dc6eb76eaa540de9dd1287df1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T19:11:03.067 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:11:02 smithi062 podman[70362]: 2024-04-25 19:11:02.716668941 +0000 UTC m=+0.018407849 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:11:03.067 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:11:02 smithi062 podman[70362]: 2024-04-25 19:11:02.830604722 +0000 UTC m=+0.132343655 container create 6c928ee20b5984baa68ea8a63284ce3532cca6a873595da783de18b56af2eabe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T19:11:03.364 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:11:03 smithi062 podman[70362]: 2024-04-25 19:11:03.066484428 +0000 UTC m=+0.368223321 container init 6c928ee20b5984baa68ea8a63284ce3532cca6a873595da783de18b56af2eabe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T19:11:03.364 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:11:03 smithi062 podman[70362]: 2024-04-25 19:11:03.06984762 +0000 UTC m=+0.371586513 container start 6c928ee20b5984baa68ea8a63284ce3532cca6a873595da783de18b56af2eabe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T19:11:03.364 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:11:03 smithi062 podman[70362]: 2024-04-25 19:11:03.157960859 +0000 UTC m=+0.459699759 container attach 6c928ee20b5984baa68ea8a63284ce3532cca6a873595da783de18b56af2eabe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:11:03.670 INFO:journalctl@ceph.osd.1.smithi062.stdout:Apr 25 19:11:03 smithi062 podman[70362]: 2024-04-25 19:11:03.378062017 +0000 UTC m=+0.679800899 container died 6c928ee20b5984baa68ea8a63284ce3532cca6a873595da783de18b56af2eabe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T19:11:03.943 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T19:11:03.984 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:11:03.984 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T19:11:03.984 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T19:11:03.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.2 2024-04-25T19:11:04.284 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:04 smithi079 systemd[1]: Stopping Ceph osd.2 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:11:04.685 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:04 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2[38165]: 2024-04-25T19:11:04.281+0000 7f45fc5d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:11:04.685 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:04 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2[38165]: 2024-04-25T19:11:04.281+0000 7f45fc5d1700 -1 osd.2 109 *** Got signal Terminated *** 2024-04-25T19:11:04.685 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:04 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2[38165]: 2024-04-25T19:11:04.281+0000 7f45fc5d1700 -1 osd.2 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:11:09.934 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:09 smithi079 podman[71124]: 2024-04-25 19:11:09.66842672 +0000 UTC m=+5.496805654 container died 52100f97f58cd410775bf74c054e238bb804af0fdc6581fcc48663a5c47ec30f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:11:10.293 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:10 smithi079 podman[71124]: 2024-04-25 19:11:10.065120683 +0000 UTC m=+5.893499622 container cleanup 52100f97f58cd410775bf74c054e238bb804af0fdc6581fcc48663a5c47ec30f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T19:11:10.293 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:10 smithi079 bash[71124]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2 2024-04-25T19:11:10.623 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:10 smithi079 podman[71141]: 2024-04-25 19:11:10.29129651 +0000 UTC m=+0.618610854 container remove 52100f97f58cd410775bf74c054e238bb804af0fdc6581fcc48663a5c47ec30f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T19:11:10.881 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:10 smithi079 podman[71232]: 2024-04-25 19:11:10.524914988 +0000 UTC m=+0.018038071 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:11:10.881 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:10 smithi079 podman[71232]: 2024-04-25 19:11:10.642167529 +0000 UTC m=+0.135290596 container create 98869eea8dfd3171c60b62027973e2011ef7a7c6891ace36924ebbea9ff66c6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T19:11:10.881 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:10 smithi079 podman[71232]: 2024-04-25 19:11:10.87769164 +0000 UTC m=+0.370814702 container init 98869eea8dfd3171c60b62027973e2011ef7a7c6891ace36924ebbea9ff66c6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T19:11:11.176 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:10 smithi079 podman[71232]: 2024-04-25 19:11:10.880826656 +0000 UTC m=+0.373949727 container start 98869eea8dfd3171c60b62027973e2011ef7a7c6891ace36924ebbea9ff66c6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:11:11.177 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:10 smithi079 podman[71232]: 2024-04-25 19:11:10.961134291 +0000 UTC m=+0.454257357 container attach 98869eea8dfd3171c60b62027973e2011ef7a7c6891ace36924ebbea9ff66c6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T19:11:11.434 INFO:journalctl@ceph.osd.2.smithi079.stdout:Apr 25 19:11:11 smithi079 podman[71232]: 2024-04-25 19:11:11.175759888 +0000 UTC m=+0.668882944 container died 98869eea8dfd3171c60b62027973e2011ef7a7c6891ace36924ebbea9ff66c6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:11:11.828 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T19:11:11.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:11:11.864 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T19:11:11.865 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T19:11:11.865 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.3 2024-04-25T19:11:12.213 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:11 smithi045 systemd[1]: Stopping Ceph osd.3 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:11:12.714 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:12 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3[46402]: 2024-04-25T19:11:12.440+0000 7f0f91e00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:11:12.714 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:12 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3[46402]: 2024-04-25T19:11:12.440+0000 7f0f91e00700 -1 osd.3 109 *** Got signal Terminated *** 2024-04-25T19:11:12.714 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:12 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3[46402]: 2024-04-25T19:11:12.440+0000 7f0f91e00700 -1 osd.3 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:11:13.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:13.407+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:13.685 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:13.591+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:14.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:14.421+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:14.684 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:14.637+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:14.714 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:14 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3[46402]: 2024-04-25T19:11:14.360+0000 7f0f88600700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:47.760629+0000 front 2024-04-25T19:10:47.760733+0000 (oldest deadline 2024-04-25T19:11:13.660297+0000) 2024-04-25T19:11:15.613 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:15.275+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:15.714 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:15 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3[46402]: 2024-04-25T19:11:15.371+0000 7f0f88600700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:47.760629+0000 front 2024-04-25T19:10:47.760733+0000 (oldest deadline 2024-04-25T19:11:13.660297+0000) 2024-04-25T19:11:15.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:15 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:15.218+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:15.934 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:15.453+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:15.934 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:15.607+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:16.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:15 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:15.784+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:16.581 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:16 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:16.269+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:16.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:16 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:16.218+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:16.714 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:16 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3[46402]: 2024-04-25T19:11:16.389+0000 7f0f88600700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:47.760629+0000 front 2024-04-25T19:10:47.760733+0000 (oldest deadline 2024-04-25T19:11:13.660297+0000) 2024-04-25T19:11:16.743 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:16 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:16.439+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:16.744 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:16 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:16.591+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:17.184 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:16 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:16.742+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:17.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:16 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:16.740+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:17.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:17 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:17.254+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:17.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:17 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:17.241+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:17.714 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:17 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3[46402]: 2024-04-25T19:11:17.402+0000 7f0f88600700 -1 osd.3 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:47.760629+0000 front 2024-04-25T19:10:47.760733+0000 (oldest deadline 2024-04-25T19:11:13.660297+0000) 2024-04-25T19:11:17.774 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:17 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:17.472+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:17.774 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:17 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:17.626+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:18.056 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:17 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:17.756+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:18.184 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:17 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:17.772+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:18.331 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:18 smithi045 podman[112727]: 2024-04-25 19:11:18.055210576 +0000 UTC m=+5.740145997 container died 195f82b657df4c93c83d117fe5acaaad288c449d47e733bf0baa20a7569731eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T19:11:18.331 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:18 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:18.265+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:18.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:18 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:18.217+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:18.626 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:18 smithi045 podman[112727]: 2024-04-25 19:11:18.448863096 +0000 UTC m=+6.133798536 container cleanup 195f82b657df4c93c83d117fe5acaaad288c449d47e733bf0baa20a7569731eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T19:11:18.626 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:18 smithi045 bash[112727]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3 2024-04-25T19:11:18.934 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:18.586+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:18.934 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:18.514+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:18.935 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:18.731+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:18.965 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:18 smithi045 podman[112748]: 2024-04-25 19:11:18.624768394 +0000 UTC m=+0.556560609 container remove 195f82b657df4c93c83d117fe5acaaad288c449d47e733bf0baa20a7569731eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T19:11:18.965 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:18 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:18.745+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:19.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:19 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:19.218+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:19.654 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:19.279+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:19.654 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:19 smithi045 podman[112835]: 2024-04-25 19:11:19.384477421 +0000 UTC m=+0.040782007 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:11:19.654 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:19 smithi045 podman[112835]: 2024-04-25 19:11:19.49218766 +0000 UTC m=+0.148492240 container create ccd19bf759613129dbf8ecd265e9d105952b52f89f4aa00bcf93928a1af78c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T19:11:19.934 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:19.556+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:19.935 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:19.708+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:19.935 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:19.561+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:19.964 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:19 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:19.742+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:19.964 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:19 smithi045 podman[112835]: 2024-04-25 19:11:19.756655183 +0000 UTC m=+0.412959776 container init ccd19bf759613129dbf8ecd265e9d105952b52f89f4aa00bcf93928a1af78c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T19:11:19.964 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:19 smithi045 podman[112835]: 2024-04-25 19:11:19.767817717 +0000 UTC m=+0.424122301 container start ccd19bf759613129dbf8ecd265e9d105952b52f89f4aa00bcf93928a1af78c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:11:19.965 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:19 smithi045 podman[112835]: 2024-04-25 19:11:19.844307084 +0000 UTC m=+0.500611724 container attach ccd19bf759613129dbf8ecd265e9d105952b52f89f4aa00bcf93928a1af78c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T19:11:20.527 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:20 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:20.320+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:20.529 INFO:journalctl@ceph.osd.3.smithi045.stdout:Apr 25 19:11:20 smithi045 podman[112835]: 2024-04-25 19:11:20.24410276 +0000 UTC m=+0.900407339 container died ccd19bf759613129dbf8ecd265e9d105952b52f89f4aa00bcf93928a1af78c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T19:11:20.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:20 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:20.179+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:20.842 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:20 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:20.745+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:20.899 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T19:11:20.934 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:20.673+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:20.934 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:20.510+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:20.935 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:20.583+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:21.003 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:11:21.004 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T19:11:21.004 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T19:11:21.004 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.4 2024-04-25T19:11:21.324 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:21 smithi062 systemd[1]: Stopping Ceph osd.4 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:11:21.324 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:21 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:21.219+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:21.614 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:21 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4[44345]: 2024-04-25T19:11:21.321+0000 7f48a383a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:11:21.614 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:21 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4[44345]: 2024-04-25T19:11:21.321+0000 7f48a383a700 -1 osd.4 109 *** Got signal Terminated *** 2024-04-25T19:11:21.614 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:21 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4[44345]: 2024-04-25T19:11:21.321+0000 7f48a383a700 -1 osd.4 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:11:21.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:21 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:21.365+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:21.885 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:21 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:21.464+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:21.886 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:21 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:21.564+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:21.886 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:21 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:21.643+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:22.021 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:21 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:21.747+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:22.614 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:22 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4[44345]: 2024-04-25T19:11:22.313+0000 7f489fe2e700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.922902+0000 front 2024-04-25T19:10:59.923085+0000 (oldest deadline 2024-04-25T19:11:22.223084+0000) 2024-04-25T19:11:22.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:22 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:22.225+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:22.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:22 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:22.334+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:22.934 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:22.657+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:22.934 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:22.514+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:22.935 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:22.588+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:23.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:22 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:22.777+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:23.614 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:23 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4[44345]: 2024-04-25T19:11:23.317+0000 7f489fe2e700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.922902+0000 front 2024-04-25T19:10:59.923085+0000 (oldest deadline 2024-04-25T19:11:22.223084+0000) 2024-04-25T19:11:23.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:23 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:23.194+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:23.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:23 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:23.320+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:23.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:23 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:23.320+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:23.934 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:23 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:23.488+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:23.934 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:23 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:23.630+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:23.935 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:23 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:23.631+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:24.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:23 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:23.807+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:24.614 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:24 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4[44345]: 2024-04-25T19:11:24.321+0000 7f489fe2e700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.922902+0000 front 2024-04-25T19:10:59.923085+0000 (oldest deadline 2024-04-25T19:11:22.223084+0000) 2024-04-25T19:11:24.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:24 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:24.216+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:24.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:24.294+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:24.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:24.294+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:24.934 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:24.631+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:24.934 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:24.448+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:24.935 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:24.668+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:25.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:24.818+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:25.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:24 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:24.818+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:25.614 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:25 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4[44345]: 2024-04-25T19:11:25.305+0000 7f489fe2e700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.922902+0000 front 2024-04-25T19:10:59.923085+0000 (oldest deadline 2024-04-25T19:11:22.223084+0000) 2024-04-25T19:11:25.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:25 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:25.215+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:25.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:25 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:25.215+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:25.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:25 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:25.319+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:25.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:25 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:25.319+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:25.934 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:25 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:25.645+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:25.934 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:25 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:25.645+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.058256+0000 front 2024-04-25T19:10:59.058051+0000 (oldest deadline 2024-04-25T19:11:24.958023+0000) 2024-04-25T19:11:25.935 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:25 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:25.459+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:25.935 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:25 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:25.696+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:25.935 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:25 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:25.696+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:26.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:25 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:25.823+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:26.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:25 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:25.823+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:26.249 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:25 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:25.880+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:26.614 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:26 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4[44345]: 2024-04-25T19:11:26.280+0000 7f489fe2e700 -1 osd.4 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.922902+0000 front 2024-04-25T19:10:59.923085+0000 (oldest deadline 2024-04-25T19:11:22.223084+0000) 2024-04-25T19:11:26.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:26 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:26.248+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:26.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:26 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:26.248+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:26.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:26 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:26.311+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:26.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:26 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:26.311+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:26.732 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:26 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:26.469+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:26.732 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:26 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:26.678+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:26.732 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:26 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:26.678+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.058256+0000 front 2024-04-25T19:10:59.058051+0000 (oldest deadline 2024-04-25T19:11:24.958023+0000) 2024-04-25T19:11:26.933 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:26 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:26.841+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:26.934 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:26 smithi062 podman[70491]: 2024-04-25 19:11:26.676483388 +0000 UTC m=+5.478066649 container died 00bbd8fe0418e909e3856abd4c4d6cd17822a0c8fa7ae183e2079a14ee9ab0ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T19:11:27.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:26 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:26.731+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:27.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:26 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:26.731+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:27.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:26 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:26.807+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:27.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:26 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:26.807+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:27.244 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:27 smithi062 podman[70491]: 2024-04-25 19:11:27.022806926 +0000 UTC m=+5.824390186 container cleanup 00bbd8fe0418e909e3856abd4c4d6cd17822a0c8fa7ae183e2079a14ee9ab0ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T19:11:27.244 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:27 smithi062 bash[70491]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4 2024-04-25T19:11:27.614 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:27 smithi062 podman[70508]: 2024-04-25 19:11:27.316303241 +0000 UTC m=+0.639403421 container remove 00bbd8fe0418e909e3856abd4c4d6cd17822a0c8fa7ae183e2079a14ee9ab0ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:11:27.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:27 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:27.242+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:27.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:27 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:27.242+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:27.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:27 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:27.281+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:27.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:27 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:27.281+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:27.773 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:27 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:27.474+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:27.774 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:27 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:27.642+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:27.774 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:27 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:27.642+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.058256+0000 front 2024-04-25T19:10:59.058051+0000 (oldest deadline 2024-04-25T19:11:24.958023+0000) 2024-04-25T19:11:27.962 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:27 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:27.838+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:27.963 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:27 smithi062 podman[70597]: 2024-04-25 19:11:27.63360797 +0000 UTC m=+0.107402018 container create 48760a87838531b12e4b500c6d97aa862012407b675903fb8f775b697ae1d6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T19:11:27.963 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:27 smithi062 podman[70597]: 2024-04-25 19:11:27.544862209 +0000 UTC m=+0.018656264 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:11:27.963 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:27 smithi062 podman[70597]: 2024-04-25 19:11:27.877664542 +0000 UTC m=+0.351458590 container init 48760a87838531b12e4b500c6d97aa862012407b675903fb8f775b697ae1d6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T19:11:27.963 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:27 smithi062 podman[70597]: 2024-04-25 19:11:27.880816864 +0000 UTC m=+0.354610909 container start 48760a87838531b12e4b500c6d97aa862012407b675903fb8f775b697ae1d6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T19:11:27.963 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:27 smithi062 podman[70597]: 2024-04-25 19:11:27.960961083 +0000 UTC m=+0.434755133 container attach 48760a87838531b12e4b500c6d97aa862012407b675903fb8f775b697ae1d6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:11:28.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:27 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:27.772+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:28.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:27 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:27.772+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:28.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:27 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:27.795+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:28.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:27 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:27.795+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:28.285 INFO:journalctl@ceph.osd.4.smithi062.stdout:Apr 25 19:11:28 smithi062 podman[70597]: 2024-04-25 19:11:28.182624946 +0000 UTC m=+0.656418991 container died 48760a87838531b12e4b500c6d97aa862012407b675903fb8f775b697ae1d6f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T19:11:28.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:28 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:28.283+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:28.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:28 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:28.283+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:28.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:28 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:28.428+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:28.684 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:28 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:28.632+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:28.684 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:28 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:28.632+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.058256+0000 front 2024-04-25T19:10:59.058051+0000 (oldest deadline 2024-04-25T19:11:24.958023+0000) 2024-04-25T19:11:28.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:28 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:28.304+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:28.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:28 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:28.304+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:28.742 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T19:11:28.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:11:28.780 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T19:11:28.780 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T19:11:28.780 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.5 2024-04-25T19:11:29.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:28 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:28.838+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:29.184 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:28 smithi079 systemd[1]: Stopping Ceph osd.5 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:11:29.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:28 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:28.725+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:29.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:28 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:28.725+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:29.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:28 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:28.844+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:29.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:28 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:28.844+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:29.479 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:29 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:29.212+0000 7f115d409700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:11:29.479 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:29 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:29.212+0000 7f115d409700 -1 osd.5 109 *** Got signal Terminated *** 2024-04-25T19:11:29.479 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:29 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:29.212+0000 7f115d409700 -1 osd.5 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:11:29.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:29 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:29.279+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:29.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:29 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:29.279+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:29.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:29 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:29.324+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:29.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:29 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:29.324+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:29.934 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:29 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:29.476+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:29.935 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:29 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:29.710+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:29.935 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:29 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:29.710+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:29.935 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:29 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:29.605+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:29.935 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:29 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:29.605+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.058256+0000 front 2024-04-25T19:10:59.058051+0000 (oldest deadline 2024-04-25T19:11:24.958023+0000) 2024-04-25T19:11:30.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:29 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:29.831+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:30.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:29 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:29.842+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:30.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:29 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:29.842+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:30.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:30 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:30.245+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:30.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:30 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:30.245+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:30.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:30 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:30.300+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:30.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:30 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:30.300+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:30.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:30 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:30.300+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:30.934 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:30 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:30.466+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:30.934 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:30 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:30.569+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:30.935 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:30 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:30.569+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.058256+0000 front 2024-04-25T19:10:59.058051+0000 (oldest deadline 2024-04-25T19:11:24.958023+0000) 2024-04-25T19:11:30.935 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:30 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:30.709+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:30.935 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:30 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:30.709+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:31.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:30 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:30.787+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:31.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:30 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:30.871+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:31.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:30 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:30.871+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:31.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:31 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:31.202+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:31.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:31 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:31.202+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:31.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:31 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:31.270+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:31.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:31 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:31.270+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:31.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:31 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:31.270+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:31.733 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:31 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:31.468+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:31.733 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:31 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:31.586+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:31.733 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:31 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:31.586+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.058256+0000 front 2024-04-25T19:10:59.058051+0000 (oldest deadline 2024-04-25T19:11:24.958023+0000) 2024-04-25T19:11:32.091 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:31 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:31.825+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:32.091 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:31 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:31.825+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:32.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:31 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:31.795+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:32.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:31 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:31.730+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:32.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:31 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:31.730+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:32.464 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:32 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:32.262+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:32.464 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:32 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:32.262+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:32.464 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:32 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:32.262+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:32.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:32 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:32.208+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:32.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:32 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:32.208+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:32.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:32 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:32.208+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:32.774 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:32 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:32.472+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:32.775 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:32 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:32.596+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:32.775 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:32 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:32.596+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.058256+0000 front 2024-04-25T19:10:59.058051+0000 (oldest deadline 2024-04-25T19:11:24.958023+0000) 2024-04-25T19:11:32.775 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:32 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:32.596+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:06.059151+0000 front 2024-04-25T19:11:06.059176+0000 (oldest deadline 2024-04-25T19:11:31.959340+0000) 2024-04-25T19:11:33.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:32 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:32.782+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:33.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:32 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:32.782+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:33.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:32 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:32.772+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:33.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:32 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:32.772+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:33.213 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:32 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:32.857+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:33.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:32 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:32.857+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:33.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:32 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:32.857+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:33.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:33 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:33.251+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:33.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:33 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:33.251+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:33.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:33 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:33.251+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:33.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:33 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:33.222+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:33.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:33 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:33.222+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:33.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:33 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:33.222+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:33.759 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:33 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:33.506+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:33.759 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:33 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:33.558+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.257173+0000 front 2024-04-25T19:10:50.257066+0000 (oldest deadline 2024-04-25T19:11:16.156889+0000) 2024-04-25T19:11:33.759 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:33 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:33.558+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.058256+0000 front 2024-04-25T19:10:59.058051+0000 (oldest deadline 2024-04-25T19:11:24.958023+0000) 2024-04-25T19:11:33.759 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:33 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5[45125]: 2024-04-25T19:11:33.558+0000 7f11599fd700 -1 osd.5 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:06.059151+0000 front 2024-04-25T19:11:06.059176+0000 (oldest deadline 2024-04-25T19:11:31.959340+0000) 2024-04-25T19:11:34.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:33 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:33.753+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:34.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:33 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:33.753+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:34.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:33 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:33.757+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:34.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:33 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:33.757+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:34.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:33 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:33.757+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:34.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:33 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:33.864+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:34.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:33 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:33.864+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:34.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:33 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:33.864+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:34.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:34 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:34.300+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:34.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:34 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:34.300+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:34.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:34 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:34.300+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:34.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:34 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:34.232+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:34.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:34 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:34.232+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:34.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:34 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:34.232+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:34.782 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:34 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:34.470+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:34.782 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:34 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:34.470+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:34.782 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:34 smithi079 podman[71348]: 2024-04-25 19:11:34.562375937 +0000 UTC m=+5.593343646 container died 159235f83c85d98a2e3c4860e0c248f411f1ae5fa2cc3a72e4a1b488aa75779c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:11:35.095 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:34 smithi079 podman[71348]: 2024-04-25 19:11:34.901071063 +0000 UTC m=+5.932038770 container cleanup 159235f83c85d98a2e3c4860e0c248f411f1ae5fa2cc3a72e4a1b488aa75779c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T19:11:35.095 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:34 smithi079 bash[71348]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5 2024-04-25T19:11:35.095 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:34 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:34.779+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:35.095 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:34 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:34.779+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:35.095 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:34 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:34.779+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:35.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:34 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:34.709+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:35.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:34 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:34.709+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:35.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:34 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:34.827+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:35.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:34 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:34.827+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:35.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:34 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:34.827+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:35.434 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:35 smithi079 podman[71367]: 2024-04-25 19:11:35.093997346 +0000 UTC m=+0.532639558 container remove 159235f83c85d98a2e3c4860e0c248f411f1ae5fa2cc3a72e4a1b488aa75779c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T19:11:35.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:35 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:35.274+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:35.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:35 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:35.274+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:35.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:35 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:35.274+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:35.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:35 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:35.282+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:35.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:35 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:35.282+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:35.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:35 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:35.282+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:35.749 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:35 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:35.445+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:35.749 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:35 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:35.445+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:35.749 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:35 smithi079 podman[71457]: 2024-04-25 19:11:35.511618429 +0000 UTC m=+0.114344708 container create cbec2abda1384280571a01194563e31278c1dc52939b2fac669b113503dd0e1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:11:35.749 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:35 smithi079 podman[71457]: 2024-04-25 19:11:35.415310205 +0000 UTC m=+0.018036498 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:11:36.041 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:35 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:35.806+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:36.042 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:35 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:35.806+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:36.042 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:35 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:35.806+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:36.042 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:35 smithi079 podman[71457]: 2024-04-25 19:11:35.747908574 +0000 UTC m=+0.350634863 container init cbec2abda1384280571a01194563e31278c1dc52939b2fac669b113503dd0e1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:11:36.042 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:35 smithi079 podman[71457]: 2024-04-25 19:11:35.751122283 +0000 UTC m=+0.353848565 container start cbec2abda1384280571a01194563e31278c1dc52939b2fac669b113503dd0e1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:11:36.042 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:35 smithi079 podman[71457]: 2024-04-25 19:11:35.830760742 +0000 UTC m=+0.433487035 container attach cbec2abda1384280571a01194563e31278c1dc52939b2fac669b113503dd0e1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T19:11:36.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:35 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:35.722+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:36.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:35 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:35.722+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:36.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:35 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:35.801+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:36.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:35 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:35.801+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:36.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:35 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:35.801+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:36.334 INFO:journalctl@ceph.osd.5.smithi079.stdout:Apr 25 19:11:36 smithi079 podman[71457]: 2024-04-25 19:11:36.040873957 +0000 UTC m=+0.643600252 container died cbec2abda1384280571a01194563e31278c1dc52939b2fac669b113503dd0e1a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T19:11:36.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:36 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:36.290+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:36.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:36 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:36.290+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:36.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:36 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:36.290+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:36.619 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:36 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:36.427+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:36.619 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:36 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:36.427+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:36.641 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T19:11:36.682 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:11:36.683 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T19:11:36.683 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T19:11:36.683 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.6 2024-04-25T19:11:36.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:36 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:36.292+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:36.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:36 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:36.292+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:36.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:36 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:36.292+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:36.934 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:36 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:36.792+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:36.934 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:36 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:36.792+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:36.934 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:36 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:36.792+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:37.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:36 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:36.680+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:37.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:36 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:36.680+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:37.214 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:36 smithi045 systemd[1]: Stopping Ceph osd.6 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:11:37.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:36 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:36.771+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:37.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:36 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:36.771+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:37.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:36 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:36.771+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:37.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:37 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:37.288+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:37.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:37 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:37.288+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:37.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:37 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:37.288+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:37.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:37 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:37.429+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:37.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:37 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:37.429+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:37.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:37.269+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:37.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:37.269+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:37.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:37.269+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:37.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:37.290+0000 7f5bdf800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:11:37.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:37.290+0000 7f5bdf800700 -1 osd.6 109 *** Got signal Terminated *** 2024-04-25T19:11:37.715 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:37.290+0000 7f5bdf800700 -1 osd.6 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:11:38.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:37 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:37.724+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:38.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:37 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:37.724+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:38.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:37 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:37.808+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:38.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:37 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:37.808+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:38.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:37 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:37.808+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:38.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:37.779+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:38.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:37.779+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:38.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:37 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:37.779+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:38.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:38 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:38.275+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:38.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:38 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:38.275+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:38.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:38 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:38.275+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:38.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:38 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:38.394+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:38.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:38 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:38.394+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:38.699 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:38 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:38.696+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:38.699 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:38 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:38.696+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:38.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:38 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:38.270+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:38.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:38 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:38.270+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:38.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:38 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:38.270+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:39.185 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:38 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:38.815+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:39.185 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:38 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:38.815+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:39.185 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:38 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:38.815+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:39.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:38 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:38.828+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:39.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:38 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:38.828+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:39.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:38 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:38.828+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:39.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:39 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:39.306+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:39.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:39 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:39.306+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:39.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:39 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:39.306+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:39.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:39 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:39.364+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:39.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:39 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:39.364+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:39.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:39.314+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:39.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:39.314+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:39.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:39.314+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:40.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:39 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:39.659+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:40.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:39 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:39.659+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:40.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:39 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:39.812+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:40.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:39 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:39.812+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:40.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:39 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:39.812+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:40.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:39.871+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:40.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:39.871+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:40.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:39.871+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:40.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:40.341+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:40.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:40.341+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:40.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:40.341+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:40.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:40 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:40.384+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:40.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:40 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:40.384+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:40.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:40.274+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:40.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:40.274+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:40.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:40.274+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:40.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:40.274+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.283898+0000 front 2024-04-25T19:11:17.283834+0000 (oldest deadline 2024-04-25T19:11:40.183521+0000) 2024-04-25T19:11:41.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:40.686+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:41.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:40 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:40.686+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:41.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:40 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:40.768+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:41.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:40 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:40.768+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:41.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:40 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:40.768+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:41.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:40 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:40.768+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:41.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:40.884+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:41.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:40.884+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:41.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:40.884+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:41.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:40 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:40.884+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:41.664 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:41 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:41.386+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:41.694 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:41 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:41.386+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:41.694 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:41 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:41.386+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:41.695 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:41.403+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:41.695 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:41.403+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:41.695 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:41.403+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:41.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:41.314+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.074990+0000 front 2024-04-25T19:10:51.074842+0000 (oldest deadline 2024-04-25T19:11:15.173779+0000) 2024-04-25T19:11:41.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:41.314+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.275883+0000 front 2024-04-25T19:10:59.275988+0000 (oldest deadline 2024-04-25T19:11:22.776097+0000) 2024-04-25T19:11:41.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:41.314+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.380208+0000 front 2024-04-25T19:11:07.380285+0000 (oldest deadline 2024-04-25T19:11:30.280485+0000) 2024-04-25T19:11:41.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6[53225]: 2024-04-25T19:11:41.314+0000 7f5bdb000700 -1 osd.6 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.283898+0000 front 2024-04-25T19:11:17.283834+0000 (oldest deadline 2024-04-25T19:11:40.183521+0000) 2024-04-25T19:11:42.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:41 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:41.662+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:42.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:41 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:41.662+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:42.115 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:41 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:41.662+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.140007+0000 front 2024-04-25T19:11:17.139827+0000 (oldest deadline 2024-04-25T19:11:41.240016+0000) 2024-04-25T19:11:42.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:41.760+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:42.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:41.760+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:42.185 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:41.760+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:42.185 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:41 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:41.760+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:42.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:41.932+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:42.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:41.932+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:42.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:41.932+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:42.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:41 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:41.932+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:42.453 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:42.377+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:42.453 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:42.377+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:42.454 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:42.377+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:42.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:42.345+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:42.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:42.345+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:42.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:42.345+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:42.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:42.345+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:43.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:42.691+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:43.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:42.691+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:43.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:42.691+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.140007+0000 front 2024-04-25T19:11:17.139827+0000 (oldest deadline 2024-04-25T19:11:41.240016+0000) 2024-04-25T19:11:43.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:42.753+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:43.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:42.753+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:43.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:42.753+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:43.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:42 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:42.753+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:43.198 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:42 smithi045 podman[112954]: 2024-04-25 19:11:42.893015261 +0000 UTC m=+5.743797727 container died baa69a5f6f9cb96a9fdb4c70332560d35db9757652fe38355b2fc2d89bb387fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T19:11:43.199 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:42 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:42.901+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:43.199 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:42 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:42.901+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:43.199 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:42 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:42.901+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:43.199 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:42 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:42.901+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:43.464 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:43 smithi045 podman[112954]: 2024-04-25 19:11:43.345364266 +0000 UTC m=+6.196146692 container cleanup baa69a5f6f9cb96a9fdb4c70332560d35db9757652fe38355b2fc2d89bb387fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T19:11:43.464 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:43 smithi045 bash[112954]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6 2024-04-25T19:11:43.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:43 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:43.365+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:43.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:43 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:43.365+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:43.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:43 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:43.365+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:43.741 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:43.394+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:43.742 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:43.394+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:43.742 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:43.394+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:43.742 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:43.394+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:43.905 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:43 smithi045 podman[112972]: 2024-04-25 19:11:43.58393917 +0000 UTC m=+0.673234536 container remove baa69a5f6f9cb96a9fdb4c70332560d35db9757652fe38355b2fc2d89bb387fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T19:11:44.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:43.739+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:44.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:43.739+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:44.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:43 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:43.739+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.140007+0000 front 2024-04-25T19:11:17.139827+0000 (oldest deadline 2024-04-25T19:11:41.240016+0000) 2024-04-25T19:11:44.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:43 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:43.736+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:44.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:43 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:43.736+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:44.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:43 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:43.736+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:44.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:43 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:43.736+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:44.213 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:43 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:43.902+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:44.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:43 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:43.902+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:44.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:43 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:43.902+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:44.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:43 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:43.902+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:44.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:44 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:44.336+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:44.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:44 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:44.336+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:44.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:44 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:44.336+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:44.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:44 smithi045 podman[113062]: 2024-04-25 19:11:44.328375766 +0000 UTC m=+0.037247949 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:11:44.714 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:44 smithi045 podman[113062]: 2024-04-25 19:11:44.47639078 +0000 UTC m=+0.185262987 container create 65c6359726b108e33488f9265c22885d39deeebe8740fe503ebfc4ab85c9cb77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:11:44.791 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:44.387+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:44.792 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:44.387+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:44.792 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:44.387+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:44.792 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:44.387+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:45.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:44.790+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:45.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:44.790+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:45.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:44 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:44.790+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.140007+0000 front 2024-04-25T19:11:17.139827+0000 (oldest deadline 2024-04-25T19:11:41.240016+0000) 2024-04-25T19:11:45.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:44 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:44.703+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:45.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:44 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:44.703+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:45.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:44 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:44.703+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:45.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:44 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:44.703+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:45.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:44 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:44.890+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:45.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:44 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:44.890+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:45.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:44 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:44.890+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:45.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:44 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:44.890+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:45.215 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:44 smithi045 podman[113062]: 2024-04-25 19:11:44.781302275 +0000 UTC m=+0.490174488 container init 65c6359726b108e33488f9265c22885d39deeebe8740fe503ebfc4ab85c9cb77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T19:11:45.215 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:44 smithi045 podman[113062]: 2024-04-25 19:11:44.794066906 +0000 UTC m=+0.502939116 container start 65c6359726b108e33488f9265c22885d39deeebe8740fe503ebfc4ab85c9cb77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T19:11:45.215 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:44 smithi045 podman[113062]: 2024-04-25 19:11:44.87892838 +0000 UTC m=+0.587800631 container attach 65c6359726b108e33488f9265c22885d39deeebe8740fe503ebfc4ab85c9cb77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T19:11:45.573 INFO:journalctl@ceph.osd.6.smithi045.stdout:Apr 25 19:11:45 smithi045 podman[113062]: 2024-04-25 19:11:45.271149469 +0000 UTC m=+0.980021673 container died 65c6359726b108e33488f9265c22885d39deeebe8740fe503ebfc4ab85c9cb77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T19:11:45.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:45.360+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:45.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:45.360+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:45.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:45.360+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:45.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:45.360+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:45.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:45 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:45.315+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:45.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:45 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:45.315+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:45.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:45 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:45.315+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:45.902 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:45.741+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:45.902 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:45.741+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:45.902 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:45 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:45.741+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.140007+0000 front 2024-04-25T19:11:17.139827+0000 (oldest deadline 2024-04-25T19:11:41.240016+0000) 2024-04-25T19:11:45.931 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T19:11:46.093 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:11:46.094 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T19:11:46.094 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T19:11:46.094 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.7 2024-04-25T19:11:46.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:45 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:45.750+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:46.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:45 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:45.750+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:46.185 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:45 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:45.750+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:46.185 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:45 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:45.750+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:46.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:45 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:45.927+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:46.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:45 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:45.927+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:46.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:45 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:45.927+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:46.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:45 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:45.927+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:46.344 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:46 smithi062 systemd[1]: Stopping Ceph osd.7 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:11:46.614 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:46.517+0000 7f9fe5496700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:11:46.614 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:46.517+0000 7f9fe5496700 -1 osd.7 109 *** Got signal Terminated *** 2024-04-25T19:11:46.614 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:46.517+0000 7f9fe5496700 -1 osd.7 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:11:46.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:46.341+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:46.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:46.341+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:46.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:46.341+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:46.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:46.341+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:46.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:46 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:46.329+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:46.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:46 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:46.329+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:46.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:46 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:46.329+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:47.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:46.720+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:47.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:46.720+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:47.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:46.720+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.140007+0000 front 2024-04-25T19:11:17.139827+0000 (oldest deadline 2024-04-25T19:11:41.240016+0000) 2024-04-25T19:11:47.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:46 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:46.780+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:47.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:46 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:46.780+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:47.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:46 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:46.780+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:47.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:46 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:46.780+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:47.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:46 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:46.906+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:47.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:46 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:46.906+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:47.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:46 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:46.906+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:47.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:46 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:46.906+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:47.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:47 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:47.320+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:47.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:47 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:47.320+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:47.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:47 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:47.320+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:47.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:47 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:47.320+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:47.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:47 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:47.337+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:47.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:47 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:47.337+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:47.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:47 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:47.337+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:48.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:47 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:47.675+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:48.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:47 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:47.675+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:48.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:47 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:47.675+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.140007+0000 front 2024-04-25T19:11:17.139827+0000 (oldest deadline 2024-04-25T19:11:41.240016+0000) 2024-04-25T19:11:48.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:47 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:47.675+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:21.240403+0000 front 2024-04-25T19:11:21.240419+0000 (oldest deadline 2024-04-25T19:11:47.641262+0000) 2024-04-25T19:11:48.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:47 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:47.776+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:48.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:47 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:47.776+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:48.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:47 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:47.776+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:48.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:47 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:47.776+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:48.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:47 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:47.887+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:48.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:47 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:47.887+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:48.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:47 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:47.887+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:48.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:47 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:47.887+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:48.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:48 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:48.354+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:48.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:48 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:48.354+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:48.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:48 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:48.354+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:48.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:48 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:48.354+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:48.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:48 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:48.319+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:48.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:48 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:48.319+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:48.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:48 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:48.319+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:49.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:48 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:48.650+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:49.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:48 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:48.650+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:49.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:48 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:48.650+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.140007+0000 front 2024-04-25T19:11:17.139827+0000 (oldest deadline 2024-04-25T19:11:41.240016+0000) 2024-04-25T19:11:49.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:48 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:48.650+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:21.240403+0000 front 2024-04-25T19:11:21.240419+0000 (oldest deadline 2024-04-25T19:11:47.641262+0000) 2024-04-25T19:11:49.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:48 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:48.820+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:49.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:48 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:48.820+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:49.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:48 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:48.820+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:49.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:48 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:48.820+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:49.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:48 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:48.820+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:49.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:48.878+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:49.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:48.878+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:49.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:48.878+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:49.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:48.878+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:49.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:48 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:48.878+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:49.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:49 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:49.305+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:49.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:49 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:49.305+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:49.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:49 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:49.305+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:49.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:49 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:49.305+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.491392+0000 front 2024-04-25T19:11:25.491530+0000 (oldest deadline 2024-04-25T19:11:48.391410+0000) 2024-04-25T19:11:49.866 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:49.632+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:49.866 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:49.632+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:49.866 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:49.632+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.140007+0000 front 2024-04-25T19:11:17.139827+0000 (oldest deadline 2024-04-25T19:11:41.240016+0000) 2024-04-25T19:11:49.866 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:49.632+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:21.240403+0000 front 2024-04-25T19:11:21.240419+0000 (oldest deadline 2024-04-25T19:11:47.641262+0000) 2024-04-25T19:11:49.866 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:49.397+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:49.867 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:49.397+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:49.867 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:49.397+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:49.867 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:49.397+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:50.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:49 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:49.861+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:50.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:49 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:49.861+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:50.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:49 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:49.861+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:50.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:49 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:49.861+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:50.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:49 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:49.861+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:50.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:49 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:49.916+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:50.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:49 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:49.916+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:50.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:49 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:49.916+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:50.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:49 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:49.916+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:50.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:49 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:49.916+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:50.664 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:50 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:50.368+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:50.664 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:50 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:50.368+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:50.664 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:50 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:50.368+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:50.664 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:50 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:50.368+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:50.664 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:50 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:50.368+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:11:50.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:50 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:50.321+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:50.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:50 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:50.321+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:50.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:50 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:50.321+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:50.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:50 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:50.321+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.491392+0000 front 2024-04-25T19:11:25.491530+0000 (oldest deadline 2024-04-25T19:11:48.391410+0000) 2024-04-25T19:11:51.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:50 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:50.662+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.636240+0000 front 2024-04-25T19:10:59.636280+0000 (oldest deadline 2024-04-25T19:11:25.536419+0000) 2024-04-25T19:11:51.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:50 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:50.662+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.938212+0000 front 2024-04-25T19:11:08.938190+0000 (oldest deadline 2024-04-25T19:11:32.438281+0000) 2024-04-25T19:11:51.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:50 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:50.662+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.140007+0000 front 2024-04-25T19:11:17.139827+0000 (oldest deadline 2024-04-25T19:11:41.240016+0000) 2024-04-25T19:11:51.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:50 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7[51351]: 2024-04-25T19:11:50.662+0000 7f9fdda82700 -1 osd.7 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:21.240403+0000 front 2024-04-25T19:11:21.240419+0000 (oldest deadline 2024-04-25T19:11:47.641262+0000) 2024-04-25T19:11:51.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:50 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:50.887+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:51.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:50 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:50.887+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:51.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:50 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:50.887+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:51.185 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:50 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:50.887+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:51.185 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:50 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:50.887+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:51.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:50 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:50.940+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:51.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:50 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:50.940+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:51.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:50 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:50.940+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:51.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:50 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:50.940+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:51.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:50 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:50.940+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:51.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:51 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:51.305+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:51.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:51 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:51.305+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:51.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:51 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:51.305+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:51.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:51 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:51.305+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.491392+0000 front 2024-04-25T19:11:25.491530+0000 (oldest deadline 2024-04-25T19:11:48.391410+0000) 2024-04-25T19:11:51.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:51 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:51.384+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:51.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:51 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:51.384+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:51.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:51 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:51.384+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:51.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:51 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:51.384+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:51.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:51 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:51.384+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:11:52.178 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:51 smithi062 podman[70712]: 2024-04-25 19:11:51.89236368 +0000 UTC m=+5.604515400 container died 5fe7527872ca4a99f55db14b4bf40b2fdd96114a22c83f5c1bb2e263e0573278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:11:52.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:51 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:51.915+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:52.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:51 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:51.915+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:52.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:51 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:51.915+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:52.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:51 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:51.915+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:52.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:51 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:51.915+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:52.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:51 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:51.899+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:52.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:51 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:51.899+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:52.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:51 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:51.899+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:52.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:51 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:51.899+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:52.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:51 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:51.899+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:52.503 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:52 smithi062 podman[70712]: 2024-04-25 19:11:52.275846839 +0000 UTC m=+5.987998558 container cleanup 5fe7527872ca4a99f55db14b4bf40b2fdd96114a22c83f5c1bb2e263e0573278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T19:11:52.503 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:52 smithi062 bash[70712]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7 2024-04-25T19:11:52.503 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:52 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:52.407+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:52.504 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:52 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:52.407+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:52.504 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:52 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:52.407+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:52.504 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:52 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:52.407+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:52.504 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:52 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:52.407+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:11:52.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:52 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:52.295+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:52.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:52 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:52.295+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:52.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:52 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:52.295+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:52.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:52 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:52.295+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.491392+0000 front 2024-04-25T19:11:25.491530+0000 (oldest deadline 2024-04-25T19:11:48.391410+0000) 2024-04-25T19:11:52.826 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:52 smithi062 podman[70732]: 2024-04-25 19:11:52.501988894 +0000 UTC m=+0.604518207 container remove 5fe7527872ca4a99f55db14b4bf40b2fdd96114a22c83f5c1bb2e263e0573278 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:11:53.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:52 smithi062 podman[70819]: 2024-04-25 19:11:52.72868042 +0000 UTC m=+0.018574036 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:11:53.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:52 smithi062 podman[70819]: 2024-04-25 19:11:52.827853258 +0000 UTC m=+0.117746859 container create 870fe76b74f41f5cb57f07b16d1fb31edc8b1b9c16c1530da2836f7a9e3428bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T19:11:53.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:53 smithi062 podman[70819]: 2024-04-25 19:11:53.055659473 +0000 UTC m=+0.345553074 container init 870fe76b74f41f5cb57f07b16d1fb31edc8b1b9c16c1530da2836f7a9e3428bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T19:11:53.114 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:53 smithi062 podman[70819]: 2024-04-25 19:11:53.058942769 +0000 UTC m=+0.348836373 container start 870fe76b74f41f5cb57f07b16d1fb31edc8b1b9c16c1530da2836f7a9e3428bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:11:53.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:52 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:52.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:53.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:52 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:52.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:53.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:52 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:52.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:53.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:52 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:52.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:53.185 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:52 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:52.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:53.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:52 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:52.922+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:53.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:52 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:52.922+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:53.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:52 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:52.922+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:53.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:52 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:52.922+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:53.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:52 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:52.922+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:53.365 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:53 smithi062 podman[70819]: 2024-04-25 19:11:53.141894316 +0000 UTC m=+0.431787921 container attach 870fe76b74f41f5cb57f07b16d1fb31edc8b1b9c16c1530da2836f7a9e3428bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-04-25T19:11:53.626 INFO:journalctl@ceph.osd.7.smithi062.stdout:Apr 25 19:11:53 smithi062 podman[70819]: 2024-04-25 19:11:53.364150341 +0000 UTC m=+0.654043943 container died 870fe76b74f41f5cb57f07b16d1fb31edc8b1b9c16c1530da2836f7a9e3428bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T19:11:53.626 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:53 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:53.410+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:53.626 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:53 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:53.410+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:53.626 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:53 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:53.410+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:53.626 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:53 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:53.410+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:53.626 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:53 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:53.410+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:11:53.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:53 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:53.280+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:53.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:53 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:53.280+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:53.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:53 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:53.280+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:53.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:53 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:53.280+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.491392+0000 front 2024-04-25T19:11:25.491530+0000 (oldest deadline 2024-04-25T19:11:48.391410+0000) 2024-04-25T19:11:53.934 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T19:11:53.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:11:53.977 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T19:11:53.977 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T19:11:53.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.8 2024-04-25T19:11:54.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:53 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:53.912+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:54.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:53 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:53.912+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:54.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:53 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:53.912+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:54.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:53 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:53.912+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:54.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:53 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:53.912+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:54.297 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:54 smithi079 systemd[1]: Stopping Ceph osd.8 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:11:54.297 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:54.000+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:54.297 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:54.000+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:54.297 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:54.000+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:54.298 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:54.000+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:54.298 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:54.000+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:54.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:54.295+0000 7fedabf54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:11:54.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:54.295+0000 7fedabf54700 -1 osd.8 109 *** Got signal Terminated *** 2024-04-25T19:11:54.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:54.295+0000 7fedabf54700 -1 osd.8 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:11:54.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:54.314+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:54.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:54.314+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:54.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:54.314+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:54.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:54 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:54.314+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.491392+0000 front 2024-04-25T19:11:25.491530+0000 (oldest deadline 2024-04-25T19:11:48.391410+0000) 2024-04-25T19:11:54.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:54 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:54.423+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:54.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:54 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:54.423+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:54.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:54 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:54.423+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:54.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:54 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:54.423+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:54.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:54 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:54.423+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:11:55.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:54 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:54.939+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:55.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:54 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:54.939+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:55.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:54 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:54.939+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:55.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:54 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:54.939+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:55.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:54 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:54.939+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:55.319 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:55 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:55.048+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:55.319 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:55 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:55.048+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:55.319 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:55 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:55.048+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:55.319 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:55 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:55.048+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:55.319 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:55 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:55.048+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:55.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:55 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:55.317+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:55.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:55 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:55.317+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:55.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:55 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:55.317+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:55.685 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:55 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:55.317+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.491392+0000 front 2024-04-25T19:11:25.491530+0000 (oldest deadline 2024-04-25T19:11:48.391410+0000) 2024-04-25T19:11:55.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:55 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:55.376+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:55.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:55 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:55.376+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:55.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:55 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:55.376+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:55.865 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:55 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:55.376+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:55.865 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:55 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:55.376+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:11:55.865 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:55 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:55.376+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:11:56.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:55 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:55.946+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:56.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:55 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:55.946+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:56.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:55 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:55.946+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:56.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:55 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:55.946+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:56.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:55 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:55.946+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:56.336 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:56.078+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:56.336 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:56.078+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:56.336 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:56.078+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:56.336 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:56.078+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:56.336 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:56.078+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:56.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:56 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:56.330+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:56.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:56 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:56.330+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:56.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:56 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:56.330+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:56.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:56 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:56.330+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:56.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:56 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:56.330+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:11:56.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:56 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:56.330+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:11:56.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:56.334+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:56.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:56.334+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:56.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:56.334+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:56.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:56.334+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.491392+0000 front 2024-04-25T19:11:25.491530+0000 (oldest deadline 2024-04-25T19:11:48.391410+0000) 2024-04-25T19:11:57.434 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:57.353+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:57.434 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:57.353+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:57.434 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:57.353+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:57.434 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:57.353+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.491392+0000 front 2024-04-25T19:11:25.491530+0000 (oldest deadline 2024-04-25T19:11:48.391410+0000) 2024-04-25T19:11:57.434 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:57.353+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:34.093280+0000 front 2024-04-25T19:11:34.093328+0000 (oldest deadline 2024-04-25T19:11:56.993346+0000) 2024-04-25T19:11:57.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:57.120+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:57.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:57.120+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:57.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:57.120+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:57.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:57.120+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:57.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:57.120+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:57.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:57 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:57.120+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:11:57.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:56 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:56.983+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:57.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:56 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:56.983+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:57.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:56 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:56.983+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:57.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:56 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:56.983+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:57.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:56 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:56.983+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:57 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:57.294+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:57 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:57.294+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:57 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:57.294+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:57 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:57.294+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:57 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:57.294+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:11:57.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:57 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:57.294+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:11:58.434 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:58.322+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:48.582930+0000 front 2024-04-25T19:10:48.582897+0000 (oldest deadline 2024-04-25T19:11:13.282883+0000) 2024-04-25T19:11:58.434 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:58.322+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:04.185956+0000 front 2024-04-25T19:11:04.186151+0000 (oldest deadline 2024-04-25T19:11:33.586700+0000) 2024-04-25T19:11:58.434 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:58.322+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.888923+0000 front 2024-04-25T19:11:16.888795+0000 (oldest deadline 2024-04-25T19:11:40.889321+0000) 2024-04-25T19:11:58.434 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:58.322+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.491392+0000 front 2024-04-25T19:11:25.491530+0000 (oldest deadline 2024-04-25T19:11:48.391410+0000) 2024-04-25T19:11:58.435 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8[51965]: 2024-04-25T19:11:58.322+0000 7feda8548700 -1 osd.8 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:34.093280+0000 front 2024-04-25T19:11:34.093328+0000 (oldest deadline 2024-04-25T19:11:56.993346+0000) 2024-04-25T19:11:58.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:58.156+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:58.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:58.156+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:58.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:58.156+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:58.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:58.156+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:58.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:58.156+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:58.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:58 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:58.156+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:11:58.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.005+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:58.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.005+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:58.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.005+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:58.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.005+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:58.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.005+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:58.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.005+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:31.885364+0000 front 2024-04-25T19:11:31.885296+0000 (oldest deadline 2024-04-25T19:11:57.185361+0000) 2024-04-25T19:11:58.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:58 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:58.250+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:58.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:58 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:58.250+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:58.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:58 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:58.250+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:58.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:58 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:58.250+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:58.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:58 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:58.250+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:11:58.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:58 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:58.250+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:11:59.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.956+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:11:59.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.956+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:11:59.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.956+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:11:59.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.956+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:11:59.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.956+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:11:59.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:58 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:58.956+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:31.885364+0000 front 2024-04-25T19:11:31.885296+0000 (oldest deadline 2024-04-25T19:11:57.185361+0000) 2024-04-25T19:11:59.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:59 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:59.107+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:11:59.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:59 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:59.107+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:11:59.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:59 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:59.107+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:11:59.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:59 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:59.107+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:11:59.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:59 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:59.107+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:11:59.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:11:59 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:11:59.107+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:11:59.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:59 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:59.230+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:11:59.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:59 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:59.230+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:11:59.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:59 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:59.230+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:11:59.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:59 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:59.230+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:11:59.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:59 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:59.230+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:11:59.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:11:59 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:11:59.230+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:11:59.974 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:11:59 smithi079 podman[71576]: 2024-04-25 19:11:59.716361159 +0000 UTC m=+5.548628970 container died 2e6bd85a1f9f15b7012e54296092c080629dbdea8ca45b96efb202c6dbd7ad96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:12:00.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:59 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:59.948+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:12:00.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:59 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:59.948+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:12:00.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:59 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:59.948+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:12:00.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:59 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:59.948+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:12:00.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:59 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:59.948+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:12:00.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:11:59 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:11:59.948+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:31.885364+0000 front 2024-04-25T19:11:31.885296+0000 (oldest deadline 2024-04-25T19:11:57.185361+0000) 2024-04-25T19:12:00.349 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:12:00 smithi079 podman[71576]: 2024-04-25 19:12:00.072132759 +0000 UTC m=+5.904400569 container cleanup 2e6bd85a1f9f15b7012e54296092c080629dbdea8ca45b96efb202c6dbd7ad96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T19:12:00.349 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:12:00 smithi079 bash[71576]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8 2024-04-25T19:12:00.349 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:00 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:00.075+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:00.350 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:00 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:00.075+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:00.350 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:00 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:00.075+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:00.350 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:00 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:00.075+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:00.350 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:00 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:00.075+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:00.350 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:00 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:00.075+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:00.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:00 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:00.211+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:00.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:00 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:00.211+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:00.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:00 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:00.211+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:00.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:00 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:00.211+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:00.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:00 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:00.211+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:00.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:00 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:00.211+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:00.684 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:12:00 smithi079 podman[71593]: 2024-04-25 19:12:00.348430847 +0000 UTC m=+0.626602897 container remove 2e6bd85a1f9f15b7012e54296092c080629dbdea8ca45b96efb202c6dbd7ad96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:12:00.970 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:12:00 smithi079 podman[71689]: 2024-04-25 19:12:00.593900661 +0000 UTC m=+0.017827704 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:12:00.970 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:12:00 smithi079 podman[71689]: 2024-04-25 19:12:00.724578768 +0000 UTC m=+0.148505795 container create a86fb820ab8594b441e88538dcdfa34c50d7f5eef12e9c671100e5f1dd2d7d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T19:12:01.272 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:12:00 smithi079 podman[71689]: 2024-04-25 19:12:00.969071442 +0000 UTC m=+0.392998472 container init a86fb820ab8594b441e88538dcdfa34c50d7f5eef12e9c671100e5f1dd2d7d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T19:12:01.272 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:12:00 smithi079 podman[71689]: 2024-04-25 19:12:00.972483834 +0000 UTC m=+0.396410865 container start a86fb820ab8594b441e88538dcdfa34c50d7f5eef12e9c671100e5f1dd2d7d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T19:12:01.272 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:12:01 smithi079 podman[71689]: 2024-04-25 19:12:01.052271537 +0000 UTC m=+0.476198584 container attach a86fb820ab8594b441e88538dcdfa34c50d7f5eef12e9c671100e5f1dd2d7d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T19:12:01.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:01 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:01.100+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:01.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:01 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:01.100+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:01.272 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:01 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:01.100+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:01.273 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:01 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:01.100+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:01.273 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:01 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:01.100+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:01.273 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:01 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:01.100+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:01.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:00.968+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:12:01.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:00.968+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:12:01.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:00.968+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:12:01.464 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:00.968+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:12:01.465 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:00.968+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:12:01.465 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:00 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:00.968+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:31.885364+0000 front 2024-04-25T19:11:31.885296+0000 (oldest deadline 2024-04-25T19:11:57.185361+0000) 2024-04-25T19:12:01.538 INFO:journalctl@ceph.osd.8.smithi079.stdout:Apr 25 19:12:01 smithi079 podman[71689]: 2024-04-25 19:12:01.270917742 +0000 UTC m=+0.694844761 container died a86fb820ab8594b441e88538dcdfa34c50d7f5eef12e9c671100e5f1dd2d7d08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T19:12:01.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:01 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:01.168+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:01.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:01 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:01.168+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:01.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:01 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:01.168+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:01.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:01 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:01.168+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:01.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:01 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:01.168+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:01.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:01 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:01.168+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:01.933 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T19:12:01.970 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:01.971 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T19:12:01.971 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T19:12:01.971 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.9 2024-04-25T19:12:02.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:02 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:02.129+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:02.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:02 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:02.129+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:02.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:02 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:02.129+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:02.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:02 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:02.129+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:02.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:02 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:02.129+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:02.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:02 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:02.129+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:02.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:01 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:01.944+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:12:02.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:01 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:01.944+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:12:02.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:01 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:01.944+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:12:02.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:01 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:01.944+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:12:02.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:01 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:01.944+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:12:02.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:01 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:01.944+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:31.885364+0000 front 2024-04-25T19:11:31.885296+0000 (oldest deadline 2024-04-25T19:11:57.185361+0000) 2024-04-25T19:12:02.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:02 smithi045 systemd[1]: Stopping Ceph osd.9 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:02.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:02 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:02.210+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:02.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:02 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:02.210+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:02.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:02 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:02.210+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:02.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:02 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:02.210+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:02.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:02 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:02.210+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:02.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:02 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:02.210+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:02.900 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:02 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:02.541+0000 7f273fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:02.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:02 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:02.541+0000 7f273fe00700 -1 osd.9 109 *** Got signal Terminated *** 2024-04-25T19:12:02.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:02 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:02.541+0000 7f273fe00700 -1 osd.9 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:12:02.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:02 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:02.897+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:12:02.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:02 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:02.897+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:12:02.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:02 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:02.897+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:12:02.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:02 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:02.897+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:12:02.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:02 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:02.897+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:12:02.901 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:02 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:02.897+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:31.885364+0000 front 2024-04-25T19:11:31.885296+0000 (oldest deadline 2024-04-25T19:11:57.185361+0000) 2024-04-25T19:12:03.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:03 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:03.121+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:03.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:03 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:03.121+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:03.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:03 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:03.121+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:03.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:03 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:03.121+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:03.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:03 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:03.121+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:03.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:03 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:03.121+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:03.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:03 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:03.165+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:03.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:03 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:03.165+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:03.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:03 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:03.165+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:03.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:03 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:03.165+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:03.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:03 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:03.165+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:03.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:03 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:03.165+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:04.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:03 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:03.874+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:12:04.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:03 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:03.874+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:12:04.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:03 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:03.874+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:12:04.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:03 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:03.874+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:12:04.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:03 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:03.874+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:12:04.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:03 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:03.874+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:31.885364+0000 front 2024-04-25T19:11:31.885296+0000 (oldest deadline 2024-04-25T19:11:57.185361+0000) 2024-04-25T19:12:04.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:03 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:03.874+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:39.486850+0000 front 2024-04-25T19:11:39.486647+0000 (oldest deadline 2024-04-25T19:12:02.986334+0000) 2024-04-25T19:12:04.364 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:04 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:04.156+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:04.364 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:04 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:04.156+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:04.364 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:04 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:04.156+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:04.364 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:04 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:04.156+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:04.364 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:04 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:04.156+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:04.364 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:04 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:04.156+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:04.365 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:04 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:04.156+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:04.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:04 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:04.076+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:04.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:04 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:04.076+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:04.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:04 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:04.076+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:04.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:04 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:04.076+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:04.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:04 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:04.076+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:04.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:04 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:04.076+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:05.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:04 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:04.903+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:12:05.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:04 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:04.903+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:12:05.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:04 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:04.903+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:12:05.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:04 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:04.903+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:12:05.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:04 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:04.903+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:12:05.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:04 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:04.903+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:31.885364+0000 front 2024-04-25T19:11:31.885296+0000 (oldest deadline 2024-04-25T19:11:57.185361+0000) 2024-04-25T19:12:05.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:04 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:04.903+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:39.486850+0000 front 2024-04-25T19:11:39.486647+0000 (oldest deadline 2024-04-25T19:12:02.986334+0000) 2024-04-25T19:12:05.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:05 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:05.092+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:05.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:05 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:05.092+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:05.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:05 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:05.092+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:05.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:05 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:05.092+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:05.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:05 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:05.092+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:05.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:05 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:05.092+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:05.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:05 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:05.191+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:05.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:05 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:05.191+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:05.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:05 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:05.191+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:05.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:05 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:05.191+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:05.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:05 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:05.191+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:05.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:05 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:05.191+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:05.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:05 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:05.191+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:06.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:05 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:05.862+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:12:06.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:05 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:05.862+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:12:06.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:05 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:05.862+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:12:06.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:05 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:05.862+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:12:06.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:05 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:05.862+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:12:06.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:05 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:05.862+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:31.885364+0000 front 2024-04-25T19:11:31.885296+0000 (oldest deadline 2024-04-25T19:11:57.185361+0000) 2024-04-25T19:12:06.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:05 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:05.862+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:39.486850+0000 front 2024-04-25T19:11:39.486647+0000 (oldest deadline 2024-04-25T19:12:02.986334+0000) 2024-04-25T19:12:06.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:06 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:06.084+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:06.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:06 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:06.084+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:06.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:06 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:06.084+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:06.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:06 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:06.084+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:06.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:06 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:06.084+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:06.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:06 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:06.084+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:06.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:06 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:06.174+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:06.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:06 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:06.174+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:06.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:06 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:06.174+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:06.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:06 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:06.174+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:06.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:06 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:06.174+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:06.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:06 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:06.174+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:06.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:06 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:06.174+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:07.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:06 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:06.825+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:52.275397+0000 front 2024-04-25T19:10:52.274991+0000 (oldest deadline 2024-04-25T19:11:15.773942+0000) 2024-04-25T19:12:07.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:06 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:06.825+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:10:59.875996+0000 front 2024-04-25T19:10:59.875939+0000 (oldest deadline 2024-04-25T19:11:24.575951+0000) 2024-04-25T19:12:07.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:06 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:06.825+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:09.179350+0000 front 2024-04-25T19:11:09.178890+0000 (oldest deadline 2024-04-25T19:11:32.078756+0000) 2024-04-25T19:12:07.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:06 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:06.825+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:17.281551+0000 front 2024-04-25T19:11:17.281673+0000 (oldest deadline 2024-04-25T19:11:40.181080+0000) 2024-04-25T19:12:07.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:06 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:06.825+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:23.083075+0000 front 2024-04-25T19:11:23.083136+0000 (oldest deadline 2024-04-25T19:11:48.383322+0000) 2024-04-25T19:12:07.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:06 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:06.825+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:31.885364+0000 front 2024-04-25T19:11:31.885296+0000 (oldest deadline 2024-04-25T19:11:57.185361+0000) 2024-04-25T19:12:07.215 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:06 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9[59927]: 2024-04-25T19:12:06.825+0000 7f2736600700 -1 osd.9 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:39.486850+0000 front 2024-04-25T19:11:39.486647+0000 (oldest deadline 2024-04-25T19:12:02.986334+0000) 2024-04-25T19:12:07.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:07 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:07.089+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:07.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:07 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:07.089+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:07.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:07 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:07.089+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:07.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:07 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:07.089+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:07.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:07 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:07.089+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:07.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:07 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:07.089+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:07.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:07 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:07.177+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:07.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:07 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:07.177+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:07.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:07 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:07.177+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:07.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:07 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:07.177+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:07.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:07 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:07.177+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:07.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:07 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:07.177+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:07.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:07 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:07.177+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:08.425 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:08 smithi045 podman[113180]: 2024-04-25 19:12:08.040558408 +0000 UTC m=+5.620896650 container died e3df6e03419ceaacac4205c44484061b3fe3e307c2ef8dcf3aeb4b1d98c24ee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:12:08.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:08 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:08.042+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:08.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:08 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:08.042+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:08.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:08 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:08.042+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:08.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:08 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:08.042+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:08.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:08 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:08.042+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:08.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:08 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:08.042+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:08.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:08 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:08.227+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:08.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:08 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:08.227+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:08.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:08 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:08.227+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:08.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:08 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:08.227+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:08.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:08 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:08.227+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:08.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:08 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:08.227+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:08.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:08 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:08.227+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:08.714 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:08 smithi045 podman[113180]: 2024-04-25 19:12:08.541092947 +0000 UTC m=+6.121431205 container cleanup e3df6e03419ceaacac4205c44484061b3fe3e307c2ef8dcf3aeb4b1d98c24ee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T19:12:08.714 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:08 smithi045 bash[113180]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9 2024-04-25T19:12:09.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:08 smithi045 podman[113198]: 2024-04-25 19:12:08.742287921 +0000 UTC m=+0.687825587 container remove e3df6e03419ceaacac4205c44484061b3fe3e307c2ef8dcf3aeb4b1d98c24ee2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T19:12:09.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:09.090+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:09.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:09.090+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:09.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:09.090+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:09.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:09.090+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:09.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:09.090+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:09.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:09.090+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:09.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:09.178+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:09.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:09.178+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:09.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:09.178+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:09.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:09.178+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:09.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:09.178+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:09.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:09.178+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:09.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:09 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:09.178+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:09.933 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:09 smithi045 podman[113289]: 2024-04-25 19:12:09.523048488 +0000 UTC m=+0.040701787 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:12:09.933 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:09 smithi045 podman[113289]: 2024-04-25 19:12:09.759798441 +0000 UTC m=+0.277451777 container create 157f3798c560981c3a148e775ba2c2eabdeaea4fc5c617b7aa6a7814e8c1ca8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T19:12:10.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:10 smithi045 podman[113289]: 2024-04-25 19:12:10.038137868 +0000 UTC m=+0.555791227 container init 157f3798c560981c3a148e775ba2c2eabdeaea4fc5c617b7aa6a7814e8c1ca8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T19:12:10.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:10 smithi045 podman[113289]: 2024-04-25 19:12:10.05103899 +0000 UTC m=+0.568692316 container start 157f3798c560981c3a148e775ba2c2eabdeaea4fc5c617b7aa6a7814e8c1ca8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T19:12:10.214 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:10 smithi045 podman[113289]: 2024-04-25 19:12:10.128395299 +0000 UTC m=+0.646048665 container attach 157f3798c560981c3a148e775ba2c2eabdeaea4fc5c617b7aa6a7814e8c1ca8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9-deactivate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T19:12:10.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:10 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:10.047+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:10.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:10 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:10.047+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:10.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:10 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:10.047+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:10.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:10 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:10.047+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:10.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:10 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:10.047+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:10.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:10 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:10.047+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:10.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:10 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:10.223+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:10.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:10 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:10.223+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:10.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:10 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:10.223+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:10.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:10 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:10.223+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:10.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:10 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:10.223+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:10.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:10 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:10.223+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:10.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:10 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:10.223+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:10.888 INFO:journalctl@ceph.osd.9.smithi045.stdout:Apr 25 19:12:10 smithi045 podman[113289]: 2024-04-25 19:12:10.511465245 +0000 UTC m=+1.029118588 container died 157f3798c560981c3a148e775ba2c2eabdeaea4fc5c617b7aa6a7814e8c1ca8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T19:12:11.335 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:11.058+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:11.335 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:11.058+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:11.336 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:11.058+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:11.336 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:11.058+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:11.336 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:11.058+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:11.336 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:11 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:11.058+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:11.362 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T19:12:11.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:11.459 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T19:12:11.459 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T19:12:11.459 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.10 2024-04-25T19:12:11.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:11.266+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:11.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:11.266+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:11.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:11.266+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:11.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:11.266+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:11.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:11.266+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:11.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:11.266+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:11.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:11.266+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:11.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 systemd[1]: Stopping Ceph osd.10 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:12.114 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:11.752+0000 7f597aafb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:12.114 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:11.752+0000 7f597aafb700 -1 osd.10 109 *** Got signal Terminated *** 2024-04-25T19:12:12.114 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:11 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:11.752+0000 7f597aafb700 -1 osd.10 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:12:12.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:12 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:12.038+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:12.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:12 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:12.038+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:12.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:12 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:12.038+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:12.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:12 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:12.039+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:12.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:12 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:12.039+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:12.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:12 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:12.039+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:12.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:12 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:12.039+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:12.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:12 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:12.252+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:12.660 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:12 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:12.252+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:12.660 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:12 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:12.252+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:12.660 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:12 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:12.252+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:12.660 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:12 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:12.252+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:12.660 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:12 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:12.252+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:12.660 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:12 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:12.252+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:13.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:13.014+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:13.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:13.014+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:13.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:13.014+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:13.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:13.014+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:13.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:13.014+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:13.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:13.014+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:13.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:13 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:13.014+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:13.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:13.207+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:13.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:13.207+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:13.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:13.207+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:13.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:13.207+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:13.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:13.207+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:13.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:13.207+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:13.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:13 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:13.207+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:14.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:14.025+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:14.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:14.025+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:14.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:14.025+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:14.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:14.025+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:14.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:14.025+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:14.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:14.025+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:14.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:14 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:14.025+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:14.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:14.218+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:14.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:14.218+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:14.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:14.218+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:14.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:14.218+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:14.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:14.218+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:14.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:14.218+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:14.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:14 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:14.218+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:15.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.002+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:15.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.002+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:15.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.002+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:15.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.002+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:15.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.002+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:15.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.002+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:15.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.002+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:15.516 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:15.175+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:15.516 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:15.175+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:15.516 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:15.175+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:15.516 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:15.175+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:15.516 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:15.175+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:15.516 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:15.175+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:15.516 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:15 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:15.175+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:16.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.992+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:16.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.992+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:16.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.992+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:16.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.992+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:16.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.992+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:16.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.992+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:16.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:15 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:15.992+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:16.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:16 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:16.137+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:50.913150+0000 front 2024-04-25T19:10:50.913223+0000 (oldest deadline 2024-04-25T19:11:15.013105+0000) 2024-04-25T19:12:16.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:16 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:16.137+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:00.215162+0000 front 2024-04-25T19:11:00.215289+0000 (oldest deadline 2024-04-25T19:11:24.915275+0000) 2024-04-25T19:12:16.614 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:16 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:16.137+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:08.416186+0000 front 2024-04-25T19:11:08.416206+0000 (oldest deadline 2024-04-25T19:11:31.316076+0000) 2024-04-25T19:12:16.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:16 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:16.137+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.017211+0000 front 2024-04-25T19:11:16.017247+0000 (oldest deadline 2024-04-25T19:11:41.916942+0000) 2024-04-25T19:12:16.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:16 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:16.137+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:24.118667+0000 front 2024-04-25T19:11:24.118712+0000 (oldest deadline 2024-04-25T19:11:49.418813+0000) 2024-04-25T19:12:16.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:16 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:16.137+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:29.419461+0000 front 2024-04-25T19:11:29.419345+0000 (oldest deadline 2024-04-25T19:11:54.719254+0000) 2024-04-25T19:12:16.615 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:16 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10[58227]: 2024-04-25T19:12:16.137+0000 7f59730e7700 -1 osd.10 109 heartbeat_check: no reply from 172.21.15.45:6822 osd.6 since back 2024-04-25T19:11:41.621613+0000 front 2024-04-25T19:11:41.621736+0000 (oldest deadline 2024-04-25T19:12:03.921223+0000) 2024-04-25T19:12:17.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:17 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:17.010+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:17.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:17 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:17.010+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:17.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:17 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:17.010+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:17.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:17 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:17.010+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:17.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:17 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:17.010+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:17.436 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:17 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:17.010+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:17.436 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:17 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:17.010+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:17.547 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:17 smithi062 podman[70924]: 2024-04-25 19:12:17.239519258 +0000 UTC m=+5.583193166 container died 0fbf5dc1aac6591b0ea94ab3a8ba2517853004db06bc887f1ca2dbf62b709b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-25T19:12:17.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:17 smithi062 podman[70924]: 2024-04-25 19:12:17.628136202 +0000 UTC m=+5.971810108 container cleanup 0fbf5dc1aac6591b0ea94ab3a8ba2517853004db06bc887f1ca2dbf62b709b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T19:12:17.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:17 smithi062 bash[70924]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10 2024-04-25T19:12:18.197 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:17 smithi062 podman[70953]: 2024-04-25 19:12:17.879207001 +0000 UTC m=+0.631587409 container remove 0fbf5dc1aac6591b0ea94ab3a8ba2517853004db06bc887f1ca2dbf62b709b3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T19:12:18.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:18.011+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:18.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:18.011+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:18.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:18.011+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:18.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:18.011+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:18.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:18.011+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:18.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:18.011+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:18.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:18 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:18.011+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:18.500 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:18 smithi062 podman[71044]: 2024-04-25 19:12:18.196537751 +0000 UTC m=+0.108732363 container create ed7387ff635314b89886a0a6a000027b6f5a281f2aa7d184eb2a496ddb9b03b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T19:12:18.500 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:18 smithi062 podman[71044]: 2024-04-25 19:12:18.105918511 +0000 UTC m=+0.018113139 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:12:18.500 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:18 smithi062 podman[71044]: 2024-04-25 19:12:18.42443479 +0000 UTC m=+0.336629413 container init ed7387ff635314b89886a0a6a000027b6f5a281f2aa7d184eb2a496ddb9b03b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:12:18.500 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:18 smithi062 podman[71044]: 2024-04-25 19:12:18.428100171 +0000 UTC m=+0.340294789 container start ed7387ff635314b89886a0a6a000027b6f5a281f2aa7d184eb2a496ddb9b03b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T19:12:18.501 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:18 smithi062 podman[71044]: 2024-04-25 19:12:18.4988574 +0000 UTC m=+0.411052027 container attach ed7387ff635314b89886a0a6a000027b6f5a281f2aa7d184eb2a496ddb9b03b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T19:12:18.864 INFO:journalctl@ceph.osd.10.smithi062.stdout:Apr 25 19:12:18 smithi062 podman[71044]: 2024-04-25 19:12:18.730825946 +0000 UTC m=+0.643020561 container died ed7387ff635314b89886a0a6a000027b6f5a281f2aa7d184eb2a496ddb9b03b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T19:12:19.238 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T19:12:19.280 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:19.280 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T19:12:19.280 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T19:12:19.281 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.11 2024-04-25T19:12:19.306 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.038+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:19.306 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.038+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:19.306 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.038+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:19.306 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.038+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:19.306 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.038+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:19.306 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.038+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:19.306 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.038+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:19.576 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 systemd[1]: Stopping Ceph osd.11 for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:19.577 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.573+0000 7f430bf07700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:19.577 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.573+0000 7f430bf07700 -1 osd.11 109 *** Got signal Terminated *** 2024-04-25T19:12:19.577 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.573+0000 7f430bf07700 -1 osd.11 109 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T19:12:20.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.994+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:20.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.994+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:20.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.994+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:20.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.994+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:20.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.994+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:20.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.994+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:20.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:19 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:19.994+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:21.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:20.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:21.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:20.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:21.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:20.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:21.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:20.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:21.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:20.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:21.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:20.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:21.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:20.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:21.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:20 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:20.964+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6820 osd.8 since back 2024-04-25T19:11:56.622644+0000 front 2024-04-25T19:11:56.622681+0000 (oldest deadline 2024-04-25T19:12:20.722747+0000) 2024-04-25T19:12:22.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.005+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:22.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.005+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:22.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.005+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:22.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.005+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:22.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.005+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:22.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.005+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:22.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.005+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:22.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.005+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6820 osd.8 since back 2024-04-25T19:11:56.622644+0000 front 2024-04-25T19:11:56.622681+0000 (oldest deadline 2024-04-25T19:12:20.722747+0000) 2024-04-25T19:12:23.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.977+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:23.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.977+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:23.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.977+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:23.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.977+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:23.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.977+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:23.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.977+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:23.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.977+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:23.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:22 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:22.977+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6820 osd.8 since back 2024-04-25T19:11:56.622644+0000 front 2024-04-25T19:11:56.622681+0000 (oldest deadline 2024-04-25T19:12:20.722747+0000) 2024-04-25T19:12:24.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:23.999+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6806 osd.0 since back 2024-04-25T19:10:51.610456+0000 front 2024-04-25T19:10:51.610530+0000 (oldest deadline 2024-04-25T19:11:13.310552+0000) 2024-04-25T19:12:24.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:23.999+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6804 osd.1 since back 2024-04-25T19:11:01.214423+0000 front 2024-04-25T19:11:01.214507+0000 (oldest deadline 2024-04-25T19:11:25.314421+0000) 2024-04-25T19:12:24.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:23.999+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6804 osd.2 since back 2024-04-25T19:11:07.015454+0000 front 2024-04-25T19:11:07.015578+0000 (oldest deadline 2024-04-25T19:11:32.915582+0000) 2024-04-25T19:12:24.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:23.999+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.45:6814 osd.3 since back 2024-04-25T19:11:16.316938+0000 front 2024-04-25T19:11:16.317010+0000 (oldest deadline 2024-04-25T19:11:39.816755+0000) 2024-04-25T19:12:24.434 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:23.999+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6812 osd.4 since back 2024-04-25T19:11:25.718055+0000 front 2024-04-25T19:11:25.718130+0000 (oldest deadline 2024-04-25T19:11:48.617919+0000) 2024-04-25T19:12:24.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:23.999+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6812 osd.5 since back 2024-04-25T19:11:32.718925+0000 front 2024-04-25T19:11:32.719010+0000 (oldest deadline 2024-04-25T19:11:56.219043+0000) 2024-04-25T19:12:24.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:23.999+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.62:6820 osd.7 since back 2024-04-25T19:11:50.221942+0000 front 2024-04-25T19:11:50.221821+0000 (oldest deadline 2024-04-25T19:12:11.921672+0000) 2024-04-25T19:12:24.435 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:24 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11[58986]: 2024-04-25T19:12:23.999+0000 7f43044f3700 -1 osd.11 109 heartbeat_check: no reply from 172.21.15.79:6820 osd.8 since back 2024-04-25T19:11:56.622644+0000 front 2024-04-25T19:11:56.622681+0000 (oldest deadline 2024-04-25T19:12:20.722747+0000) 2024-04-25T19:12:25.210 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:24 smithi079 podman[71805]: 2024-04-25 19:12:24.950711506 +0000 UTC m=+5.479158029 container died 8c1c6f37a70cdb43c29979a856bc971c603c4ee3b88d78821fa5d19b1ceee156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:12:25.486 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:25 smithi079 podman[71805]: 2024-04-25 19:12:25.300299377 +0000 UTC m=+5.828745900 container cleanup 8c1c6f37a70cdb43c29979a856bc971c603c4ee3b88d78821fa5d19b1ceee156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T19:12:25.486 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:25 smithi079 bash[71805]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11 2024-04-25T19:12:25.915 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:25 smithi079 podman[71820]: 2024-04-25 19:12:25.484821001 +0000 UTC m=+0.528100167 container remove 8c1c6f37a70cdb43c29979a856bc971c603c4ee3b88d78821fa5d19b1ceee156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T19:12:26.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:25 smithi079 podman[71914]: 2024-04-25 19:12:25.818188862 +0000 UTC m=+0.018190610 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T19:12:26.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:25 smithi079 podman[71914]: 2024-04-25 19:12:25.927686714 +0000 UTC m=+0.127688443 container create 3fb6b503c229a110647e4f73d79050c19d9fae33db1ce6d9f3abb4094ee09082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T19:12:26.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:26 smithi079 podman[71914]: 2024-04-25 19:12:26.147439562 +0000 UTC m=+0.347441306 container init 3fb6b503c229a110647e4f73d79050c19d9fae33db1ce6d9f3abb4094ee09082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:12:26.184 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:26 smithi079 podman[71914]: 2024-04-25 19:12:26.150557319 +0000 UTC m=+0.350559048 container start 3fb6b503c229a110647e4f73d79050c19d9fae33db1ce6d9f3abb4094ee09082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T19:12:26.684 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:26 smithi079 podman[71914]: 2024-04-25 19:12:26.239173166 +0000 UTC m=+0.439174923 container attach 3fb6b503c229a110647e4f73d79050c19d9fae33db1ce6d9f3abb4094ee09082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T19:12:26.684 INFO:journalctl@ceph.osd.11.smithi079.stdout:Apr 25 19:12:26 smithi079 podman[71914]: 2024-04-25 19:12:26.439701736 +0000 UTC m=+0.639703466 container died 3fb6b503c229a110647e4f73d79050c19d9fae33db1ce6d9f3abb4094ee09082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T19:12:27.154 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T19:12:27.231 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:27.231 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T19:12:27.231 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T19:12:27.232 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.a 2024-04-25T19:12:27.714 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:12:27 smithi045 systemd[1]: Stopping Ceph mds.a for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:28.214 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:12:27 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a[67075]: 2024-04-25T19:12:27.808+0000 7fc45be00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:28.215 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:12:27 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a[67075]: 2024-04-25T19:12:27.808+0000 7fc45be00700 -1 mds.a *** got signal Terminated *** 2024-04-25T19:12:29.214 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:12:28 smithi045 podman[113417]: 2024-04-25 19:12:28.954872368 +0000 UTC m=+1.297899126 container died 6f834433d81b2861bef19178f38eaecca216c2af72abd168813163ce5671c5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T19:12:29.510 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:12:29 smithi045 podman[113417]: 2024-04-25 19:12:29.333124746 +0000 UTC m=+1.676151537 container cleanup 6f834433d81b2861bef19178f38eaecca216c2af72abd168813163ce5671c5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T19:12:29.511 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:12:29 smithi045 bash[113417]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a 2024-04-25T19:12:29.964 INFO:journalctl@ceph.mds.a.smithi045.stdout:Apr 25 19:12:29 smithi045 podman[113431]: 2024-04-25 19:12:29.508833581 +0000 UTC m=+0.541083883 container remove 6f834433d81b2861bef19178f38eaecca216c2af72abd168813163ce5671c5da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:12:30.067 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T19:12:30.181 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:30.181 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T19:12:30.181 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T19:12:30.181 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.d 2024-04-25T19:12:30.714 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 19:12:30 smithi045 systemd[1]: Stopping Ceph mds.d for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:31.214 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 19:12:30 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-d[65832]: 2024-04-25T19:12:30.768+0000 7f4592000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:31.214 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 19:12:30 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-d[65832]: 2024-04-25T19:12:30.768+0000 7f4592000700 -1 mds.d *** got signal Terminated *** 2024-04-25T19:12:33.714 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 19:12:33 smithi045 podman[113539]: 2024-04-25 19:12:33.498978493 +0000 UTC m=+2.848184318 container died 571d4c0dbe360ab9f3a68dd2e40dd30297ac44eaef901a81ed93760b891a0f55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T19:12:34.114 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 19:12:33 smithi045 podman[113539]: 2024-04-25 19:12:33.894734255 +0000 UTC m=+3.243940102 container cleanup 571d4c0dbe360ab9f3a68dd2e40dd30297ac44eaef901a81ed93760b891a0f55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T19:12:34.114 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 19:12:33 smithi045 bash[113539]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-d 2024-04-25T19:12:34.463 INFO:journalctl@ceph.mds.d.smithi045.stdout:Apr 25 19:12:34 smithi045 podman[113553]: 2024-04-25 19:12:34.112380941 +0000 UTC m=+0.599596524 container remove 571d4c0dbe360ab9f3a68dd2e40dd30297ac44eaef901a81ed93760b891a0f55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T19:12:34.638 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T19:12:34.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:34.746 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T19:12:34.746 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T19:12:34.746 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.g 2024-04-25T19:12:35.214 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 19:12:34 smithi045 systemd[1]: Stopping Ceph mds.g for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:35.714 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 19:12:35 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-g[66662]: 2024-04-25T19:12:35.421+0000 7fc2d6c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:35.714 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 19:12:35 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-g[66662]: 2024-04-25T19:12:35.421+0000 7fc2d6c00700 -1 mds.g *** got signal Terminated *** 2024-04-25T19:12:38.031 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 19:12:37 smithi045 podman[113660]: 2024-04-25 19:12:37.755069544 +0000 UTC m=+2.465686561 container died 753e25814128eb144bc50ba9d3baaa9827e325e93f52252f5d4577e684449e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T19:12:38.326 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 19:12:38 smithi045 podman[113660]: 2024-04-25 19:12:38.139761016 +0000 UTC m=+2.850378023 container cleanup 753e25814128eb144bc50ba9d3baaa9827e325e93f52252f5d4577e684449e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-04-25T19:12:38.326 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 19:12:38 smithi045 bash[113660]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-g 2024-04-25T19:12:38.714 INFO:journalctl@ceph.mds.g.smithi045.stdout:Apr 25 19:12:38 smithi045 podman[113674]: 2024-04-25 19:12:38.323944649 +0000 UTC m=+0.553639946 container remove 753e25814128eb144bc50ba9d3baaa9827e325e93f52252f5d4577e684449e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T19:12:38.881 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T19:12:38.988 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:38.988 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T19:12:38.988 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T19:12:38.989 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.j 2024-04-25T19:12:39.463 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 19:12:39 smithi045 systemd[1]: Stopping Ceph mds.j for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:39.964 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 19:12:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-j[65420]: 2024-04-25T19:12:39.607+0000 7f400e000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:39.964 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 19:12:39 smithi045 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-j[65420]: 2024-04-25T19:12:39.607+0000 7f400e000700 -1 mds.j *** got signal Terminated *** 2024-04-25T19:12:41.214 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 19:12:40 smithi045 podman[113780]: 2024-04-25 19:12:40.940049657 +0000 UTC m=+1.468035429 container died 9f0469c61ac39a388a1913f761cec7b8e8e080d2c444c71928662ebe6ed7e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:12:41.504 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 19:12:41 smithi045 podman[113780]: 2024-04-25 19:12:41.32638666 +0000 UTC m=+1.854372425 container cleanup 9f0469c61ac39a388a1913f761cec7b8e8e080d2c444c71928662ebe6ed7e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T19:12:41.504 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 19:12:41 smithi045 bash[113780]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-j 2024-04-25T19:12:41.964 INFO:journalctl@ceph.mds.j.smithi045.stdout:Apr 25 19:12:41 smithi045 podman[113794]: 2024-04-25 19:12:41.502138244 +0000 UTC m=+0.542078579 container remove 9f0469c61ac39a388a1913f761cec7b8e8e080d2c444c71928662ebe6ed7e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:12:42.051 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T19:12:42.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:42.162 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T19:12:42.162 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T19:12:42.163 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.b 2024-04-25T19:12:42.463 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 19:12:42 smithi062 systemd[1]: Stopping Ceph mds.b for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:42.864 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 19:12:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b[63904]: 2024-04-25T19:12:42.461+0000 7f02642d2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:42.864 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 19:12:42 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b[63904]: 2024-04-25T19:12:42.461+0000 7f02642d2700 -1 mds.b *** got signal Terminated *** 2024-04-25T19:12:45.147 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 19:12:44 smithi062 podman[71151]: 2024-04-25 19:12:44.882544852 +0000 UTC m=+2.524466950 container died 5d57a1af6b5fdd50cc391ee5486e43b6393708869656f877ed4599dcdd8aed65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T19:12:45.505 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 19:12:45 smithi062 podman[71151]: 2024-04-25 19:12:45.236704563 +0000 UTC m=+2.878626667 container cleanup 5d57a1af6b5fdd50cc391ee5486e43b6393708869656f877ed4599dcdd8aed65 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T19:12:45.505 INFO:journalctl@ceph.mds.b.smithi062.stdout:Apr 25 19:12:45 smithi062 bash[71151]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-b 2024-04-25T19:12:45.615 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T19:12:45.650 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:45.650 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T19:12:45.650 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T19:12:45.651 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.e 2024-04-25T19:12:45.864 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 19:12:45 smithi062 systemd[1]: Stopping Ceph mds.e for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:46.364 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 19:12:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-e[63287]: 2024-04-25T19:12:46.081+0000 7f6a2a83f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:46.364 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 19:12:46 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-e[63287]: 2024-04-25T19:12:46.081+0000 7f6a2a83f700 -1 mds.e *** got signal Terminated *** 2024-04-25T19:12:48.864 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 19:12:48 smithi062 podman[71274]: 2024-04-25 19:12:48.607421053 +0000 UTC m=+2.732843008 container died 8de2efef5e02bfb3f2e9a68068688e7cd0c489dcb0be453de1e116bae53c0b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T19:12:49.134 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 19:12:48 smithi062 podman[71274]: 2024-04-25 19:12:48.940154065 +0000 UTC m=+3.065576019 container cleanup 8de2efef5e02bfb3f2e9a68068688e7cd0c489dcb0be453de1e116bae53c0b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T19:12:49.134 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 19:12:48 smithi062 bash[71274]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-e 2024-04-25T19:12:49.371 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T19:12:49.392 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 19:12:49 smithi062 podman[71300]: 2024-04-25 19:12:49.133286949 +0000 UTC m=+0.526204475 container remove 8de2efef5e02bfb3f2e9a68068688e7cd0c489dcb0be453de1e116bae53c0b81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-e, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:12:49.392 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 19:12:49 smithi062 systemd[1]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-25T19:12:49.392 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 19:12:49 smithi062 systemd[1]: Stopped Ceph mds.e for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T19:12:49.393 INFO:journalctl@ceph.mds.e.smithi062.stdout:Apr 25 19:12:49 smithi062 systemd[1]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.e.service: Consumed 1.113s CPU time. 2024-04-25T19:12:49.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:49.405 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T19:12:49.405 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T19:12:49.406 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.h 2024-04-25T19:12:49.730 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 19:12:49 smithi062 systemd[1]: Stopping Ceph mds.h for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:50.114 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 19:12:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h[63594]: 2024-04-25T19:12:49.728+0000 7f50c5992700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:50.114 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 19:12:49 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h[63594]: 2024-04-25T19:12:49.728+0000 7f50c5992700 -1 mds.h *** got signal Terminated *** 2024-04-25T19:12:53.138 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 19:12:52 smithi062 podman[71408]: 2024-04-25 19:12:52.878967454 +0000 UTC m=+3.251070026 container died fdce5165caa71e416d7a306da4c187319d60a6e3b5b50e71b07f944a362ea2b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-25T19:12:53.596 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 19:12:53 smithi062 podman[71408]: 2024-04-25 19:12:53.344025228 +0000 UTC m=+3.716127783 container cleanup fdce5165caa71e416d7a306da4c187319d60a6e3b5b50e71b07f944a362ea2b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-25T19:12:53.596 INFO:journalctl@ceph.mds.h.smithi062.stdout:Apr 25 19:12:53 smithi062 bash[71408]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-h 2024-04-25T19:12:53.738 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T19:12:53.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:53.771 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T19:12:53.771 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T19:12:53.771 DEBUG:teuthology.orchestra.run.smithi062:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.k 2024-04-25T19:12:54.114 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 19:12:53 smithi062 systemd[1]: Stopping Ceph mds.k for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:54.614 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 19:12:54 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-k[62965]: 2024-04-25T19:12:54.114+0000 7f126f24e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:54.614 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 19:12:54 smithi062 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-k[62965]: 2024-04-25T19:12:54.114+0000 7f126f24e700 -1 mds.k *** got signal Terminated *** 2024-04-25T19:12:56.114 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 19:12:55 smithi062 podman[71527]: 2024-04-25 19:12:55.859355163 +0000 UTC m=+1.871542115 container died 7e4a287086ff13620f40b51bf4921fd3e5bf6b19a1e4812e1ce6f9dce612d9ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:12:56.559 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 19:12:56 smithi062 podman[71527]: 2024-04-25 19:12:56.189684843 +0000 UTC m=+2.201871789 container cleanup 7e4a287086ff13620f40b51bf4921fd3e5bf6b19a1e4812e1ce6f9dce612d9ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T19:12:56.559 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 19:12:56 smithi062 bash[71527]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-k 2024-04-25T19:12:56.559 INFO:journalctl@ceph.mds.k.smithi062.stdout:Apr 25 19:12:56 smithi062 podman[71542]: 2024-04-25 19:12:56.357047476 +0000 UTC m=+0.493980918 container remove 7e4a287086ff13620f40b51bf4921fd3e5bf6b19a1e4812e1ce6f9dce612d9ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T19:12:56.574 DEBUG:teuthology.orchestra.run.smithi062:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T19:12:56.607 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:12:56.608 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T19:12:56.608 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T19:12:56.608 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.c 2024-04-25T19:12:56.934 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:12:56 smithi079 systemd[1]: Stopping Ceph mds.c for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:12:57.193 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:12:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c[63996]: 2024-04-25T19:12:56.991+0000 7eff17fda700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:12:57.193 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:12:56 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c[63996]: 2024-04-25T19:12:56.991+0000 7eff17fda700 -1 mds.c *** got signal Terminated *** 2024-04-25T19:13:01.184 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:13:00 smithi079 podman[72019]: 2024-04-25 19:13:00.872272953 +0000 UTC m=+4.069566213 container died 3a4bf803c7fb359cb854cfca5e2741e8bd85843d5eead41e8048b655e083f91b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T19:13:01.485 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:13:01 smithi079 podman[72019]: 2024-04-25 19:13:01.308060012 +0000 UTC m=+4.505353272 container cleanup 3a4bf803c7fb359cb854cfca5e2741e8bd85843d5eead41e8048b655e083f91b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T19:13:01.486 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:13:01 smithi079 bash[72019]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c 2024-04-25T19:13:01.729 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T19:13:01.749 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:13:01 smithi079 podman[72045]: 2024-04-25 19:13:01.484308694 +0000 UTC m=+0.605623886 container remove 3a4bf803c7fb359cb854cfca5e2741e8bd85843d5eead41e8048b655e083f91b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-c, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:13:01.749 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:13:01 smithi079 systemd[1]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.c.service: Deactivated successfully. 2024-04-25T19:13:01.749 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:13:01 smithi079 systemd[1]: Stopped Ceph mds.c for ea2833ba-0333-11ef-bc93-c7b262605968. 2024-04-25T19:13:01.750 INFO:journalctl@ceph.mds.c.smithi079.stdout:Apr 25 19:13:01 smithi079 systemd[1]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.c.service: Consumed 10.923s CPU time. 2024-04-25T19:13:01.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:13:01.762 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T19:13:01.762 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T19:13:01.763 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.f 2024-04-25T19:13:02.089 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 19:13:01 smithi079 systemd[1]: Stopping Ceph mds.f for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:13:02.434 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 19:13:02 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f[63686]: 2024-04-25T19:13:02.087+0000 7f366988f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:13:02.434 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 19:13:02 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f[63686]: 2024-04-25T19:13:02.087+0000 7f366988f700 -1 mds.f *** got signal Terminated *** 2024-04-25T19:13:05.184 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 19:13:04 smithi079 podman[72152]: 2024-04-25 19:13:04.869659955 +0000 UTC m=+2.887071946 container died 1de7d6a87493e0cc9d27efac7d99bb3aab0250c489b92fa5f0a096b2b767daa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-25T19:13:05.598 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 19:13:05 smithi079 podman[72152]: 2024-04-25 19:13:05.328956003 +0000 UTC m=+3.346368007 container cleanup 1de7d6a87493e0cc9d27efac7d99bb3aab0250c489b92fa5f0a096b2b767daa0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T19:13:05.598 INFO:journalctl@ceph.mds.f.smithi079.stdout:Apr 25 19:13:05 smithi079 bash[72152]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-f 2024-04-25T19:13:05.747 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T19:13:05.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:13:05.779 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T19:13:05.779 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T19:13:05.779 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.i 2024-04-25T19:13:06.184 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:13:05 smithi079 systemd[1]: Stopping Ceph mds.i for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:13:06.185 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:13:06 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i[64306]: 2024-04-25T19:13:06.090+0000 7fda5ecf6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:13:06.185 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:13:06 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i[64306]: 2024-04-25T19:13:06.090+0000 7fda5ecf6700 -1 mds.i *** got signal Terminated *** 2024-04-25T19:13:09.184 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:13:08 smithi079 podman[72274]: 2024-04-25 19:13:08.877422692 +0000 UTC m=+2.889016924 container died d9faa66d057d660b0bd5af454191bd62b67a7b954277e85605116e849e944624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T19:13:09.621 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:13:09 smithi079 podman[72274]: 2024-04-25 19:13:09.20831791 +0000 UTC m=+3.219912144 container cleanup d9faa66d057d660b0bd5af454191bd62b67a7b954277e85605116e849e944624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T19:13:09.621 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:13:09 smithi079 bash[72274]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i 2024-04-25T19:13:09.621 INFO:journalctl@ceph.mds.i.smithi079.stdout:Apr 25 19:13:09 smithi079 podman[72288]: 2024-04-25 19:13:09.426708128 +0000 UTC m=+0.552020313 container remove d9faa66d057d660b0bd5af454191bd62b67a7b954277e85605116e849e944624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T19:13:09.640 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T19:13:09.670 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:13:09.671 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T19:13:09.671 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T19:13:09.671 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.l 2024-04-25T19:13:09.934 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 19:13:09 smithi079 systemd[1]: Stopping Ceph mds.l for ea2833ba-0333-11ef-bc93-c7b262605968... 2024-04-25T19:13:10.434 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 19:13:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-l[63369]: 2024-04-25T19:13:09.987+0000 7fa8445d0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T19:13:10.434 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 19:13:09 smithi079 ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-l[63369]: 2024-04-25T19:13:09.987+0000 7fa8445d0700 -1 mds.l *** got signal Terminated *** 2024-04-25T19:13:13.184 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 19:13:12 smithi079 podman[72394]: 2024-04-25 19:13:12.929512864 +0000 UTC m=+3.047383667 container died b7ee805eec8b7d6560eed48e49d66ff50548c4ce2983e0dc4b6ff9a1225760f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T19:13:13.740 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 19:13:13 smithi079 podman[72394]: 2024-04-25 19:13:13.512915538 +0000 UTC m=+3.630786356 container cleanup b7ee805eec8b7d6560eed48e49d66ff50548c4ce2983e0dc4b6ff9a1225760f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T19:13:13.740 INFO:journalctl@ceph.mds.l.smithi079.stdout:Apr 25 19:13:13 smithi079 bash[72394]: ceph-ea2833ba-0333-11ef-bc93-c7b262605968-mds-l 2024-04-25T19:13:13.860 DEBUG:teuthology.orchestra.run.smithi079:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ea2833ba-0333-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T19:13:13.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T19:13:13.890 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T19:13:13.890 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ea2833ba-0333-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T19:13:14.226 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T19:13:19.362 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ea2833ba-0333-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T19:13:19.564 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T19:13:22.100 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ea2833ba-0333-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T19:13:22.291 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T19:13:24.600 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:13:24.703 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:13:24.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T19:13:24.760 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T19:13:24.762 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615/remote/smithi045/crash 2024-04-25T19:13:24.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/crash -- . 2024-04-25T19:13:24.862 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T19:13:24.862 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-04-25T19:13:24.866 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615/remote/smithi062/crash 2024-04-25T19:13:24.867 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/crash -- . 2024-04-25T19:13:24.908 INFO:teuthology.orchestra.run.smithi062.stderr:tar: /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T19:13:24.909 INFO:teuthology.orchestra.run.smithi062.stderr:tar: Error is not recoverable: exiting now 2024-04-25T19:13:24.910 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615/remote/smithi079/crash 2024-04-25T19:13:24.911 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/crash -- . 2024-04-25T19:13:24.951 INFO:teuthology.orchestra.run.smithi079.stderr:tar: /var/lib/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T19:13:24.952 INFO:teuthology.orchestra.run.smithi079.stderr:tar: Error is not recoverable: exiting now 2024-04-25T19:13:24.953 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T19:13:24.953 DEBUG:teuthology.orchestra.run.smithi045:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T19:13:25.060 INFO:tasks.cephadm:Compressing logs... 2024-04-25T19:13:25.061 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:13:25.065 DEBUG:teuthology.orchestra.run.smithi062:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:13:25.067 DEBUG:teuthology.orchestra.run.smithi079:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T19:13:25.091 INFO:teuthology.orchestra.run.smithi079.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.log 2024-04-25T19:13:25.091 INFO:teuthology.orchestra.run.smithi079.stderr:: No such file or directory 2024-04-25T19:13:25.091 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T19:13:25.092 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T19:13:25.092 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T19:13:25.094 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T19:13:25.094 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T19:13:25.094 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T19:13:25.094 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T19:13:25.094 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T19:13:25.094 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.1% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T19:13:25.108 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T19:13:25.108 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T19:13:25.108 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T19:13:25.108 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T19:13:25.108 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T19:13:25.109 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.1% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T19:13:25.109 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T19:13:25.116 INFO:teuthology.orchestra.run.smithi062.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.log 2024-04-25T19:13:25.116 INFO:teuthology.orchestra.run.smithi062.stderr:: No such file or directory 2024-04-25T19:13:25.116 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T19:13:25.117 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T19:13:25.117 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T19:13:25.117 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T19:13:25.117 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T19:13:25.117 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T19:13:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T19:13:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T19:13:25.118 INFO:teuthology.orchestra.run.smithi062.stderr: 87.1% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.log.gz/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.h.log: 2024-04-25T19:13:25.118 INFO:teuthology.orchestra.run.smithi062.stderr: 90.4% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T19:13:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T19:13:25.118 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T19:13:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.6% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T19:13:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T19:13:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.4% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T19:13:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T19:13:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-volume.log: 92.0% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T19:13:25.119 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T19:13:25.121 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T19:13:25.123 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.1% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T19:13:25.123 INFO:teuthology.orchestra.run.smithi062.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T19:13:25.124 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.log 2024-04-25T19:13:25.124 INFO:teuthology.orchestra.run.smithi045.stderr:: No such file or directory 2024-04-25T19:13:25.124 INFO:teuthology.orchestra.run.smithi062.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T19:13:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T19:13:25.125 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T19:13:25.126 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T19:13:25.128 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T19:13:25.128 INFO:teuthology.orchestra.run.smithi045.stderr: 87.1% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T19:13:25.129 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T19:13:25.130 INFO:teuthology.orchestra.run.smithi045.stderr: 90.3%/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.3.log: -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T19:13:25.130 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T19:13:25.131 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T19:13:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mgr.x.log: 94.4% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T19:13:25.133 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T19:13:25.134 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T19:13:25.135 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T19:13:25.135 INFO:teuthology.orchestra.run.smithi045.stderr: 94.4% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T19:13:25.136 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T19:13:25.137 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T19:13:25.137 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T19:13:25.138 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T19:13:25.140 INFO:teuthology.orchestra.run.smithi062.stderr: 94.4% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T19:13:25.141 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T19:13:25.141 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% 94.4% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T19:13:25.145 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T19:13:25.147 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/cephadm.log: -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T19:13:25.155 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: 90.9% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T19:13:25.157 INFO:teuthology.orchestra.run.smithi045.stderr: 94.2% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T19:13:25.159 INFO:teuthology.orchestra.run.smithi079.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T19:13:25.167 INFO:teuthology.orchestra.run.smithi045.stderr: 93.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T19:13:25.190 INFO:teuthology.orchestra.run.smithi062.stderr: 93.8% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T19:13:25.203 INFO:teuthology.orchestra.run.smithi079.stderr: 94.3% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T19:13:25.466 INFO:teuthology.orchestra.run.smithi079.stderr: 91.7% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T19:13:25.562 INFO:teuthology.orchestra.run.smithi045.stderr: 90.0% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T19:13:25.573 INFO:teuthology.orchestra.run.smithi045.stderr: 93.6% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T19:13:25.753 INFO:teuthology.orchestra.run.smithi062.stderr: 91.6% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T19:13:25.801 INFO:teuthology.orchestra.run.smithi079.stderr: 93.9% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T19:13:26.543 INFO:teuthology.orchestra.run.smithi045.stderr: 90.4% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T19:13:27.270 INFO:teuthology.orchestra.run.smithi062.stderr: 94.1% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T19:13:28.451 INFO:teuthology.orchestra.run.smithi045.stderr: 93.5% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T19:13:28.452 INFO:teuthology.orchestra.run.smithi045.stderr: 93.4% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T19:13:28.489 INFO:teuthology.orchestra.run.smithi079.stderr: 95.8% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T19:13:28.891 INFO:teuthology.orchestra.run.smithi062.stderr: 93.9% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T19:13:29.546 INFO:teuthology.orchestra.run.smithi062.stderr: 93.9% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T19:13:29.824 INFO:teuthology.orchestra.run.smithi045.stderr: 93.7% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T19:13:30.047 INFO:teuthology.orchestra.run.smithi045.stderr: 93.6% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T19:13:30.051 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-25T19:13:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m4.953s 2024-04-25T19:13:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m17.186s 2024-04-25T19:13:30.052 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m1.457s 2024-04-25T19:13:30.085 INFO:teuthology.orchestra.run.smithi062.stderr: 94.1% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T19:13:30.559 INFO:teuthology.orchestra.run.smithi079.stderr: 94.0% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T19:13:30.911 INFO:teuthology.orchestra.run.smithi079.stderr: 93.8% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T19:13:31.419 INFO:teuthology.orchestra.run.smithi079.stderr: 94.0% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T19:13:34.043 INFO:teuthology.orchestra.run.smithi079.stderr: 94.1% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T19:13:34.045 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-25T19:13:34.045 INFO:teuthology.orchestra.run.smithi079.stderr:real 0m8.963s 2024-04-25T19:13:34.045 INFO:teuthology.orchestra.run.smithi079.stderr:user 0m30.033s 2024-04-25T19:13:34.046 INFO:teuthology.orchestra.run.smithi079.stderr:sys 0m1.080s 2024-04-25T19:13:58.128 INFO:teuthology.orchestra.run.smithi062.stderr: 93.3% -- replaced with /var/log/ceph/ea2833ba-0333-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T19:13:58.130 INFO:teuthology.orchestra.run.smithi062.stderr: 2024-04-25T19:13:58.130 INFO:teuthology.orchestra.run.smithi062.stderr:real 0m33.048s 2024-04-25T19:13:58.130 INFO:teuthology.orchestra.run.smithi062.stderr:user 0m47.665s 2024-04-25T19:13:58.131 INFO:teuthology.orchestra.run.smithi062.stderr:sys 0m1.499s 2024-04-25T19:13:58.131 INFO:tasks.cephadm:Archiving logs... 2024-04-25T19:13:58.131 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615/remote/smithi045/log 2024-04-25T19:13:58.132 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:14:00.661 DEBUG:teuthology.misc:Transferring archived files from smithi062:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615/remote/smithi062/log 2024-04-25T19:14:00.662 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:14:07.428 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615/remote/smithi079/log 2024-04-25T19:14:07.429 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T19:14:11.403 INFO:tasks.cephadm:Removing cluster... 2024-04-25T19:14:11.403 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ea2833ba-0333-11ef-bc93-c7b262605968 --force 2024-04-25T19:14:11.743 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T19:14:12.368 DEBUG:teuthology.orchestra.run.smithi062:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ea2833ba-0333-11ef-bc93-c7b262605968 --force 2024-04-25T19:14:12.569 INFO:teuthology.orchestra.run.smithi062.stdout:Deleting cluster with fsid: ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T19:14:12.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ea2833ba-0333-11ef-bc93-c7b262605968 --force 2024-04-25T19:14:13.081 INFO:teuthology.orchestra.run.smithi079.stdout:Deleting cluster with fsid: ea2833ba-0333-11ef-bc93-c7b262605968 2024-04-25T19:14:13.366 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T19:14:13.366 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T19:14:13.427 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T19:14:13.444 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T19:14:13.459 INFO:tasks.cephadm:Teardown complete 2024-04-25T19:14:13.460 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T19:14:13.472 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T19:14:13.472 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:14:13.479 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:14:13.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T19:14:13.630 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T19:14:13.641 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T19:14:13.641 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:14:13.649 DEBUG:teuthology.orchestra.run.smithi062:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:14:13.652 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T19:14:13.665 INFO:teuthology.orchestra.run.smithi062.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:14:13.666 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:14:13.669 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:14:13.669 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-04-25T19:14:13.669 INFO:teuthology.orchestra.run.smithi079.stdout:^* hv01.front.sepia.ceph.com 3 6 377 57 +904us[ +920us] +/- 52ms 2024-04-25T19:14:13.670 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 57 -1498us[-1498us] +/- 75ms 2024-04-25T19:14:13.670 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +187us[ +203us] +/- 85ms 2024-04-25T19:14:13.670 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:14:13.671 INFO:teuthology.orchestra.run.smithi062.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:14:13.671 INFO:teuthology.orchestra.run.smithi062.stdout:=============================================================================== 2024-04-25T19:14:13.671 INFO:teuthology.orchestra.run.smithi062.stdout:^* hv01.front.sepia.ceph.com 3 6 377 58 +885us[ +893us] +/- 52ms 2024-04-25T19:14:13.671 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -1525us[-1517us] +/- 75ms 2024-04-25T19:14:13.671 INFO:teuthology.orchestra.run.smithi062.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +170us[ +170us] +/- 85ms 2024-04-25T19:14:13.671 INFO:teuthology.orchestra.run.smithi062.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:14:13.699 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-04-25T19:14:13.710 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T19:14:13.710 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-04-25T19:14:13.710 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 58 +862us[ +886us] +/- 52ms 2024-04-25T19:14:13.710 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 -1545us[-1521us] +/- 75ms 2024-04-25T19:14:13.710 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 121 +177us[ +198us] +/- 84ms 2024-04-25T19:14:13.710 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T19:14:13.712 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T19:14:13.723 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T19:14:13.723 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T19:14:13.740 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:14:13.963 DEBUG:teuthology.orchestra.run.smithi062:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:14:14.036 DEBUG:teuthology.orchestra.run.smithi079:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T19:14:14.102 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:14:14.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:14:14.200 DEBUG:teuthology.orchestra.run.smithi062:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:14:14.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:14:14.231 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T19:14:14.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:14:14.258 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T19:14:14.270 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T19:14:14.302 INFO:teuthology.task.internal:Duration was 2194.506802 seconds 2024-04-25T19:14:14.303 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T19:14:14.329 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T19:14:14.329 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:14:14.337 DEBUG:teuthology.orchestra.run.smithi062:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:14:14.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T19:14:14.378 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:14:14.381 INFO:teuthology.orchestra.run.smithi062.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:14:14.487 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T19:14:14.814 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T19:14:14.814 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-04-25T19:14:14.815 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:14:14.881 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi062.front.sepia.ceph.com 2024-04-25T19:14:14.881 DEBUG:teuthology.orchestra.run.smithi062:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:14:14.913 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2024-04-25T19:14:14.914 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T19:14:14.936 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T19:14:14.936 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:14:14.944 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:14:14.958 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T19:14:15.031 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T19:14:15.032 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:14:15.037 DEBUG:teuthology.orchestra.run.smithi062:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:14:15.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T19:14:15.799 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T19:14:15.811 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T19:14:15.811 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:14:15.896 DEBUG:teuthology.orchestra.run.smithi062:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:14:15.924 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T19:14:15.951 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T19:14:15.964 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:14:15.972 DEBUG:teuthology.orchestra.run.smithi062:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:14:15.976 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:14:15.999 INFO:teuthology.orchestra.run.smithi062.stdout:kernel.core_pattern = core 2024-04-25T19:14:16.016 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2024-04-25T19:14:16.067 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-04-25T19:14:16.114 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:14:16.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:14:16.168 DEBUG:teuthology.orchestra.run.smithi062:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:14:16.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:14:16.185 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T19:14:16.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T19:14:16.201 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T19:14:16.214 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T19:14:16.214 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615/remote/smithi045 2024-04-25T19:14:16.214 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:14:16.329 DEBUG:teuthology.misc:Transferring archived files from smithi062:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615/remote/smithi062 2024-04-25T19:14:16.329 DEBUG:teuthology.orchestra.run.smithi062:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:14:16.369 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615/remote/smithi079 2024-04-25T19:14:16.369 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T19:14:16.411 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T19:14:16.411 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:14:16.418 DEBUG:teuthology.orchestra.run.smithi062:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:14:16.421 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T19:14:16.473 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T19:14:16.487 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T19:14:16.488 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T19:14:16.501 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T19:14:16.502 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:14:16.509 DEBUG:teuthology.orchestra.run.smithi062:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:14:16.511 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T19:14:16.527 INFO:teuthology.orchestra.run.smithi062.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:14 /home/ubuntu/cephtest 2024-04-25T19:14:16.527 INFO:teuthology.orchestra.run.smithi079.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:14 /home/ubuntu/cephtest 2024-04-25T19:14:16.562 INFO:teuthology.orchestra.run.smithi045.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 19:14 /home/ubuntu/cephtest 2024-04-25T19:14:16.565 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T19:14:16.578 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T19:14:16.611 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T19:14:16.635 DEBUG:teuthology.nuke:shortname: smithi045 2024-04-25T19:14:16.635 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T19:14:16.663 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:29:13.721296', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA7S4l/sU2Ih5/Z1zbLtNc/c+/Km+0K4wCMDSLjjssClO32DmV+u1IWbw4q8J5/0qsU0er7khgk0DL7KjL03R+0='} 2024-04-25T19:14:16.664 DEBUG:teuthology.nuke:shortname: smithi062 2024-04-25T19:14:16.664 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T19:14:16.692 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi062.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:29:13.724195', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCubIc3+aXTOhZVxWOTDsi/fLST+zy6LnXqTd4GJ1DCphaUE3Ulzud8Wx4tqN4hq5beYKND1Xw+I2cyMSZXXfPY='} 2024-04-25T19:14:16.693 DEBUG:teuthology.nuke:shortname: smithi079 2024-04-25T19:14:16.693 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T19:14:16.716 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_17:43:55-fs-wip-lusov-quiescer-distro-default-smithi/7673615', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 18:29:13.722861', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJxJtgLGG5kFmEu3zIZmb+ohE03sGZPu8qUZeFNkYmMQJRQ2kXmeKEN1+52vZSiQPMxtEr4xwC5yC+H7gJ5u4do='} 2024-04-25T19:14:16.725 INFO:teuthology.orchestra.console.smithi045:Power off 2024-04-25T19:14:16.726 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T19:14:16.749 DEBUG:teuthology.orchestra.console.smithi045:power off output: Chassis Power Control: Down/Off 2024-04-25T19:14:16.749 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:16.764 INFO:teuthology.orchestra.console.smithi062:Power off 2024-04-25T19:14:16.765 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T19:14:16.778 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-04-25T19:14:16.787 DEBUG:teuthology.orchestra.console.smithi062:power off output: Chassis Power Control: Down/Off 2024-04-25T19:14:16.787 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:16.803 INFO:teuthology.orchestra.console.smithi079:Power off 2024-04-25T19:14:16.803 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T19:14:16.812 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-25T19:14:16.824 DEBUG:teuthology.orchestra.console.smithi079:power off output: Chassis Power Control: Down/Off 2024-04-25T19:14:16.824 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:16.846 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2024-04-25T19:14:20.779 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:20.812 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:20.846 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:20.890 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-04-25T19:14:20.922 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-25T19:14:20.955 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is on 2024-04-25T19:14:24.894 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:24.923 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:24.956 DEBUG:teuthology.orchestra.console.smithi079:pexpect command: ipmitool -H smithi079.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:25.004 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is on 2024-04-25T19:14:25.031 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is on 2024-04-25T19:14:25.065 DEBUG:teuthology.orchestra.console.smithi079:check power output: Chassis Power is off 2024-04-25T19:14:25.165 INFO:teuthology.orchestra.console.smithi079:Power off completed 2024-04-25T19:14:29.009 DEBUG:teuthology.orchestra.console.smithi045:pexpect command: ipmitool -H smithi045.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:29.032 DEBUG:teuthology.orchestra.console.smithi062:pexpect command: ipmitool -H smithi062.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T19:14:29.120 DEBUG:teuthology.orchestra.console.smithi045:check power output: Chassis Power is off 2024-04-25T19:14:29.141 DEBUG:teuthology.orchestra.console.smithi062:check power output: Chassis Power is off 2024-04-25T19:14:29.221 INFO:teuthology.orchestra.console.smithi045:Power off completed 2024-04-25T19:14:29.242 INFO:teuthology.orchestra.console.smithi062:Power off completed 2024-04-25T19:14:29.347 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 2194.5068023204803 failure_reason: 'error during quiesce thrashing: Error quiescing set ''695e6b8e'': 110 (ETIMEDOUT)' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-25T19:14:29.348 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T19:14:29.424 INFO:teuthology.run:FAIL